Restricted Library Mode

Steve Krouse

on

May 4, 2023

Val Town is a social programming environment to write, run,and share code.

Today we’re announcing a large breaking change: Restricted Library Mode.You’ll still be able to do what you were doing before, but you’ll need to spenda couple minutes upgrading your vals to our new semantics. We’re actually quiteexcited: in addition to a much-needed security upgrade, we think our newsemantics are much clearer and a stronger foundation for us to build on.

We expect 90% of you will have zero or one affected vals. We’re only changingthe semantics of what happens when one user calls another user’s function in ValTown. Nothing is changing if you are only calling your own functions or callingfunctions that refer only to public state. The changes affect functions thataccess private data or mutate state across user accounts.

Due to security concerns, we are rolling out these changes immediately. If youhave vals running in production,please read the upgrade guide to upgrade withminimal downtime. It should only take a couple of minutes.

We apologize for this no-warning rollout. We are committed to maintaining astable and secure platform. Unfortunately those commitments were at oddstoday: advance warning of these change could invite attacks to the veryvulnerabilities we are closing. We had to choose between these two priorities,and we chose security.

We will be all-hands-on-deck over the next couple days to help you all upgradeyour affected vals over to the new semantics. You can reach me by email atsteve@val.town or get help fromme, our team, and our community on Discord.

This post is part security disclosure, part breaking change, and part upgradeguide. They are all very tightly coupled, so we felt a single post made the mostsense.

The old semantics of Val Town were incredibly powerful, but not intuitive noreasy to secure. We refer to those semantics as “API Mode.” When user @A calleduser @B’s function, it was like @A was calling @B’s API. For the executionof @B’s function, @B’s permissions would take over:

the function could access and mutate @B’s private valsaccess @B’s secretsemail @B using console.email

In some ways this was very powerful and intuitive: functions as APIs. However,it was too magical. We estimate that the majority of our users had the wrongmental model about how this worked. Most importantly, API Mode was fundamentallydifficult to secure.

In the old API Mode semantics, when @A called @B’s function, we attempted toadhere to the semantics of both a function call and an API call. This allowed@A and @B to pass each other arbitrary, rich, dynamic JavaScript objects toeach other. For example, user @A could pass user @B a function which,because it was created by user @A, would have user @A’s permissions. It waswild: a whole permission model could be built out of constructing arbitraryfunctions that one could voluntarily pass to other users!

There was a more mundane reason that we needed to support passing rich objectsbetween users: most of our users expected calling a function to act like callinga function, not like calling an API. An earlier version of Val Town tried onlyallowing JSON to be passed between function calls, but that was way too limitingand unexpected. For example, if you called a function that returned even a Setor Response, getting back JSON would be incredibly frustrating.

If you are familiar with the challenges ofsandboxing user code, youmight realize that we set ourselves a herculean task: keeping user secretssandboxed while also allowing them to pass arbitrarily rich computationalobjects between each other. After playing and losing this cat-and-mouse gamewith the fantastic exploit-finder Andrew Healey onetoo many times, we decided to admit defeat and race to a more securablesemantics ASAP. Specifically, we needed semantics that allowed for processisolation and serialization between all user code.

No! Your private data and secrets are secure to the best of our knowledge.

We are confident that our former semantics were not exploited to access yourprivate data. All Val Town executions are meticulously logged. We combed throughthem, and did not find a single instance of a user accessing another user’sprivate data or secrets. It is possible that there are unknown exploits thatwere taken advantage of, but we think this is unlikely.

We were able to find a scheme that allows for virtually all former uses of theplatform to continue to exist, but in a secure way:

If you want to use another person’s function with access to _your ownsecrets, pass those secrets as arguments._If you want to use another person’s function with access to _the otherperson’s secrets, call it via API._

The limitations of this scheme are:

It is a breaking change, so it requires manual upgradesIt prevents the power of passing rich objects between users with theirseparate permissionsIt is more verbose: instead of implicitly referring to secrets (ie@me.secrets.openai), you have to explicitly pass your secrets to functionsthat need them.

We refer to our new semantics as “Restricted Library Mode”, but first you shouldunderstand what we mean by “Library Mode”, and why we needed to restrict itfurther.

“Library Mode” is the normal behavior you’d expect when using code written bysomeone else: the code runs with the permissions of the caller of the function(unlike API Mode, which ran functions with the permissions of the functionauthor).

Library Mode allows foreign code to access and affect the consumer’s privateresources. For example, when I run an npm module on my local computer, thatmodule has access to my ENV and can even read and write arbitrary files on mycomputer. This opens an attack vector: library authors can publish malicious newversions that read the user’s private data and sends it to themselves. Packageecosystems have built up a number of ways to combat this threat, such a versionpinning, package scanning, and reporting. As a four-person startup, we don’thave the resources to detect and combat this. We needed a more restrictivepermission model.

Restricted Library Mode prevents foreign code from accessing secrets and privatevals or mutating vals. The semantics for when @A calls @B’s function:

In case it wasn’t clear: when you create your own vals and call your own code,you have access to all of your own private vals and secrets, and can mutate yourown vals. This remains unchanged. The change to our semantics only applies whenyou call other others’ code from your own: their code can’t access secrets orprivate vals, and if it sends an email, the email goes to you, the caller.

Val Town has moved to Restricted Library Mode, effective immediately.

We moved away from API Mode to provide a more secure sandbox for your secrets.We moved to Restricted Library Mode to preemptively protect your secrets frommalicious library writers. This breaks vals that call other user’s vals thatread or mutate that user’s private state.

For example, this breaks some calls to one of my personal favoritevals:@patrickjm.gpt3. It is a relatively simpler wrapper around GPT3’s API,but we at Val Town decided to “sponsor” this function (up to $2 / day), suchthat anyone could call it for free gpt3 access, without any API key. We intendedit as a fun way to allow folks to play with APIs quickly and easily.

However this function refers to @me.openAiFreeUsageConfig.key, which is a valprivate to @patrickjm. Now any calls to @patrickjm.gpt3 that rely on thiskey (ie don’t pass their own) will fail, because @me.openAiFreeUsageConfigwill return undefined.

We have created a secure method to enable these sorts of use-cases, and we hopeit’s even more intuitive than before!

We are now introducing a new API that provides a secure way for you to exposeyour Val Town functions as APIs. It has API Mode semantics, but it’s secure (andhopefully intuitive), because under the hood it’s a traditional API call.There is process isolation and JSON-serialization between all user’s privatedata.

The v0 API had two routes:

/eval/:code - evaluates code passed in through the URL bar/express/:expressHandler - runs the passed in function as an ExpressJShandler, allowing you to return arbitrary HTTP responses

The v0 API is now deprecated, but will continue to work, under the newRestricted Library Semantics.

The v1 API has three routes:

/v1/eval/:code - evaluates arbitrary code, in Restricted Library Mode, witha simpler return type/v1/run/:user.:val - runs the val as in API Mode/v1/express/:user.:val - runs the val as an Express handler in API Mode

So now when @A wants to call @B’s API, instead of making a function call to@B.foo("bar"), @A makes an API request tohttps://api.val.town/v1/run/@B.foo.

We also made a convenience function, api, which accomplishes the same thingfrom within Val Town: api(@B.foo, "bar").

For the vast majority of the time, when you are calling someone else’s val, youare likely calling it as a normal function, so you’d like the semantics ofRestricted Library Mode. The most popular val (so far) @stevekrouse.fetchJSONis a perfect example! You should never use api when calling it because it’s atrue library function that doesn’t access any state. You should reserve apifor when you are truly using another’s function as an API, which means thatyou’re using state stored in their namespace, or wanting to alert them viaconsole.email.

The new pattern for using your secrets in conjunction with foreign code is topass them as arguments to other’s functions.

This will be safe as long as the function’s author doesn’t decide to bemalicious and change the code to send your secrets to themselves. You openyourself to this attach vector whenever you import someone else’s code (likefrom npm) and don’t pin to a version of their function. We plan to add versionpinning to Val Town in the coming months to improve the security story here aswell. Until then, be wary about passing your secrets as arguments to others’code, like you would about using unpinned npm dependencies. You can getincreased security by forking others’ functions to your account, which is like amanual pinning operation.

Most usage of Val Town will be unaffected by these breaking changes. If you werealready using vals like libraries (not like APIs), you might not need to takeany action.

The easiest and quickest way to find broken call sites is to check if any ofyour vals have stopped working.

One way to do that is to visit your global tracing log:https://www.val.town/settings/evaluationsand look for evaluations marked with an ❌ to indicate that the run threw anerror.

This only works for vals that have already run. You may need to trigger valsto run now to test if they are still working.

You can also find broken call sites by simply reading your code and checking ifthey call someone else’s function, which in turn refers to that person’s privatestate, recursively. This does take time, so it can be faster to just try to runall your running vals and see which throw new errors.

The vast majority of broken function calls can be seamlessly replaced by an APIcall to that user’s val:

await api(@B.foo, "hi") // new

The main gotcha to watch out for here is that where @B.foo function callscould be synchronous or asynchronous, api(@B.foo) calls are alwaysasynchronous, so you’ll need to await them where appropriate, just like normalAPI calls.

The api function is simply calling the API under the hood:

async function api(valName, ...args) {
  const response = await fetch(`https://api.val.town/v1/run/${valName}`, {
    body: JSON.stringify(args),
  const data = await response.json();

Side note: you may notice that there’s something suspicious about the valNameparameter, because we actually accept the val’s name without you needing to passit as a string, ie @B.foo, not "@B.foo". This wouldn’t be possible in normalJavaScript but we own the runtime, so we can do compiler transformations likethis internally.

If you are using Val Town’s v0 API to call functions that mutate your state oraccess your private state, you will need to upgrade to the v1 API:

https://api.val.town/eval/@B.foo("hi")
https://api.val.town/v1/run/B.foo?args=["hi"]
https://api.val.town/express/@B.foo
https://api.val.town/v1/express/B.foo

Note that the new API removes the @ symbol and has a totally new scheme forpassing arguments.Please refer to our new API docs for the full scheme.It’s a bit less intuitive but simpler and more expressive.

We’re here to help! Please reach out over the next day or two if you have anyquestions or concerns about these new semantics. You can reach me by email atsteve@val.town or get help fromme, our team, and our community on Discord.