JavaScript Code Transformation for a Niche Environment

I’ve developed a build tool just for hackmud that can take in regular JavaScript/TypeScript and output code made to run in hackmud, an online multiplayer game where you find and solve puzzles for the collection of resources and power. You don’t solve the same puzzles over and over manually however. After you’ve solved a puzzle a few times you can start to understand it enough to write scripts and bots to solve it for you.

The most interesting part of hackmud to me is not its gameplay, but instead its scripting environment. Scripts are written in JavaScript and they are uploaded to the game’s servers. Other people can run them so there are some interesting limitations for both preventing you from gaining control of the game’s servers and other player’s stuff (e.g. viewing their source code on their scripts).

An example of one of these interesting limitations is that although code is run using an modern up-to-date engine, V8 (the JavaScript engine found in Chrome), the syntax is limited to ES2015. This is thanks to Esprima, an old JavaScript parser that’s used for validating the scripts uploaded to stop people from uploading nonsense code that can break the sandbox. This means we may only partially have access to a JavaScript feature or we have to access it in a roundabout way, for example we can’t use BigInt literals (3n) so instead we have to call the BigInt() constructor.

Another limitation is the game’s non-standard syntax. Scripts are a singular function expression which is invalid JavaScript and there is also preprocessor syntax (e.g. #fs.foo.bar()). Trying to use an IDE to aid you in writing scripts can be frustrating since it’ll be stuck on saying you forgot to name the function or that private identifiers are not allowed in this context.

A final limitation is script space. At first you’ll only be allowed to upload a single script of 500 chars. Although this can be extended by playing the game, it does mean this build tool should probably also minify scripts.

2 years ago I started a project called Hackmud Script Manager. This project was originally based on an old project called hackmud_env-tools (now lost to time) which itself was just the tools pulled out from Snazzah’s project hackmud_env. Before this, I’d also developed a tool myself (which is even more lost to time) but I scrapped it when Snazzah released hackmud_env since mine was lacking features in comparison.

Hackmud Script Manager is a tool that can take in JavaScript or TypeScript code and spit out ES2015 code. It also supports minification, nicer global variables, general fixes, and even importing NPM packages.

Here’s How That Process Works

I have a function called processScript(). This function receives source code, and one of the first things it does is hand it off to Rollup with a few plugins. One of these plugins is a custom one that converts the hackmud preprocessor syntax into valid JavaScript syntax.

This is done in a function called preprocess(). It mainly does this via brute force using Babel to continually attempt to parse the code catching SyntaxErrors and replacing the invalid code with valid code until eventually, the script parses successfully. This process used to be done by a faster but dumber series of .replace()s but a downside was that it was overaggressive and replaced code it shouldn’t have.

The babel plugin is used for converting modern syntax to ES2015 syntax. And a few other plugins are used for importing packages in scripts. Rollup then spits out a bundled up module and it is transform()’s job to turn that into a single function expression. transform() finds and moves everything at the top level (except the default export) into a newly created (currently parentless) block statement. The default export acts as the main function (the function expression itself). We then iterate through the body of this parentless block statement in reverse.

When we find a declaration, we check if it is referenced outside of the block statement, if so we replace the declaration with an assignment to a property on the per script global object ($G) as well as replace references to the variable with references to that property. Then at the end, we insert the block statement with a guard that’ll make sure the block will only run once.

hackmud is missing this due to legacy security reasons and trying to transform references to this with something equivalent is very difficult. Currently, Hackmud Script Manager only replaces references to this with something equivalent in classes. In the constructor super() returns the this value so we save that to a variable and replace references to this with references to the variable. In the class methods, assuming it hasn’t been overridden, super.valueOf() returns the this value so we do the same with that. At the moment all other references to this are just replaced with undefined although this is temporary until I find workarounds for those cases as well.

If there were exports (excluding the default one), a return statement returning an object of the exports is appended to the end of the body of the main function. We also replace BigInt literals with calls to BigInt().

Next is the minification step which is handled by minify(). hackmud has some custom global constants which have aliases that are shorter so it swaps out those for the shorter ones (e.g. _TIMEOUT to _TO). We also create aliases for globals and replace references to the globals with the created aliases so the name can be shortened. Next is where we can potentially cut down a large amount on the character count. For some reason, hackmud doesn’t count comments towards the character count and scripts also have a way of accessing their own source code (#fs.scripts.quine()). This combined means we can serialize a lot of data in our script to JSON at compile time and then read it back with JSON.parse() at runtime. And there’s no measurable slowdown that I could find. This may be because JSON is simpler to parse than JavaScript so, in theory, this might even save time. Data that can be stored as JSON includes strings, numbers, and global objects and arrays. If we turn dot notation into index notation (foo.bar to foo["bar"]), we can even save and store the names of properties to the JSON comment (the same goes towards object keys as well). We can also transform template strings into regular string concatenation and do the same with that. After that, we hand off the code to terser to make the general non-hackmud-specific optimisations.

Then we do some postprocessing to undo the preprocessing and regenerate hackmud’s preprocessor syntax and we are done.