What JavaScript Programmers Need to Know about Transpilers
Want to keep up with ECMAScript without leaving behind the browsers that don’t have the newest JavaScript features yet? Or experiment with upcoming features before they make it into the standard so you can tell the ECMAScript what works for you as a developer and what doesn’t? Or just take advantage of tools that make JavaScript more efficient for large projects? There are transpilers that can help you with all that.
Transpilers, which convert code in one language to another, used to be more about alternative programming languages like CoffeeScript, ClojureScript and Elm, or even using languages like C and C++, like Emscripten, which turns LLVM bytecode into asm.js code. That’s not about replacing JavaScript, points out Dave Herman, Mozilla’s director of strategy and research; “multiple programming models for the Web can happily coexist and even provide healthy competition and cross-pollination of ideas.”
Extending JavaScript
Similarly, he views transpilers like TypeScript, PureScript, Flow and JSX that add custom extensions to JavaScript as “great for the Web.”
TypeScript is a superset of JavaScript with optional static types, plus tooling to make that efficient to write, with refactoring as well as detecting errors from typos in method names to operations that won’t work because the type is wrong. You can experiment with type safety in JavaScript that stays as human-readable JavaScript, without getting locked into alternative languages like Dart and CoffeeScript.
When Babylon.js was rewritten in TypeScript, David Catuhe pointed out “developers that use Babylon.js will not be able to see the difference between the previous version developed with JavaScript and the new version developed using TypeScript.” He also noted that porting to TypeScript helped him find many small bugs that had been in the code all along.
“Having a transpiler means that developers are exposed to the newer features and APIs as they are written, which can help the community as a whole”–Henry Zhu.
And for large teams writing a lot of code, those benefits can be a huge productivity boost. That’s what Microsoft was looking for when the TypeScript project started in 2011. The Office Online web apps had more than a million lines of code, “and back then there weren’t a lot of tools you could use to build apps like that,” former program manager for the TypeScript team Jonathan Turner told us. The plan was to get better JavaScript code using static types supported by the powerful developer tools Microsoft developers were used to for other languages.
As well TypeScript support in VS Code and Visual Studio, there are TypeScript plugins for Sublime, Emacs and Vim, plus support in an increasing number of tools. The transpiler has been picked up by projects like Angular, Asana and Dojo, and Mozilla’s Flash replacement, Shumway, as well as the Babylon.js WebGL framework and the vorlon.js remote JavaScript debugging tools.
Inside Microsoft, TypeScript is used by Bing, Visual Studio and Visual Studio Online, Azure and the Xbox team inside Microsoft, but it’s also in use at companies ranging from Adobe, Google, Palantir, Progress (for NativeScript) and SitePen to the eBay Classifieds Group.
As well as extending JavaScript, TypeScript also transpiles your code to match multiple ECMAScript standards, which gives you a way to support multiple browsers with less effort, and to try out proposed ECMAScript standards early on.
This feature is also offered by the open source Babel transpiler, another transpiler for JavaScript.
“Transpilers allow developers to write future facing code, even though the current version of the language isn’t supported in all environments,” explains Henry Zhu from the Babel core team. “For example, if you are supporting Internet Explorer, which doesn’t have any ES2015 features, you would have to transpile to do so because IE doesn’t know about the newer syntax. Babel is that layer that allows you not to have to think about what browser you are using and specify which features you need to be transpiled. Browsers take time to implement the spec, and they do so incrementally. If there isn’t an auto-updating feature, then users may never update their JavaScript version, so the only way to write new versions of JavaScript is to transpile.”
Like TypeScript, Babel is useful beyond just that transpilation, Zhu notes. “Babel is a general purpose tool for your own Javascript transformations. It’s not just about transpiling ES6 to ES5.” There are more than a thousand plugins to extend Babel; “People are writing plugins for specific libraries, tools for things like linting, optimizations for browsers, and minification.”
Standards at Scale
Plus, said Zhu, “Having a transpiler means that developers are exposed to the newer features and APIs as they are written, which can help the community as a whole.”
“The spec creators will be able to get feedback on proposals during the TC-39 stage process from stage-0 to stage-4 if someone writes a Babel plugin for it,” Zhu said. “Because there is such a wide user base, it allows a lot of users to try out experimental features which can help mold the feature into a better one than if it was just approved by the language authors without much ‘real world’ testing. Because many of the proposals are on Github, anyone can provide input on the future of the proposal as it is moving along.”
Herman is enthusiastic about what he calls “The adoption at scale of transpiled standards-track technology, in particular with the success of Babel. For developers, the immediate appeal is getting to take advantage of improvements to JavaScript even before engines (in browsers or Node.js) support them natively. And because the features are standards-based, developers can adopt them without fear of major incompatible changes. It’s hard to overstate the value of that to developers in the rapidly evolving JavaScript ecosystem.”
Brian Terlson, an editor of the ECMAScript standard, and a senior program manager on the Microsoft Edge team, agrees. “Transpilers are hugely important. JavaScript programmers generally want to use the latest features. Catering to the lowest common denominator is miserable, and no one wants to do it. What transpilers let you do is write code in that fancy new syntax that you love, that makes you productive, that makes your app maintainable – and compile it down to something that runs on old crusty browsers you wish didn’t exist in the marketplace but unfortunately do. Transpilers have been transformative in how the JavaScript community writes code.”
That early use and feedback from developers lead to a virtuous cycle, Herman says. “Transpilers have unleashed a surge of early adoption and community experimentation of new features. They give Web developers the ability to try features out in real, production apps, and they give them control over how often and when they want to upgrade to latest versions of the features. And that means more Web developers are participating in earlier vetting of standards-track features, which gives them a stronger voice in the standardization process and ultimately leads to better standards.”
“Thanks to transpilers, features from future editions are continuing to get lots of early adoption and experimentation. Decorators make it possible to abstract common patterns in class definitions and are popular with Web frameworks like Angular, Ember, and React,” Herman said. The Ember.js community was an early adopter of Babel and Herman says that led to lots of usability feedback on the module system that went into ECMAScript 2015.
Feedback also helped the standardization of decorators, said Terlson. “Features that get implemented early in transpilers can be really big, compelling features, like decorators; that can be instrumental in iterating on the design of those features.”
“If there’s a feature you know is really going to improve the code you write and the app you’re working on,” he suggested, “just pick it up in a transpiler or a polyfill and use it, and give us feedback on it.”
New Features, Faster
Transpilers are one way to handle the chicken and egg problem that new features can’t go into ECMAScript until they’ve been implemented. But browser vendors are reluctant to implement features that haven’t yet been standardized not least because that can lead to developers not keeping up with the spec of a feature that changes as it goes through the standards process.
ECMAScript 2015 didn’t require previous implementations; “as a result,” explained Terlson,” after we had ratified certain features like proxies there were issues that the implementers came across that just weren’t reflected in the spec, so we had to make changes after the fact. That highlights the importance of making sure a feature can be implemented as specified before ratifying the standard.” There’s a similar issue with tail-call optimization, and it may not be a coincidence that Zhu notes those are both features that couldn’t be tried out in transpilers.
The maintainers of languages need feedback from programmers, before a new version of a language is fully baked. Transpilers are a big part of that, Terlson believed. “Transpilers are helping us get feedback on syntax. We’re really fortunate to have tools like Babel and TypeScript that can let us experiment with syntax before we’ve done browser implementations. For certain features, we can be pretty convinced that they’re going to work if we can get a transpiler or polyfill implementation, plus a browser.”
Transpilers can develop new features faster than browsers, Herman pointed out, “Babel is implemented in JavaScript, whereas browsers are implemented in C++, so functionality is much easier to engineer. Some features may have trickier challenges for integrating with an entire browser. JavaScript engines all have sophisticated, multi-tier just-in-time (JIT) compilation architectures, which can often mean a single feature needs to be implemented multiple times, one for each tier. And browser engine development teams have many more responsibilities than just implementing JavaScript features, so they have to balance their priorities.”
Transpilers can’t give you all the new features, Herman pointed out, “Some features, like ECMAScript 2015’s proxies, or the current SharedArrayBuffer proposal, are essentially impossible to implement with a transpiler. Others, like ECMAScript 2015’s symbols, can be partially implemented but with some known limitations. This latter category requires some care on the part of developers, who have to be sure not to depend on the behavior that the transpiler is unable to implement correctly.”
Transpilers also don’t insulate you from changes in JavaScript as the ECMAScript standard develops, either. “There’s a caveat,” Terlson warned; “We will listen to feedback from developers who are using features in transpilers, and it’s possible the spec will shift because of that. We might make breaking changes to a specification before it ships so we do recommend caution when you’re using features in advance of the standard.”
But even then, they can help you transition, Herman said. “When it does come time to upgrade to a new version of a transpiler, having it break your code because of incompatible changes to experimental language features can still be frustrating and time-consuming. For that reason, transpilers like Babel allow you to set your level of tolerance for instability. If you choose to use features that are more experimental, you get the enjoy the bleeding edge of new functionality, but you also might have to deal with more churn. Alternatively, you can choose more conservative settings to reduce the risk of incompatible changes, while restricting yourself to the smaller set of more stable language features.”
