448: Next Gen Bundlers with Jason Miller and Fred Schott
This week we talk with the maintainers of Snowpack and WMR, two newish flavors bundlers, to discuss how ES Modules change the game for modern JavaScript development.
Guests
Time Jump Links
Transcript
[Banjo music]
MANTRA: Just Build Websites!
Dave Rupert: Hey there, Shop-o-maniacs. You're listening to another episode of the ShopTalk Show. I'm Dave--bundle of fun--Rupert and with me is Chris--packed up and ready to go--Coyier. [Laughter]
Chris Coyier: Yeah.
Dave: I guess it could be the other way around because I'm packing up my office.
Chris: Giggly modules square. Yeah.
Dave: Yeah.
Chris: Mm-hmm.
Dave: Hey. So, Chris, what's hot in reloading these days? Hey... [Laughter]
Chris: [Laughter]
Dave: Any--
Chris: Anything. Anything. Dave is trying to allude that we're going to do a show that's going to be all about -- I'm tempted to call it Next Gen Bundlers, but I don't even know if that's appropriate because even the word bundling is a little loaded, perhaps. So, we're going to get to all that.
We have, I think, the two most perfect possible people to be talking about this who are at the frontlines of whatever a next-gen bundler is. [Laughter] So, one of those people is Jason Miller from Google. You've been on the show before. Hey, Jason.
Jason Miller: Hi. [Laughter]
Chris: Hey. Jason is Jason Miller, Google DevRel, PReact, code golfing and, most recently--I'm sure you do lots of interesting stuff, Jason, but--I listened to another podcast (Tools Day) that had you on and you were talking all about WMR, which is one of those fake acronyms, right? It doesn't actually mean anything.
Jason: What does it mean, really?
Chris: Nobody knows, but it is kind of a -- you might use it instead of Webpack kind of thing. Is that fair?
Jason: Uh, yeah. Sure. [Laughter]
Chris: Okay. It's got a bunch of stuff you might use. We'll cover that. Don't worry about that.
We have Fred K. Schott. Did I say that last name right?
Fred Schott: Perfect.
Chris: Fred Schott. Fred, you have some big projects, one of which is Snowpack. The other is Skypack. They're both very related to each other. They are an evolution of another thing you did called Pika. Pika, R.I.P. - forever lives.
Fred: Pika lives on into the future.
Chris: Okay. Great.
Dave: Did you get sued by Nintendo? Is that the story?
Fred: [Laughter] That's not-not a reason [laughter] why I renamed.
[Laughter]
Dave: I was curious. I was immediately like, "Nintendo is really super litigious," so possibly.
Fred: I thought I was covered. I thought "pika," the word in the dictionary, is a little tiny mouse. I had a little tiny mouse logo.
Dave: Mm-hmm.
Fred: Then like a year into the project, someone is like, "You know it's pronounced pike-a, right? Like peeka is just the Nintendo thing." I thought I was covered. I wasn't. Luckily--
Dave: Yeah.
Chris: Whoa!
Dave: As a Japanologist, I would say peeka is a sound effect, so peeka-peeka means kind of vibrant and electrifying.
Fred: Oh!
Dave: Which is why Pikachu is electric rat.
Jason: Oh, interesting.
Fred: It was also a dirty word in like three different languages, I learned, after naming that, so for a lot of reasons. That was the other half of it.
Chris: Mm-hmm.
Dave: That goes into my naming things is hard list.
Fred: Yes.
Jason: Yeah.
Dave: [Laughter] Is it already being used? Is it a curse word in some other language?
Fred: I've learned, the snappier it sounds, the more likely someone has used it in an inappropriate way to get back at someone.
Dave: That's true.
Fred: Through history.
Dave: Yeah. I found out Dave is Deibu when you go into Japanese and that means fat and ugly.
Chris: [Loud gasp]
[Laughter]
Dave: I found that out when I moved to Japan.
Fred: It can happen to anyone. It can happen to anyone.
Dave: Yeah, yeah. As a larger fella in Japan, it was not missed, I guess, by the children.
Chris: I heard Chris means big, strong Canadian warrior. I don't know. It's just what I heard.
Jason: Only in Canada.
Chris: [Laughter] Okay, so these two things are related and I don't know how exactly, but I've decided that they are.
Dave: [Laughter] Forever tied together.
Chris: I'm going to take one. I'm going to throw a dart and I'm going to say the reason that they're related is, well, it's obviously JavaScript but it's a specific thing about JavaScript, which I'm just going to say right now: ES Modules or ESM. Then I want to talk about, is that term right? I'm sure everybody here knows what I talk about, but is that the right way to refer to it even? I feel like it's so complex.
Fred: It's definitely the technology that I think all of us are circling around. It's like, oh, this is going to change things. But it's very much the foundation and then what do you do with it, I think, is what everyone is exploring right now.
Chris: Yeah. I've heard it called ES6 modules, but ES means ECMAScript, and sometimes we use that, and sometimes we just don't.
Jason: Sometimes, I refer to these as JS Modules.
Chris: JS Modules, I like that better.
Jason: Yeah, partly because pedantic people have told me that that's more correct. I don't know why. But also, I find it interesting that it brings up people who are pedantic about it being called ES Modules. I don't have fuel for either argument. I just find it funny that there is one. [Laughter]
Chris: I just don't even get it. I get ECMAScript is the spec or something, but it's like we never say that, so why would we just only say it in this one circumstance?
Fred: Yeah. I think ESM, as an acronym, caught on and now that's the baggage that we carry. But in a world where that's existed for years, it probably just becomes native modules, JavaScript modules.
Chris: One more. I'm going to call it native import and export because I think of it as the actual browsers actually understand what these statements are in a way that feels kind of new. While we didn't define it, we already got into the pedantic of it a little bit.
[Laughter]
Chris: What it means to me is that a JavaScript file that a literal browser is going to look at and read, if you write import x from x.js, that's like a relative file path at the end and it's actually going to understand what to do with that and import it. There are things that can go wrong and all that, but it's just like any other JavaScript API. It just works in the browser.
It's not even particularly new, is it? Does somebody have a handle on when that dropped? Years ago.
Fred: 2015 was the spec, I think.
Chris: Yeah.
Fred: So, 2016, 2015, around then, yeah. Developers have been using this for many, many years now, so it's one of those things where you hear, "What is ESM?" a lot.
Chris: Yeah.
Fred: But really, it's just that import/export, which, "Oh, yeah. I know that." A lot more people use it than know that that's what the spec and history behind it come from.
Dave: But when did it show up in Safari, I guess, is the...?
[Laughter]
Chris: Yeah.
Dave: (Indiscernible)
Jason: Yeah. Uh... Safari 10.3?
Fred: Yeah. They all, I think, started showing up like 2018. I think Firefox was last added in 2018, so even now we're kind of getting into the world where IE11 is really the only browser that you kind of need to worry about that doesn't have the support - for most people. Microsoft is even pretty much doing what they can to push that into the past, so we're getting close to a world where every browser supports it.
Dave: There is also kind of a Node story, right? Maybe I'm jumping too far ahead, but Node had this require syntax and it supports the import syntax now, but that was sort of a deal to get into Node itself. But that's as of whatever.
Chris: Why is it any different? I don't get why it has to be different.
Jason: Hundreds of thousands of NPM modules that are just expected to work out of the box.
Chris: But why is the syntax weird? I understand that the code that exists on NPM already, but why can't Node itself just be like, "Oh, yeah. Import, we'll just make that work now"? I'm sure it's more complicated. I just don't get why.
Jason: I think one of the things that doesn't get talked about much is that there was one modules working group for Node and obviously they were tasked with bringing the import/export syntax to Node natively, but part of that was aligning module loading in Node with module loading in JavaScript (a language) now that JavaScript (a language) offered it.
Even the ECMAScript spec doesn't define all of module loading. They just define a semantics for what kind of gets executed after modules are loaded. But it would have been, to some extent, weird and, to a possible extent, a spec violation if Node just grafted its module loading semantics onto ES Module syntax. Then we probably would have ended up in a world where you'd have the syntax but that would be the entirety of what you could assume would be supported and everything that falls inside of the specifier string of what module to go and load is just totally implementation-dependent and you could not count on anything working.
That working group had to figure out, can Node do spec compliance? Is there a way for us to take Node there? Obviously, we can't just magically make it happen because it's scanning the file system and stuff. But yeah, and it kind of got wrapped into one whole thing.
Chris: They did it, though, to their credit. Right? It has arrived and now import and export statements, you can write in Node, and people are like, "Yay! Good job," and doing it and it feels like the future of NPM, as well as the future of JavaScript in the browser. That feels celebratory to me, like, "Whoa! Sweet!"
Jason: It was an immense effort.
Fred: Yeah, for many years.
Jason: I remember talking to Guy Bedford two years ago at a conference and he was nervous because they were about to ship, I think it was, Node 12.x, whatever the first 12.x release was, and he was trying to figure out, "Can we land ES Module syntax with export maps or without export maps?" Just to see him and others put all those pieces into place and actually just have it become a thing that people can start using. I guess I kind of assumed that there would be roadblocks that just would end up killing everything and I'm very happy that that wasn't the case.
[Laughter]
Fred: I think it did kind of stop/start a lot. This isn't like, "Oh, how can we add this one feature to Node?" It's like, how can we change the underlying module system, the thing that everything hooks into at a certain point? That took many years of effort to get right.
Someone else -- I heard someone say it's, in a way, the largest software migration maybe ever (if you think of the entire NPM ecosystem).
Chris: Hmm. Right.
Fred: Slowly moving from one low-level thing to one other low-level thing, it's a pretty huge undertaking.
Dave: No big deal.
Fred: Yeah. [Laughter]
Chris: It's cool to hear the authors, like the largest NPM authors being like, "I'm going to convert all my stuff. Here we go. I'm going to do it this year." You know?
Dave: Sindre Sorhus, I think, right?
Chris: Yeah, just--
Dave: Notable.
Jason: Baller.
Fred: There's definitely going to be pain as people -- you know, we've all been writing ESM, this kind of import/export, and then turning it into something else. So, we are going to, I think, get this pain first of everyone thinks they know how it works and then they're going to change how they're actually -- it'll look the same but run different. That's going to be a real -- from experience, it's a weird feeling, but the end result is very cool.
Jason: That's kind of where things like Snowpack and WR and Vite (veet, or whatever it's called). I think that's where they come in is sitting kind of on the sidelines, initially, but then looking at this. Fred and I have both maintained other bundlers before. There was definitely this growing sense (at least for me) of, like, I think people need to be exposed more directly to this ES Module syntax and also all of the implications of it that now have landed in Node so that they start to understand it in their own codebases. That way when folks go to author NPM modules, publish them, or install them, they know what to expect. It's not like they have to go and figure out this thing that is replacing CommonJS. It would more be like there is just a setlist of knowns that you can expect from an NPM module and CommonJS kind of becomes the weird case.
[Banjo music starts]
Chris: This episode of ShopTalk Show is brought to you in part by Netlify. Thanks for the support, Netlify.
You know one of the things it can do is help you with functions, things you're like, "Oh, but Netlify is just static hosting. I need to do server stuff." No, it can totally help you with that stuff too because it supports the idea of functions, which they run in AWS Lambdas. You can do server-side stuff with them.
They're starting to be this whole, like, suite of functions that you can choose from. The kind of default ones--I don't know--maybe you'd consider the most important ones, the ones that just run on a normal Lambda. You put a JavaScript file in a folder called functions--all this is configurable, by the way--and then you have an internal URL that you can hit that will run that function, which is cool because then it circumvents cores and stuff, which is great.
It's like you don't even need an AWS account. It just gets deployed automatically. Deploying a Lambda is like a whole thing, so you get to skip that with Netlify, which is tremendous - super-duper cool and useful thing that Netlify can do. That would be like--I don't know--go fetch some data and bring it back to me. That's probably, like, maybe the most useful case but there's anything. Anything you need a server to do, you can use a function for.
There's limited execution time of those. Those are supposed to run really fast and they're really quick, cheap, and all that. If you need one that runs longer, this is the second one in the suite is they have these things called background functions, which run for up to 15 minutes. If you need to do something in a Lambda that just takes a long time because it's just lots of API requests or slow ones or it needs to spin up lots of dependencies to do what it's going to do--or who knows what--it's the same type of deployment. You just name the file a little different with dash background in it. Then it will be a background function and you can run it that way. I think you have to be on the pro plan to use that one, which just makes sense.
Then there are the edge handers. I've mentioned that before on this show. It's a different type of function. It's like a function that automatically runs and it runs on the request before the browser even gets it. In the process of how the Web works, it happens at the CDN level. It would be like, "I am also going to hit a database and get some data, but I'm going to do it immediately, as the URL gets hit, and transform the HTML before it even arrives." It's not like a client-side request to the function. The client doesn't do anything. It's already happened by the time it gets to the client.
Super cool. All are possible on Netlify. Thanks for the support.
[Banjo music stops]
Chris: All right, so we're in a weird place, right? Most of stuff, maybe, on NPM isn't ESM or JSM. We're going to call it right here.
Jason: MJS.
Chris: They're called JavaScript.
[Laughter]
Chris: They're called JavaScript modules, are not ready for this, but wouldn't it be nice if they were? Everybody has heard of unpackage, right? They're kind of like a middle generation CDN, I feel like, because there were things like jsDelivr, and there was CDN JS, and they are both weird. They were like, "You want to be on our CDN? Make a pull request against our repo with your bundled library that's definitely probably in UMD because who knows who's going to use it: Node or the browser or whatever." Uh! So weird.
Then unpackage comes along and says, "No, we don't have any of that. We're just a CDN for everything that's on NPM. But, caveat, it better be -- we're not going to change your file in any way. It's just whatever is there, we'll link up." If you want to take advantage of the CDN, whatever you push to NPM has to be built already, or whatever. I don't know how common that is because I don't live in that world all that much. But if you wanted it to be JavaScript modules ready, too bad. If it's not, it's not. Which sucks, to me.
Then along comes Skypack and says, "Oh, we'll make them JavaScript modules ready." That's like, "Oh, shiz. That's good. I like that."
Now everything on NPM, I can just link to with one line and if it wasn't ready, it is now. I'm sure there are caveats to that because that's the world you're living in, Fred. That's what I'm -- that's the Skypack part and not the Snowpack part, but do you want to talk about that...?
Fred: It all comes out of the same project. Pika, in the early days, we were like -- it was just a research project of, like, let's build a package creator. Let's build a Web app builder. Let's build a registry and a code editor. We were kind of just throwing stuff at the wall. There was so much to uncover with what you could do with this new technology.
What every project came back to was the NPM ecosystem is the thing that's essentially holding the story back. You have a million-plus packages that are all written one way and then everyone wants to unlock what happens when they're all written another way in the CSM.
Chris: Yeah.
Fred: But the second one of those is brought into your project and it's not in the new style, you kind of end up in the weird world of supporting both or going back down to the lowest common denominator. You have this kind of weakest link in the chain, which is any package that isn't up to date.
What we ended up doing and what Snowpack really comes out of was this idea of, okay, let's focus on the package itself as the thing that needs a bundler. Take the package, bundle it into ESM, which is what Webpack, Rollup, what they've all been doing is essentially taking a website and bundling it. We're just going to only take the package.
Chris: Right.
Fred: Now, you write the ESM that works in these modern browsers. You have the same exact dev experience that you're used to in terms of developer setup in DX. But the packages have now come along and they are now ... for you.
Chris: Yeah, and you could have done that with any of them. You could have done it with Webpack. You could have done it with Rollup. You could have done it with whatever. But that story is changing too. That would have costed you money.
Fred: Yep.
Chris: That would have cost you straight up bucks to run every single package on NPM through Webpack. I don't know what your cloud bill is, but it wouldn't be good, probably. Not that Webpack is slow. It's just, there's a lot of resources.
Fred: A million-plus packages, yeah, it's pretty big.
Chris: Yeah, so along comes this thing called ES Build that everybody is freaking out about, right? It's what, a thousand times faster than Webpack, or something, at this particular job. You, maybe, with your entrepreneur hat on, says, "Well, I'll use that then." Surely, that's behind....
Fred: [Laughter] We've been solving this problem before ES Build, but ES Build certainly helps.
Chris: Okay. Yeah, then you were able to -- whatever. Do you use ES Build now?
Fred: See, you have a financial mind. [Laughter] I just have a, "Oh, this is cool," mind.
Chris: [Laughter]
Jason: Oh, this is cool. I hope that bill won't be as big as I'm thinking it's going to be.
[Laughter]
Fred: Right. Exactly. Exactly. No, I mean that's the big thing about ESM as a technology is that it kind of flips the script where you start to leverage Web things in a way that, up until this point, we've all been thinking in Node first, like Webpack is a Node tool, Rollup is a Node tool. But ESM, for the first time, that idea of, like, "I want to import a package from a URL," it's a really simple feature that totally blows your mind in terms of what you can do with that because that URL can be a CDN, it can be local, it can be a relative path, it can be a remote thing.
Then on that level at the remote side, if it's a CDN, you can start to cache. So, our cloud build, there's a lot of complexity going on, but there's a lot of caching as well. The idea is that we build it for the first person who asks it and then we cache it as long as we can after that.
You get all of these Web technologies kind of coming back into the story that have always been there but kind of pushed out from the tooling world. They're now kind of back and a major part of the story.
Chris: That's good. That's a good story. We've covered that pretty well. There's lots more to talk about.
Now, let's diverge. You have two different tools. Snowpack and WMR are different-ish.
Fred: Ish. [Laughter]
Chris: Let's do Snowpack first. Yeah. [Laughter] I also think that Snowpack, pre 3.0 and post 3.0, are pretty different too, aren't they? I always thought of Snowpack as, like, "Well, you've still got to NPM install stuff. Then you run Snowpack, which is a little bit of additional complexity on top of that. But if you do it, then you get these pre-built packages that don't change so that you're not running a bundler on every command S, which is cool." Is that still the core benefit of Snowpack? How has that changed?
Fred: It's a very new story and idea that you don't need to run NPM install. Essentially, it's kind of the completion of this vision that these two projects came out of this Pika research project and now they're kind of coming back together to support each other.
Snowpack, as a Web app builder tool, can pull from any ESM CDN that supports a couple of hooks that Skypack does and grab your ESM code that is ready to run in the browser, already built for you. It's the Web build tool that's doing that package, fetching for you.
There's no more NPM install. There's no more Node modules folder for your front-end code.
Chris: Okay.
Fred: It's all just run through Snowpack as both the build tool but also realizing the packages then are just fetched. There's not as much work that needs to be done.
Chris: You literally said no NPM install, and I believe WMR has that as a benefit too.
Can people out there -- I know you can't respond. This is rhetorical -- imagine a world in which you're working on a JavaScript project and you don't even NPM install?
Fred: [Mind blown]
Chris: Yeah, [mind blown] indeed. Is that just a developer story? Do you still ultimately need to in your build process for production? Is that a different story?
Fred: Yeah. Jason definitely gets credit for this. I think he was the first one to really launch this feature. He can probably speak. I'd actually be curious to hear how he came to this idea. I know he's been playing around and kind of secretly tweeting screenshots.
But for us, it was this idea of Node and JavaScript and NPM as your core tooling ecosystem. It's really great for JavaScript developers (not so much for a Rails or a PHP developer) where you're kind of having to learn Node, NPM, semantics, and tooling just to build a website that really you're kind of living in this Rails world.
We have this kind of bigger vision of, if you can just turn Snowpack ... install maybe via Homebrew or some other ecosystem where Snowpack is the one thing you need to install, now you're fetching packages on demand. You've essentially left the NPM -- not the ecosystem. You're still getting all the same packages, but that whole Node tooling, you wouldn't even really need a Node dependency other than what comes with the Snowpack installation.
Dave: I literally had this conversation with a Java developer last week. He was like, "Hey. They're using React. Why do I need Node for React?"
Fred: [Laughter]
Dave: I was like, "Oh, buddy."
[Laughter]
Jason: You are in for a fun time.
Dave: How much time do you got? You know. But in his brain, he's a Java and he's got Spring and Tomcat or whatever crud they use. He has a server and, in his brain, Node is a server. You know? He's like, "Why do I need a server?" I was like, "Oh, okay, so a server just spits out the files you want. That then goes and runs in your spring templates."
Fred: Yeah. Yeah, the model is totally different. Today it's like, get these packages installed and then do all this work. All of that tooling is written in the Node ecosystem, so it uses NPM and it uses more Node and more packages. The whole toolchain today lives as a product of the NPM ecosystem.
What we're trying to do is turn that toolchain into kind of somewhat of the roam vision of, like, if you use Snowpack, you can fetch your packages and you can kind of just -- that's your new foundation where, if you want to bring in NPM and Node for other tools, that's great. But if you're pretty happy in your world and just kind of want this front-end part of it to be a little more superpowered, that's where we try to fit in and then try to reduce that complexity.
Chris: Imagine then -- we did cover that this is native, right? There's this real simple story where you have, let's say, an index.html file. At the bottom of it, it says, "Script type equals module," you know, "Source equals main JS." Main JS at the top of it says, "Import something from Skypack," whatever. Guaranteed, ready to go package from NPM, and you use it.
You need no build tool at all - nothing. It will just run. You can put it on Netlify or an S3 bucket or whatever. There's nothing. There's no build step or whatever in between. But some people like to work that way. A lot of people are like, "Yeah, that's my style. I don't do nothin'. I'm just an HTML kind of guy," or whatever.
But then there are probably more people that are like, "Yeah, but I need a local dev server."
Jason: JSX. [Laughter]
Chris: JSX, yeah.
Jason: Yeah.
Chris: Yeah, but JSX.
Jason: Yeah. TypeScript.
Chris: Yeah, but a little TypeScript. Yeah, so there are all the buddies that come along. Buddies have always been the problem. It's not React's fault. It's the buddy's fault.
Jason: [Laughter]
Chris: I don't know what I mean by "fault," but you know what I mean.
Jason: I think the interesting thing is that is the exact disconnect that ESM made formal. Right? The specification now fully provides a way to do module loading and the spec says it's a URL, so the idea of pulling a module from a CDN is by no means novel, right? It's literally what the spec told us all to do.
But there are all these development time kind of oops that come out of that like, oh, if the URL is one byte different then the same URL you used in some other module somewhere to import what was supposed to be the same dependency, they are two different things now. Sorry about that, right?
For a while, unpackage got really popular to use, like the question mark module query string parameter. There were a whole bunch of things with that where that would convert dependencies to URLs that referred back to unpackage. But if they weren't exactly the same version already pre-resolved in the specifier, you're getting two copies of the package. I think that, combined with the TypeScript and the JSX and various other things (MDX, all these sorts of compile to JS file formats) most projects at some point hit a roadblock using just standard, in-browser URL imports.
Chris: Mm-hmm.
Jason: It's like, ah, is this the point where I have to go from zero tooling to the polar extreme of just being tooling-based?
Chris: Hundreds of megabytes of tools.
Jason: Right.
Fred: Mm-hmm.
Jason: Where you're no longer developing a Web application. You are developing a codebase for producing that Web application. Your product is the byproduct of not your code but your code and your tools.
Dave: I didn't expect you to come here and throw Svelte under the bus just right out of the gate.
Jason: [Laughter] I actually love Svelte. But that's kind of the point, right? Svelte, to its credit, I don't think Svelte says, "Oh, you can start programming in just bare-bones JavaScript and there's no template language and no magic," or whatever. They don't give you that promise knowing that they're going to have to take it away later.
Interestingly, Vue actually does an interestingly good job of this where they give you the promise and they say, "It's just a little bit slower, but we'll do it."
Dave: Mm-hmm.
Jason: Then, really, their tooling ecosystem is like, "And and and and."
Chris: Right.
Jason: It's sort of add-ons that....
Chris: Meaning, you can use Vue right in the browser.
Jason: Right.
Chris: It works just fine. As a matter of fact, we've had Evan You right on the show and he said to the world, "This is a huge priority for Vue. A lot of people use Vue in that way. It will always be a priority to the project. Rock and roll." You know?
Jason: I think all of these tools basically stem from what Evan was describing there, which is the disconnect between starting a project, working on a prototype, getting something done, starting from a script tag in an HTML file, and what you actually have to do to get something into production has been so, so wide for so long.
Chris: Mm-hmm.
Jason: Now that we have ESM, the number of reasons why we need all of those tools to come in later in the game is just sort of starting to erode.
Chris: Mm-hmm. So, that's what your questioning, that wideness. You want to shorten it, both of you, in a way.
Jason: Basically.
Chris: What's the smallest amount? What's the narrowest thing between the promise of all the fun developer, ergonomical stuff (like TypeScript and JSX)? Can I provide that in a very narrow way? I guess that's a weird word.
Jason: Literally, the genesis of WMR was, I always code on Glitch. That's where I make most of my prototypes: Glitch, CodePen, Code Sandbox - depending on my use case. For some reason, for design stuff, I go to CodePen.
Dave: We'll bleep all the competitors. We'll bleep all the competitors.
Fred: [Laughter] It's just Chris and CodePen, CodePen, CodePen, and CodePen.
[Laughter]
Jason: That actually -- no, so that actually -- that comes to my point, which is, Glitch has a static mode that is more like the one-to-one with Code Sandbox -- sorry, with CodePen. Constantly, I'm conflating those two names. It's like Shopify versus--
Fred: Spotify. [Laughter]
Jason: Spotify. Yeah.
Chris: Oh, god.
Jason: I never even try to type the URLs.
Fred: Always do it.
Jason: Glitch has a static mode, right? Like CodePen, that is sort of quick. There's no server involved. You can somewhat rapidly prototype. It doesn't have the library load stuff that CodePen does, but they also have their dynamic mode, which is, you write a Web server in Node to serve your stuff.
To me, the fact that there is no middle ground there is broken. This is not a criticism of Glitch. This is a criticism of the way that we build stuff. It's either dynamic or it's static and there is no middle ground there.
WMR was literally just a project I've been toting around, one file I've been toting around from Glitch to Glitch at various points in time where I would take the dynamic Glitch that lets you run a Web server and run the smallest possible Web server that just did JSX and swapped out import specifiers to point to unpackage.
Chris: It's your opinionated preferences, but they're pretty popular preferences.
Jason: Well, and it was also the roadblocks that I continually ran into as I was trying to build these things and as I was trying to avoid jumping into the build tool stage. As soon as I do that, I kind of lose out on it being a rapid prototype.
Right now I'm thinking about, like, how does this stuff get compiled? In 99% of cases, that is beyond unnecessary and it is only that way because of the tooling ecosystem's decisions that led to this point.
Dave: Oh. Well, I was going to say, on those tools, you don't always have a file system. I can't just write a hundred bundles out on CodePen. You're right. Once you're at the tooling phase, you're just like, "I'm sort of -- this is -- okay, I've got to do it locally, I guess."
Jason: Yeah.
Dave: You know? And so now your velocity has just zeroed.
Jason: Well, and you lose out on all of the value that that rapid feedback cycle from CodePen and JS ... all these things. That is the value proposition and, honestly, it's the reason why most of the interesting things that I've ever built have started in those rapid feedback editor environments. They either die when I excise them or I do it late enough that the idea has been fully formed and I can start to justify a bit of tooling investment. But that cutoff sucks. [Laughter]
Dave: No, I was a building a game in Vue, having fun, and I was like, "This is out of control." [Laughter] I'm at 500 lines of code in a CodePen editor. I just was like, "All right. Let's pull it down. Let's do the responsible thing." [Brakes] It's done. It's gone. It's not going to ever work on it ever again, but it's not because I don't think it's cool or whatever but it was just like, I wanted to go to something--
Chris: We need to change that moment, don't we? Let's work on that. We'll get there.
Fred: Well, I think Jason brought up -- I really like the phrase, "The tooling decisions that got us here." There was really, I think, once the foundation is tooling, it becomes really seductive to just keep adding to that. That idea of, like, "Okay, we're going to use React. Okay, React is CommonJS. It's this older format. It won't run in the Web. Okay, so we're going to use a bundler."
Once every React developer starts from, "We're going to use a bunder," they essentially give every tool that they might want a hook into their code. Now they're making changes. They're assuming -- you know all of these things build off of that technical assumption that, "Okay, we're just going to have a lot of tooling."
Once you already have a lot of tooling, what's one more? What's one more? Throw it all in. The next thing you know, you create React apps and your starter apps almost have to be thousands of dependencies because there's so much complexity to how the kitchen sink (all features included) that you're not really removing complexity. Now you're just trying to hide it, like, "Okay, there's this really hundreds of lines Webpack config inside of Create React App. We're just going to hide that from you."
The downside there is now you can't extend it. Create React App is basically like either eject and we don't want to touch it anymore, or we have the complexity but we've quarantined it away from you.
Jason: Yeah. I guess, conceptually, my knock on this -- and this is going to sound negative towards Webpack and I should preface it with saying I actually think Webpack is great. I just reviewed Sokra's PR converting our tooling.report stats for Webpack over to version 5, and they've done some awesome stuff. Their score is through the roof.
But, conceptually, if you start a project by installing a bundler and then piping your code through that bundler, if Webpack is a million lines of code, your project is starting at a million lines of code. Maybe some of that code is dark. Maybe it's only 500,000 lines of code. That is still astronomically more complexity than a file or a thing that isn't even a file - a text area.
Chris: Yeah. Certainly.
Dave: Yeah. I used to laugh at people who downloaded IntelliJ or whatever. It was just like [snicker]. I'm like 17 PHP files up here--
[Laughter]
Jason: Notepad++.
Dave: --doing business. Yeah.
Fred: [Laughter]
Dave: But now it's like, that's what I do. I download hundreds of megs on the hour, so you know.
Fred: Just to get JavaScript, which runs in the browser, to run in the browser. [Laughter]
Chris: [Laughter]
Dave: Yeah, just to change it a smidge.
Fred: Yeah.
Chris: Yeah. Yeah.
Dave: Put in commas or semicolons.
Chris: It seems weird that it's not smart about this, about ESM in the browser, a JavaScript. Like if you send something through Webpack and it's got import x from Skypack something in it, it will not leave it alone! You cannot teach Webpack to leave it alone, which you definitely want it to leave it alone. I authored that URL on purpose.
Jason: I think this is actually one of the most important things that -- I don't even know if WMR and Skypack and Snowpack and Vite -- I'm going to get butchered for saying that wrong -- I know it's French - "veet." I think this is the thing that they're all kind of skirting around and none of us have technically taken it to the full extreme of de-no-style. Everything is a URL. There are no local packages. Like as the authoring format. But that's part of it is, like--
Chris: Weird.
Jason: We are all still working out of the mindset that everything starts as a file on disk or that everything starts as an import that would be expected to run in Node. There's so much unlearning that has to happen and that unlearning also then has to get put into our tooling.
Webpack's footprint is a bundler. It doesn't even do transpilation. I think now we're at the point where the footprints that we happen to carve out and make project boundaries, we all know they have to change a little bit. They have to move over time, and we don't have a system (as a community) for moving those boundaries. We don't have a system for Webpack to incorporate some of what Babble is currently doing.
If Webpack started auto-installing dependencies, that would be massive upheaval, right? Is that even Webpack at this point? Is that something that's wrapping Webpack?
That's why I mentioned some of the Webpack 5 stuff. Tobias has been starting to bring some of these additional footprint or change in footprint things into Webpack's configuration, but I know that they are still struggling to come up with a way to explain to you, like, "Oh, I know it seems like Webpack is just bundling modules, but it's actually doing a lot of code generation, so you should probably tell us what your browser's list is because we're generating 50% of the code that you're shipping and we don't know what browsers we're targeting."
Dave: Oh, yeah.
Jason: It's like, ugh, okay, yeah. We have been missing a lot of data for a lot of years that had just been taken for granted.
Chris: I don't even understand all -- like what it does exactly, but I know you can do something like a dynamic import in Webpack, right? Say, "Only under these circumstances load this file," which means that it must have some kind of fetch mechanism built into it, right? Webpack then has to--
Jason: Yeah, it uses script tags and--
Chris: Yeah. Who knows. But that's transpilation to me. I mean I guess it's a different word. It's more like--
Jason: Compilation, almost.
Chris: --processing. Yeah, but it's not changing one syntax into another. It's doing all kinds of stuff.
[Banjo music starts]
Chris: This episode of ShopTalk Show is brought to you in part by Linode. Linode is great. They've been around forever. One of the first companies in cloud computing, three years before even AWS was a thing, so they are there. They are their own beast, in a way.
I think this is a pertinent way to describe it. If it runs on Linux, it runs on Linode. You can do anything. These are cloud computers that you buy to do anything you want.
For example, let's say you're a gamer. They have all kinds of one-click apps that are gaming related. You want to deploy your own Minecraft server or Counter-Strike or whatever? You just click a button and spin it up on your Linode server. That's kind of cool.
It's probably mostly for Web stuff. It's also notable that Linode is a step past entry-level hosting, I'd say. [Laughter] If you want to be super in control of everything, actually own every detail of your hosting, Linode is a step up to totally customizable cloud computing. It's VPN friendly - all that stuff.
Obviously, they have great human beings that are going to support you in what you're trying to do. They've got GPU hosting if what you're trying to do is machine-learning based - that kind of thing. Again, if cloud computing is what you need, if it runs on Linux, it runs on Linode.
Go to the link in the show notes. They have a special page just for ShopTalk Show listeners where you get $100 of free credit, which is pretty generous of them, I think. Try that out. Thanks for the support.
[Banjo music stops]
Chris: WMR is JSX. You brought up Babble. That's normally what you'd think of as "the thing" that you need to make JSX to work. I don't know what you're doing for it. There's that, but what's the Snowpack story then for JSX? If you need JSX, is it Babble somewhere or do you have some other secret way of dealing with this?
Fred: Yeah, so it's really a kind of flipping of that model, so the Webpack rollup kind of traditional of the last decade setup has been, you have a bundler, you plug things into that bundler, and you're doing all this work that's bundling and also transpiling at the same time. It's one tool to kind of rule them all, to do all these things.
Snowpack flips it a bit where the bundling is something that you add only if you want it for production, so your production build bundling can happen. But during development, every file is essentially one-to-one built as it's needed. If it's a .jsx file, we have built-in JSX support. If it's a .svelte file, you can add a Svelte plugin.
Chris: Your answer is, don't worry about it.
Fred: Yeah.
Chris: We'll do the JSX for you.
Fred: Don't worry about it or grab a plugin.
Chris: Okay.
Fred: Which is not too dissimilar from the Webpack story. I think that's worth pointing out. It's similar, like, we'll support some things by default; other things need a plugin. But the difference is that the plugin is only building one-to-one files. It's not having to think about, "Okay, well, how does this fit into a bundle? How do I resolve this?"
Everything is -- it's not -- I sometimes worry about saying this but it's kind of the Gulp model. If you take out the terrible ... implementation, you just have this really nice idea of, like--
Dave: Wait. What?!
Fred: --here's your source file.
Jason: How many copies of through to do you have?
[Laughter]
Dave: (Indiscernible)
Fred: ...Sorry.
Dave: I don't know. Chokidar crashed, so whatever.
[Laughter]
Jason: That's probably a sore spot for both Fred and I in this particular case.
Fred: Every error just somehow disappears. Yeah. No, Gulp -- this is not me saying Snowpack is Gulp, but it's a similar model of a stream of builders, essentially. I have a JSX file and I want to turn it into JS. Maybe I want to add some HMR code so that you can get these hot reloads. Then I want to output to the browser. It's a much simpler model of, like, I'm not trying to configure how something bundles. I'm just saying this file gets built these three ways. A Svelte file gets built only this one way. The simplicity there is really apparent when you start to add these plugins and add these configurations.
Jason: I don't know about Snowpack's internals for this, but one of the things that I had peeked in the Vite source code and I saw this kind of after we went down this road in WMR (and it was kind of validating) is this idea that there are certain language transforms or features that they're either so common that they need to be part of this basic process of turning a module into something that runs natively in the browser or they're not common enough or they're not spec'd out enough that they become this you need to enable via a plugin class of transformation.
I think that's where JSX gets interesting is it's not a spec. It is never going to land in JavaScript and there are valid replacements for JSX that are native JavaScript. I am too biased to name names, but it's one of those things though--
Chris: Really? I don't even know what you mean. You've got to give me one name.
Jason: HTML. Just tag template literal implementation of JSX.
Fred: [Laughter]
Jason: With caveats. It doesn't support TypeScript.
Chris: Okay. Okay.
Jason: It doesn't support the JSX-type inference type stuff, which is really important and useful. But I think that's where JSX ends up striding this line. It's just like, it's really useful. It's really common. But it's not in the language, and so we can't treat it like we would dynamic import and it's not necessarily handled by default by all these tools.
That was actually one of the reasons why I originally went down the road of just trying this out in WMR specifically is, I don't think it makes a ton of sense for other tools (certainly, for Vite) to just default to assuming code would contain JSX because if you're not in the React paradigm, that might just be a pointless cost. If there's no JSX in the file, you have to parse it because we have ambiguous angle bracket syntax. Why would everyone pay that? Right?
Chris: Okay. Yeah.
Jason: All these bundlers, or all these non-bundlers end up being an exercise in, you know, we have this file pulled off disk served to the browser and we want to do that in as close to zero milliseconds as possible. We want it to be as fast as a static file server. That's the gold standard. What transforms can we apply? From a technical standpoint, how fast can we run them but, also, just from a design standpoint, what is reasonable to do? Class--
Chris: What is reasonable?
Jason: Yeah, I mean--
Dave: Yeah, would--?
Jason: Classes and async wait, those aren't fast transforms. They never will be. They're also transforms that perform more poorly when done on an individual file-by-file basis.
Chris: Is that the, like, regenerator run?
Jason: Exactly. Yeah, and even with classes.
Chris: Yeah. Okay.
Jason: If you do it on a per-file basis, you're going to end up with a myriad of different variations of transpiled classes. And so, when you combine that with 95.6% (I think) browser support for classes, you start to realize that, especially during development, that's just straight-up waste. We are just doing something because that was the default a couple of years ago.
When you look at shrinking that pipeline time per file, that's stuff that gets immediately thrown out. Then you start to look at, like, okay, if I get this list really small, if I'm not doing JSX, I'm just doing important transformations, do I even need to parse the file into an AST? Can I skip that?
Can I use a lexer and just identify where the string boundaries are and then figure out from the surrounding source tags, like, "Oh, this looks like an import. I'm going to do my import processing stuff and I never even have to do a full parse of the file"? That's where we start to see these orders of magnitude faster pipelines out to the browser is in the culling of tooling that we can kind of take a stand as being unnecessary, at least during development.
Chris: We have a RegEx at CodePen that guesses if your file needs -- because we don't want you to have to type "type=module" in the stupid script tag, which, could somebody explain to me why that's necessary? Why did HTML make that necessary? Can't it just--?
Jason: It's because modules are strict mode forcibly by default and loading code as strict mode that wasn't designed for it can break stuff. If you do stuff like assigning to arguments, it is mutable but references into arguments via local variables are not.
Chris: Are we going to have it forever until 2030? Are we going to have type=module on script tags still? Yes, probably?
Jason: Well, I think what will happen, honestly, is type=module will become the only way people load scripts, but you'll probably only type it once. It's essentially how you configure entries for all these tools and then everything else will just be an import ... import.
Chris: Oh, you're right. There'll just be one script and then it'll spider out from there. Yeah, okay. That's fair. All right, so yeah. Our stupid RegEx, it's a little complicated because there are a million ways you could import something. There's import just a word and then there's the curly brackets kind.
Jason: Are you inside a comment?
Chris: But then--
[Laughter]
Fred: Oh, gosh.
Chris: Right. You have to watch for that because if somebody comments that out, they might not want that anymore. Then there's the dynamic kind, too. A weight style one. Yeah. It's tricky.
Jason: [Laughter]
Chris: You got it wrong for a hot minute, but that's what users are for. They just write in and tell you that it's wrong. Cool.
Dave: What do you support? I imagine TypeScript, JSX, Vue SFC, Svelte stuff. Where do you--?
Chris: You had this WMR to do Vue.
Jason: Yeah, so this is a slight sore spot right now. It was supposed to. WMR, the real underlying feature in WMR, the thing that I cared about -- there's always only one small piece of any project I build that it matters to me. That's the dirty secret. I built out--
Chris: [Gasp] Which one is it going to be?
Jason: There is a Rollup compatible plugin. I say Rollup compatible with a heavy caveat. [Laughter] As Evan can attest. That's the thing I care about.
Chris: That's the thing that you care about?
Jason: Here's why. Rollup's plugin API is very good. It actually does a reasonably good job of modeling what Fred was alluding to earlier, which is, there is module transformations and bundler plugins. Those are severable ideas.
Rollup's plugin API, they're separate. Right? You know you have input plugs and input plugins, but also there are only certain plugins that operate on a per-module basis. It is not like an ordered list of transformations that happen at times. It is a grouped typed thing like list of transformations that happen as dependencies are result.
This plugin runner in WMR was a way for us to write Rollup plugins both for WMR's implementation and also for people to add them in configuration and have those run during development even though we are not using Rollup. As these files get pulled off disk and parsed and passed out to the browser, one of the steps they run through is this fake Rollup plugin layer.
To me, that was the crux of this is figuring out what transformations are safe to apply in that minimal of a plugin layer. I want to be able to do things like Vue, SFCs, and Svelte files. There is no--
Chris: What if it was a little too slow? Would you kill it? You'd be like, "Ah, that's actually eight milliseconds, so I'm going to kill it."
Jason: [Laughter] There is a ... right now that it doesn't kill it but it's very loud warnings in the console. I think probably the saving grace here is (both for Vue and for Svelte) there is a file extension, and so it's extremely clear this is a Svelte file, this is a Vue file, and that makes it opt-in, which means it's pay-per-use, and I kind of--
Chris: I do kind of like it when people use .jsx when they mean it, even though you don't have to. You know?
Jason: TypeScript has this advantage as well.
Fred: Mm-hmm.
Jason: In WMR, I didn't want to write a TypeScript transformer. That's a pain in the butt. I think, currently, we're using Sucrase, but we're contemplating going back to ES Build for this. If we see a .ts or .tsx file, we will run it through Sucrase during the plugin transformation process.
But for most files, if it's not a Svelte file, a Vue file, whatever, and it's just JavaScript, we just lex it and find the strings. We actually use ES Module lexer from Guy Bedford, which is the thing that Node is using, the CJS Module lexer. It doesn't actually have to parse the file in full, which is, I think, the win here.
Chris: I thought you were going to say CSS modules. That was your favorite. That seems like the biggest wildcard, WMR, to me.
Jason: I only put it in there because I like to use it. [Laughter]
Chris: I like it too.
Jason: It's minimal.
Chris: I think it's a good choice. Okay, so both of these tools, Snowpack 3.0 and WMR, don't require an NPM install. Mind-blowing stuff right there. That's pretty cool. They both have some special production strategy because it's kind of like there's the dev story and the production story can be different and that's okay. Is that because it's proven that, like, reducing requests is still a thing that matters for performance? I feel like I read an article just yesterday that was like, "We analyzed the top million websites and this is what they found," and a big part of it was number of requests was high. If you have a lot of requests, that's very easy to correlate between a slow website and a fast one.
Fred: Also, don't use Ember on.... I learned from that.
Jason: [Laughter]
Chris: Oh, yeah.
Jason: To be fair, I think most of those things--
Chris: Everybody ... Ember....
Jason: --when they see the request to performance correlation, it's often third-party driven. If you have a giant waterfall of iframes, you're going to have a giant waterfall of requests. Basically, yeah, there's your meg of JavaScript. [Laughter]
Chris: Yeah. Okay, so it's kind of true, though. Wasn't HTTP2 supposed to make that not matter and then that turned out to be not true?
Jason: From what I understand, this is a multifaceted problem. But right now, the reason why just shipping native ES modules in production with no bundling of any kind, including for dependencies -- let's say there are a thousand ES modules in your graph -- there are a couple of things that make that slightly slow.
Chris: Yeah. That's bad.
Jason: One is the modules are only progressively discovered each time the browser parses it, finds imports, goes and fetches those. And so, you end up with a waterfall. You can kind of collapse that by preloading things that are down the waterfall, but then it's a chicken and egg game.
Chris: By reloading, you mean a literal HTML element that says "link rel preload."
Jason: Yeah, well, "link rel preload" is going to get you network preloading, but it won't pre-parse the module. If you use "link rel module preload," it will preload it and pre-parse it in Chrome and find its dependencies and start to resolve them but it's only supported in Chrome. That's kind of an eh.
Chris: Even that isn't amazing? Oh, wow. Okay, well.
Jason: Yeah, and it has a cost, right? It's parsing. Even if you set all of that stuff aside, there is a small amount of overhead associated with parsing a module and getting back its result. Not the actual parsing, but just fettering it off to the background thread that does the compilation and the parsing and getting that back. And so, if you do that a thousand times instead of two, even if the two times the files are much larger, there's less transfer overhead, there is less machinery going on.
Chris: Okay, so we're never going to win. It's just like bundle for life. It's like we're never going to get--
Fred: This is where I think Jason and I -- Jason has his thing that he was interested in and I think the one thing that I really got into this with was this idea that bundling has been a requirement for so long. Most people don't really need it.
I think what Jason is describing is, like, I want the fastest site possible, which I think is obviously an important thing. But for a lot of people, they're just kind of like playing around or they want to build something that just kind of works. The core idea of Snowpack is you can pull in this complexity as you need it and ignore it if you don't. Just the idea that you can build a site that runs in the browser without bundling, if that's all you need, maybe you're building something for a family member or for a small company. Obviously, performance is important but, for a lot of people, it's not the most important thing.
Jason: Well, and I think that's maybe where we settled on all these tools now at this point basically pre-bundle NPM dependencies because those are cases where we can flatten that graph and it's pretty safe because we know you're not reaching into those dependencies. Then if you have 100 source files and 5 dependencies or even 50 dependencies, that's only 150 modules. That's actually not that big of a module graph versus if you have 100 source files and 50 dependencies that each contain 50 modules--can't do that head math--that's a boatload of modules. [Laughter] Right? Now that graph starts to get large. But if there are operations we can perform on the static parts of the graph to just sort of magic that away, those feel safe and they don't really deteriorate the native ES Module's play.
Fred: The other thing, which I think a lot of people -- you know it's been bundling is the right thing for so long and I think now it's a little bit more complicated. I think, in 90% of cases, it's probably the right thing. But you can think about, again, the caching story of having every ESM Module downloaded. If sending Gmail where it's something that's loaded and then opened for long periods of time and more of like an app that you keep coming back to, you can really optimize for second, third, fourth kind of future load by -- you've downloaded all these ES Modules individually and now the team behind it, they push a change, and maybe it just changes one file.
Chris: Yeah, the cache is better.
Fred: The browser is not that smart enough to cache everything and only load that one file with some caveats around how you actually do that caching, whether it's local or via some sort of header. But it's still a very different performance story. For some people, it might even be a better performance story and then bringing that preloading into....
Chris: Oh, I'll tell you. I feel guilty when I make one tiny little baby-baby-baby-baby change on CodePen and the whole bundle is gone.
Jason: Whole hash is invalidated.
Fred: Right, and you've blown up the whole site. [Laughter] Exactly.
Chris: Yeah.
Fred: That's no longer a truth anymore.
Chris: Yeah.
Fred: And I think that that's really -- that's where I come into this really excited about transforming that space.
Jason: Yeah, I think I kind of have framed it as a development time thing. But I actually totally agree with that in production. This is one of the reasons why I've been tending towards Rollup more as a production bundler rather than Webpack (of late). Not to disparage Webpack. It is working on being able to do this better. But because Rollup can output the native ES Modules, you can almost think of it more as like an operational transform on your graph that doesn't remove the graph. It just sort of collapses pieces of the graph that are one unit.
I guess the key of this is, there are a lot of heuristics in the bundlers that are doing module bundling and collapsing to try and make that work and avoid the cascading hash invalidation, and one tiny change invalidates all your CSS kind of thing. If you don't need those in the ES Module setup, there's clearly stuff we can learn from that.
I love the idea of a pure ES Module setup and I think that where we land on production should fall largely on that side of things. Right now, it seems like there's just a couple of things in play. I know the VA team is working on making the loading of ES Modules itself faster. I think as some of those pieces of underlying tech evolve too, it's going to shift the story even further in favor of the modern potentially multi-file output to the point where -- there are people that fall on both sides of the tradeoff right now when you look at it from just a technical standpoint. I think we'll start to see maybe even the majority of people falling onto the unbundled or predominantly unbundled side of that tradeoff as the tradeoff shifts.
Dave: I think what I'm kind of hearing in my head is maybe bundling is maybe something library authors do or framework authors do and maybe me, the person who is just making a website, maybe I don't need bundling. Is that kind of where things are sort of headed?
Fred: The story I picture is, today it's like I have this really complex ... setup, but everything I do is bundled because that's the foundation, so output always bundled. I might get lost along the way. It might be too complex. I might get frustrated and quit or spend a day in the Webpack config or a Rollup config.
Jason: Just a day?
Chris: Just a day?
[Laughter]
Fred: It's built in. It's a pain. It's a pain. Even the people who do it well, it's still a pain. The difference is that now the model, at least, that Snowpack is really excited about and really building towards is you pull these things down when you start to want them.
Let's say I'm working on a prototype and I'm having a great old time. I'm iterating quickly. I'm pushing up to production. I'm just prototyping. Who cares about performance.
Then you start to get your first customer or your first user. Then your second and your tenth. Now, okay, actually, performance is important. We want to look at SEO and performance is important. Then you can just grab the bundling and pull it in at that moment of it solves a problem for you.
It's really about, like, "I have a problem and now let me pull in the thing to solve it." It's a very different model.
Jason: It's the same way you treat any technical optimization like prerendering or these types of things. It's like, are you going to do it from day zero? It doesn't matter whether your prototype is prerendered, probably. And so, why inherit that complexity by default?
Also, there was a thing that had gone around.
Dave: Yeah.
Jason: It was like 3,800 modules downloading benchmark of things saying, like, "Oh, if you're using native ES Modules, it's going to be this much slower." But I think the point that that missed and the point that you kind of reminded me of there, Fred, is if in development you're watching your module graph download and you can see the performance of that and you realize that, "Oh, hey, I'm working on this modal dialog, why am I waiting for this other modal dialog to download that's not open?" That's the kind of thing where, as a developer, it's relatively intuitive and relatively easy using native dynamic import syntax to fix that problem for yourself.
It just so happens that, even if you do then go and bundle in production, that becomes a split point and your application is better as a result. It's like this idea of having production only ever be sort of like an optional optimization of the graph that you already worked with and optimized, I think there are so many ways that that plays out favorably that we have yet to even explore.
It's not even necessarily a fully technical thing, right? It's like a social thing or like a gamification type thing. Right? I'm building a mail client right now and using WMR as the dev server for it. It's interesting to see how many more cases I've just quickly and (without really thinking about it) ended up using dynamic import just because it was there, because it was native. I didn't have to think about chunk names or any of those kind of stuff. It was just like it's the other import. [Laughter]
Fred: Right, in development. The network panel actually has information like--
Jason: Who loaded it.
Fred: "Oh, that file is loading. How fascinating. I can...."
Jason: Give me a stack trace.
Fred: Exactly.
Chris: Hmm.
Jason: That's a thing. [Laughter]
Chris: Mm-hmm.
Fred: [Laughter] Versus the black box of Webpack.
Dave: You don't even the source map garbage. You're just like, "I'll do that later, like when I want heartaches."
Jason: I don't use source maps at all anymore.
Dave: Yeah. Wow.
Chris: Hmm. Oh, wow.
Dave: Okay.
Jason: For anything.
Chris: Let me throw one at you that I think would be relevant to Dave, too, because Paravel, you might roll into a place and make a design system, right? There's a folder and it's got a bunch of components in it. Then those components are going to have some opinions about them, like maybe they use CSS Modules, or maybe they're written in TypeScript, or something. But the consumer of those modules might not share those same opinions.
If you bundle them separately, doesn't that kind of like open up some possibility for how they're consumed elsewhere or something? Is that part of the story in some way? For example, at CodePen, we're like, "I'm going to spin up a NextJS app," but Next has its own Webpack config. You can tap into it and add stuff to it. I think it's pretty clever how it does that. But it doesn't share the same opinions that our existing components do elsewhere, so we're in this kind of mono-repo situation where one part of the app can't -- it can, eventually. We'll figure out a way to smash it together to get them to understand the components elsewhere, but they don't -- if they were bundled already, it wouldn't matter. I just make them.
Jason: That is a double-edged sword though, right? One of the things that we learned from the NPM, the massive-distributed test that is NPM running at scale is that when you bundle, you lose fidelity. When you bundle a JS Module that references CSS, it's usually in-lined. Trying to get it back out, that's hard.
Chris: Yeah, I can't.
Jason: [Laughter] That's partial evaluation of JavaScript bundles, which is tricky. And so, the purist in me really, really wants to answer this question by saying, "We need to have a syntax for importing CSS."
Fred: Yeah. There's really no good way to recreate it either. We've tried in Snowpack, but that idea of importing CSS is so common that we want to support it, and so we do. But it's like--
Jason: Made up. [Laughter]
Fred: There's no way to really do that and still have CSS. Yeah, it's a made-up construct, but there's no way to recreate it. It's really difficult to recreate. You kind of need a browser primitive to do that without turning the CSS into JavaScript, at the end of the day.
Jason: CSS imports in JavaScript are to JavaScript what CommonJS was to ES Modules.
Chris: Is that what The Goog is trying to do with Constructible Stylesheets, or is that...?
Jason: I think it's part of it. Part of it is if we're going to introduce CSS imports into the module graph, there needs to be a type that gets returned to you from the import. It can't just be like a random string of CSS. But the actual, the larger part of it is something that I can never remember the correct name of. There are import assertions, which is like import foo from bar with type colon JSON.
Chris: Oh, my gosh!
Jason: Yes. Still proposals moving forward.
Fred: Still proposal, right? This doesn't actually exist....
Jason: The import assertions, all that does is say "error" if this is not a JSON mind type. It's just a safety precaution. It does not allow you to transform the source code.
Chris: Okay.
Jason: There is a bullet point in the spec proposal that says, "In the future, it would be interesting if we had another thing that wasn't import assertions," I can't remember what it's called, "that allows for transformation hinting to say, like, 'Import foo as bar.css with type CSS.'" I think it's like using transform:css or whatever. Maybe some terse version of that syntax ends up landing and becoming the thing that results, but you would get a value back from that that is language defined or spec defined type. That way every bundler and NPM and module author can just take it for granted, finally, instead of having to have a bunch of Read Me that say, "We're using CSS Modules, so make sure you do whatever that means in your bundler set up, if you have on."
Chris: I know.
Jason: Any time you have to go to a Read Me to figure out how to import a package, that is broken - period. [Laughter]
Dave: Let's say I'm going to release a Web component next week and I want to do my best 2021 JavaScript I've ever done. What do I do? What do I need from a module perspective?
Chris: It's NPM bound?
Jason: Going NPM?
Dave: Yeah. I mean yeah, probably. I mean that'll be the first issue is, "Why is not on NPM." So, I don't really care, but that's the first issue.
Fred: Yeah, I spend so much time working on Web apps these days, I'm actually curious to hear Jason's answer because we're talking about a package mode for Snowpack, which funny enough is essentially what Pikapack was, if anyone ever played around with that project, which is ancient history by now. But I actually am trying to find out the same thing myself.
Chris: Isn't the answer just ship ESM and don't bundle anything - kind of, sort of?
Jason: As long as you specify in your package.json that it is ESM. You put type module in there just as a way to hint obviously to Node, but also to bundler, to say, like, "Don't assume that this is just CJS," which is still definitely--as per the package.json thing that isn't a spec--that is what .js files in a package are.
I think there is a group of people. Maybe this prototype Web component would be an example of this where it's a package that got released today and so there's really no reason to expect that it supports Node 6 or IE11.
Dave: Yeah.
Jason: Maybe that's the case where you put the hint in the package.json. You put in type module. You put in an export map that just says export colon the path to the file. You don't transpile it. You just write it as modern code and you ship that.
The expectation is: you ship it, it will work in bundlers and configurations that can support the code that is contained in that file, and it will just die in other configurations. That's the correct behavior. Somebody who has the transpile.
Dave: Okay, but it uses LitElement, right, so don't I need a bundler at some point to offer a bundle at some point?
Jason: Oh, so that would be the secondary use case, though. That would be if you want to support Webpack and Rollup in a way that doesn't require everyone to specifically add configuration to support this module, then you would use your export map to say, "Oh, there's a browser entry over here." You would set the main and/or browser field in the package.json to say--
Chris: Oh!
Jason: --"If you're a bundler that doesn't understand export maps, here's the legacy JavaScript." You would sort of choose what your lowest common denominator is.
The thing that I tend to do for this is, you have an export map and it points to the modern code. I write in 2017 just because it has good browser support, it has good syntax, and I'm not using bleeding-edge features that are going to break in, like, recent Firefox. Then I transpile once to UMD ES5. That is, so I have highest common and lowest common denominators, and I don't ship anything in between.
Dave: Okay.
Fred: I would say that there's another kind of flip here that's happening, which I think, early on, kind of like last decade, there was a big conversation about do we bundle our dependencies or not. It's going to take so long. Do we do it?
What Snowpack is definitely looking at now (and Skypack even as a way to host these) is like, that's the only thing we're bundling. Or at least that's the thing we always bundle, so it actually brings in this assumed tooling step to a dependency where it's been a weird kind of compromise with the package author having to decide which users they support.
Moving into this model, the package author, their job is just to support the latest standard. So, if you're doing something that's really experimental, compile that down. But that means ES2020 is essentially where you can go as a common language. This is agreed upon. It's standard. Then you're entering a world where everyone is taking that and deciding what they need out of that. That fidelity question, you can always go down and lose fidelity for your site but the package author is now the highest. It's the most modern it could be and everyone who is building an app, it's their responsibility to turn that in to what they need based on their constraints.
Jason: I think, to that end, in your modern output, in the thing that you're pointing to from your package exports field, generally, nowadays, I think the assumption would be that you would not pre-bundle your packages' dependencies because the expectation is either somebody is using tooling that doesn't bundle dependencies and they want unbundled dependencies (for something like Skypack or whatever) or they are using tooling and that tooling is bundling dependencies anyway, so it will be smart about that. Then maybe you would consider pre-bundling dependencies for the UMD build if you're saying, like, "This is the thing I'm going to load on unpackaged for browser support target."
Dave: Yeah. I think it's just, you know, back in the day it was like, "Just UMD. Paste this stuff in. Call it a day. It's done. It's out." You know? Then it's like, "Okay. Cool. Now everything has to be an import and an export." You're like, "Oh, gosh. Okay. Let's write all that stuff."
But then it's like, "Cool. I know how to use that, but I want the world where people," like you all are kind of saying, "where it's just easy. You can just pull these things off of a shelf, off of a Skypack shelf, and say, 'Paste this in. Use it. Does it work? Wonderful.'"
Jason: We've talked. Okay. Keep going.
Dave: Oh, just like LitElement. It's very simple. It's 2k. Whatever. Not a big deal in the grand scheme of things, but it introduces a whole level of tooling where I don't know. I'm just going to ask for LitElement. I don't know where you're getting that from. You may want to get it from somewhere else, but I probably need a version that has that in there.
Fred: Yeah. We've been talking a lot about how Skypack is this cool thing that Snowpack leverages, but Skypack, on its own, is awesome. You can just (in whatever browser you're in) open up a dev tool like the dev console, right? Import the package you want at our URL and it gets you that thing, so it's that same UMD CDN world but without having to worry about all this, like, okay, going to the Read Me. How do I import this? Do I need this flag or do I not?
It's just like, package name, you get in the browser, you can develop with that. You can prototype with that just in the dev console. It's on its own. If you already have kind of like given up on the world of tooling, it's a way to kind of reenter the NPM ecosystem but without that baggage.
Jason: It makes new assumptions that weren't available before and it so happens that if you make those assumptions, the experience is net easier than the old UMD one. There are no globals to guess and that kind of stuff. But again, you have to be using something like Skypack that will make those assumptions.
A common thing we keep running into with PReact is people try and use the unpackage module query string parameter, and for some cases, it totally works fine, but it doesn't make all of the assumptions that Skypack is making, like it won't follow export maps and it doesn't do version pre-resolution and the fancy nested import statements for version resolution. And so, there are a bunch of cases where, without those assumptions, it all falls apart.
I think that's where we run into those cases where it's like, "Oh, crap! I need to understand way more about this brand of import statement than I ever cared to understand." [Laughter]
Dave: Well, and that's probably just the balance of, like, I'm going to be slightly dumb about this code because I don't want to interfere with it too much. Yeah. Yeah.
Chris, did you have something?
Chris: No. I have one dumb thing but it's not worthy of ending this excellent conversation with, so let's call it. That was really enlightening to me. I feel validated that ESM or JSM, which nobody uses as an acronym--
Dave: Jason Santa Maria. [Laughter]
Chris: Yeah.
[Laughter]
Chris: It really was the tie that binded these tools and stuff and that's going to change. I think--whatever--2023, 2024 is going to be even extra weird. I mean we probably won't even talk about it. It'll just be the assumption because all momentum is headed that way. There's nothing trying to unseat this.
Dave: Yeah. Well, cool. Well, thank you all so much. Monster episode. I'm sure we'll call you again if we need module help.
[Laughter]
Dave: But Fred, Jason, for people who aren't following you and giving you money, how can they do that? We'll start with Fred.
Fred: I'm on Twitter and GitHub @FredKSchott. If this is all exciting, give Snowpack a try. It's at snowpack.dev. And Skypack is on skypack.dev. You can find me in any one of those places and we're always looking for new contributors, so please get involved however you can.
Jason: I'm on Twitter @_developit, on GitHub @developit, and WMR is preactjs/wmr.
Fred: And your cash tag for Venmo....
Jason: Oh, yeah. Yeah. Send all my money to Fred.
Fred: --send your money....
[Laughter]
Dave: There we go. All right, well, thank you all so much. This is awesome.
Thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to star, heart, favorite it up. That's how people find out about the show. Follow us on Twitter, @ShopTalkShow, for tons of tweets a month.
Chris, do you have anything else you'd like to say?
Chris: ShopTalkShow.com.