GOV.UK is the online home for the UK government, so we have to make sure that it works for everyone regardless of their device or connection speed. This means we’re always looking for ways to improve performance and user experience, as well as the code beneath.
For a long time our frontend code on www.gov.uk - the publishing domain for GOV.UK - has been dependent on an outdated version of jQuery, a library designed to make writing JavaScript easier. Here’s how and why we decided to remove jQuery, and what the results have been.
How we used jQuery on GOV.UK
GOV.UK consists mainly of static pages of content, so we use JavaScript mostly for analytics, cookies, and small user interface interactions like expanding and collapsing elements.
GOV.UK’s architecture is quite complex and our JavaScript is spread across multiple applications that each serve different types of pages. Like many large websites, this JavaScript is a mix of new and old, well documented and not so well documented.
jQuery was used widely across our applications both in public facing scripts and test files. We relied on a number of jQuery features, notably Ajax handling (to reload parts of the page dynamically), but we were generally using it to make our code simpler.
The main disadvantage of using jQuery is that it has to be included in the page assets, which can slow down how quickly a site loads. GOV.UK's pages vary in size, but the homepage is around 246 kB - 32 kB (13%) of which was jQuery. That’s not a huge amount, but it’s worth considering the performance impact, especially for users on low-specification devices.
The scale of the problem
GOV.UK was reliant on an old version of jQuery. We knew that we needed to upgrade, but we also knew that jQuery was causing performance issues on some parts of GOV.UK.
We decided that rather than upgrade, it would be best to remove jQuery. This was a huge task. We had at least 200 scripts, some of them only a few lines long, others hundreds, plus many more tests that also used jQuery.
While it was a large task, it mainly consisted of relatively small chunks - any piece of JavaScript on GOV.UK could have jQuery removed from it as an isolated piece of work. That meant that developers could contribute to the task even if they only had a few hours available.
How we did it
Firstly, we stopped adding any new jQuery to GOV.UK to stop the task getting larger.
Secondly, we spread the effort across the developers on GOV.UK - both frontend and backend. Backend developers on GOV.UK don’t normally write a lot of JavaScript but they recognised the importance of the work and with a bit of training and support they were soon making great progress.
The work still needed input from frontend developers for some larger and more complex JavaScript assets but after writing some documentation on how to remove jQuery and a bit of developer pairing we soon found that entire applications were becoming free of jQuery. This extra effort made a huge impact - it probably reduced the length of the project by several years.
We also made some hard decisions. Two of our applications contained a large amount of jQuery but don’t provide many public facing pages to GOV.UK. Rather than block progress, we gave these applications their own copy of the library, so that the majority of GOV.UK could still be made jQuery free.
This was punctuated by our COVID-19 work and other priorities, but by the start of 2022 we were in a position to start considering options for finally removing jQuery from GOV.UK.
The final push
We created a development version of the site without any jQuery that we could use to test each application. This allowed us to check for errors caused by any jQuery code that we might have missed, as well as provide the first real opportunity to measure the performance impact of our change. This testing was thorough and time-consuming and happily revealed few problems.
In late March 2022 we were finally ready to deploy our change to the live site. This required careful timing to first deploy those applications that received their own jQuery. Although including jQuery twice didn’t cause any errors, it did unnecessarily inflate the asset size for users and we were keen to not have that in place for long. Once those applications were deployed we were able to deploy the change to remove jQuery - a successful if nerve-wracking process.
The result
Removing jQuery means that 32Kb of JavaScript has been removed from the majority of pages on GOV.UK. GOV.UK is already quite fast to load and for many users this will make no noticeable difference. However, the change for users on a low bandwidth connection or lower specification device will be much more noticeable, resulting in significantly improved page download speed and performance. We’ll provide more detail about exactly what that means in the next blog post.
From a developer perspective removing jQuery has been a long but worthwhile process. Rewriting our code has taught us a lot about it and we’ve expanded and improved our tests. In many places we’ve been able to restructure and improve our code, and in some cases remove scripts that were no longer needed.
What initially started as a side-project to remove tech debt and legacy code has also brought added benefits in performance, security and developer familiarity with our JavaScript code. An added bonus is that we’ve also removed an external dependency from GOV.UK, which reduces our maintenance effort and possible future security risks.
If you’re interested in this work, why not join the team? Apply to work at GDS
33 comments
Comment by Linda posted on
What solution did you replace the jQuery scripts with, where the functionality it provided needed to be kept?
Comment by Andy Sellick posted on
Hi Linda, thanks for your question.
Since jQuery is written in JavaScript, it doesn’t do anything that you can’t do with normal JavaScript. Having said that, jQuery does make writing JavaScript a lot easier and we often had to add extra lines of code and in some cases polyfills for newer JavaScript features that aren’t supported by older browsers. Each JavaScript file also presented different challenges - some scripts were full of jQuery functions, others had only a handful.
Thanks, Andy
Comment by Mart posted on
Hi Andy, you mentioned adding polyfills to support older browsers. Which browsers does GOV.UK target? Are you still supporting the terribly outdated IE11?
Comment by Matt Hobbs posted on
Hi Mart,
So we recently stopped supporting IE11 as of the 16 June (https://technology.blog.gov.uk/2022/06/16/service-manual-testing-requirement-changes-for-internet-explorer-11/). We’ve been monitoring IE usage on GOV.UK for many years, and since Microsoft is pointing users towards Edge, and our usage of the browser dropped to less than the 95% we follow in the Service Manual (https://www.gov.uk/service-manual/technology/designing-for-different-browsers-and-devices). This seemed the perfect opportunity to make the commitment and help move the web platform forward. The Accessibility team also updated their guidelines for public services to coincide with this date too (https://www.gov.uk/service-manual/technology/testing-with-assistive-technologies)
I hope that helps answer your questions.
Thanks, Matt
Comment by Boris posted on
you need to remove google analytics, recaptcha anything like that
Comment by Matt Hobbs posted on
Hi, Google Analytics will only load if a user accepts the GOV.UK Cookie Banner (https://www.gov.uk/help/cookies).
In terms of reCAPTCHA we only know of one service currently using it, as by default we try to avoid the page weight it adds and the complexity it adds to our users' journeys. As far as we know it isn’t being used on “www.gov.uk” even for the feedback component on each GOV.UK page. This component was recently updated to make sure it was accessible and is progressively enhanced so works when JavaScript isn’t available. You can read about that work here: https://insidegovuk.blog.gov.uk/2022/03/28/making-the-gov-uk-feedback-component-more-accessible/
Comment by Lloyd Hardy posted on
Why is the UK Government is passing user data to Google, a US company which moves that data outside GDPR protection, instead of using our own analytics software?
You should not be running Google Analytics on GOV.UK.
Comment by Deyan posted on
This is a neat way to spend tax payer money on minor improvements requiring significant effort.
Yes, the time for first load of the homepage has improved by several seconds. But given that the static resources like jquery js file are cached in the browser, the improvement is not really noticeable.
Also, 32kb are loaded for 4-5 seconds using a 56k dial-up connection, which nobody uses nowadays.
If you refer to a 256kbps dial-up as a low-bandwidth connection, than the work you have done is even more pathetic - 1 sec load time reduction.
Now remember that the js files are cached.
Perhaps you should spend your expensive engineer’s time, paid by the government, for something more useful?
Comment by Peter Rattew posted on
They only talk about the performance gains they get from removing jQuery when the security improvements are much more important. jQuery is a massive target for exploits and removing it was a smart decision.
Comment by Mike Jones posted on
"...a massive target for exploits..."
Care to provide any examples?
This is such a prime example of government ineptitude and waste...
Comment by Paul M posted on
Hello Mike,
I read in another blog post, they were using v1.12.4 of jQuery.
If you go to a website which lists vulnerabilities within packages such as Snyk, you can see there are 4 medium ranked vulnerabilities mainly relating to cross-site scripting.
I say good job on the removal of jQuery, the documentation and process will surely be helpful for others in the same situation.
Comment by Peter Rattew posted on
jQuery is used on 77.4% of websites, making it a massive opportunity for black hat hackers and as Paul M noted, the version of jQuery they were using had four known vulnerabilities.
https://w3techs.com/technologies/details/js-jquery
https://snyk.io/test/npm/jquery/1.12.4
Comment by Anon posted on
>"...a massive target for exploits..."
>Care to provide any examples?
>This is such a prime example of government ineptitude and waste...
Jumping to a conclusion before any examples are provided, well done 👏
Comment by Brian Morgan posted on
Interested to know what you actually replaced the jQuery scripts with?
Did you write specific inline JavaScript for each replacement, or use some other library
i.e. Replace jQuery with perhaps your own library of JavaScript
(seems the most logical way)
Did you drop support for older browsers that jQuery might have given 'freebie' support for?
Comment by Andy Sellick posted on
Hi Brian, thanks for your question.
The short answer is that it varied depending on the script. In some cases we were able to directly replace jQuery functions with their JavaScript equivalents - for example replacing jQuery’s find() [https://api.jquery.com/find/] command with document.querySelectorAll() [https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelectorAll]. In some cases we had to write our own equivalent code for reuse in multiple scripts - for example jQuery’s extend() [https://api.jquery.com/jquery.extend/] function. And yes, in some cases we had to make the hard decision to drop JavaScript support for older browsers that jQuery supported out of the box - but we always made sure that there was a non-JavaScript fallback in place so that users are still able to do what they need.
Comment by Abhi Markan posted on
A brilliant and commendable effort!!
Are there any plans for Gov.uk to remove console statements from their libraries?
Comment by Andy Sellick posted on
Thanks Abhi. I’m not sure what you’re referring to although we do use console.error() in a few places to provide feedback via our Real User Monitoring where scripts unexpectedly fail, in order to identify and resolve problems.
Thanks, Andy
Comment by Abhi Markan posted on
Hello Andy,
Thanks for your reply.
I am referring to `console.log()` statements in Gov UK Frontend JS map.
Console.log has been used through-out and was wondering if there are plans to replace them with console.info() or console.error() so when deploying the code on non-dev env any console.log() statement are removed including from the libraries itself?
Thanks
Comment by Matt Hobbs posted on
Hi Abhi, thanks for your follow-up reply, would you be able to give a little more information, so we can track down the issue for you? Can you tell us what Browser you are using and Operating System, then finally the URL(s), you are seeing the console messages on?
Thanks again for your patience and helping the GOV.UK team fix the issues you have spotted.
Thanks, Matt
Comment by Matt Dendura posted on
Possibly found a bug on this blog site. Clicking the link for a GOV.UK comment author's name always seems to link to https://insidegovuk.blog.gov.uk/author/laurastevens/.
Comment by Laura Stevens posted on
Hi Matt, thanks for the message - it's not a bug. They link to my name as I edit this blog and manage the comments. Thanks
Comment by Nige posted on
I'm not the OP, but this is clearly a bug. If I click on Matt Hobbs' name I would expect to be taken to his bio or perhaps a list of all articles written by Matt. Instead I see your page. It makes no sense.
Comment by Robert posted on
You spent money to save a 32k transmission? That's less than 4.5 seconds on a 56k dialup line. How many people still use 30 year old slow technology like that? And if you did it right, it only downloads once. Seems like a big waste of tax money to me.
Comment by Sergei Startsev posted on
Thank you for sharing the results. Did it take about 3 months to get rid of jQuery? How many developers were involved? There're still a lot of sites that depend on jQuery and this information would be really useful for estimation projects like this.
Comment by Andy Sellick posted on
Hi Sergei, thanks for your question. It’s hard to estimate exactly as it was spread across three years and often paused for other priorities. We started by listing all of our scripts to get an idea of the work involved, then working through them application by application, estimating the complexity of each and assigning them to specific developers based on experience.
We had around 200 scripts to remove jQuery from, varying in length from a handful of lines to several hundred, which resulted in something like 69 pull requests. This also included reformatting the structure of some of our scripts (a problem specific to us, due to the way we load our JS) and updating test files accordingly.
Most of the effort was in the final year and was done by around 8 developers (a mix of frontend and backend devs), which involved some training, pairing and writing documentation to help with common problems (https://docs.publishing.service.gov.uk/manual/how-to-remove-jquery-from-javascript.html) as well as relying on external resources like https://youmightnotneedjquery.com/.
Thanks, Andy
Comment by Sergei Startsev posted on
Hi Andy, thanks a lot for the provided technical details. It's very useful for teams that are just planning this journey.
Comment by Rob Schoenaker posted on
I couldn't resist to check how many bytes would be transferred for images on this specific page. It turns out to be 39 kilobytes. Removing the images would have been 1 minute of work.
Also, most modern browsers cache jQuery altogether and with that it would only be an initial hit.
End result as I see it:
- time wasted
- money watsed
- more difficult JS code now than with jQuery
Job well done, I suppose 🙂
Comment by Moon Quddus posted on
You're forgetting the removal of an old version of jQuery makes the site far more secure. And working on tech debt is never a waste of money in the long-term 😊
Comment by Caleb Webber posted on
I'm not sure why you would assume the code without jQuery is more "difficult" than with jQuery.
What do you mean by difficult?
If by cognitive load, consider that most are shifting away from jQuery. I don't know anyone entering the industry today that even bothers learning it. I think this is a great move for the maintainability of the site.
As for time and money wasted, I would consider the immeasurable time saved for future engineers that have to maintain this site not having to learn legacy JS frameworks.
Comment by Mike posted on
Hi,
I’ve noticed recently, probably since jQuery was removed, that every page I visit on gov.uk displays a cookie acceptance message. It’s infuriating! Even visiting a couple of pages prompts the cookie message each time.
Comment by Andy Sellick posted on
Hi Mike, thanks for your comment.
The removal of jQuery shouldn’t have changed anything to do with cookie consent. The reason why users have to accept cookies more than once is because although GOV.UK looks like a single site, it is made up of several separate domains (you can see these listed on the GOV.UK proposition - https://www.gov.uk/government/publications/govuk-proposition/govuk-proposition#govuk-domains ). Currently, we do not have cookie consent shared across the domains.
Thanks, Andy.
Comment by Cliff Tyllick posted on
Andy, I have long advocated for website to do precisely this. It isn’t just people on dialup modems who are affected. It’s anywhere that service is spotty. It’s anywhere that bandwidth is low. And, as you pointed out, it leaves you in control of the performance of your own site, including accessibility. I once worked for a Fortune 10 company that could not remove a significant barrier to accessibility on its own site—the placement of initial focus in a modal dialog—because it was established in a Javascript. Fixing the Javascript would improve all sites relying on that script, but the fact that it was impossible to test the full scope of the impact led IT to reject the proposed change out of hand. In principle, there is always that danger with widely shared components: a change that improves it in one setting might break it in another. As in my case, that the danger exists in principle is often enough to stymie an effort, even when careful consideration reveals that in this instance there really is no danger. So as important as loading times are—and to people working online today, 4.5 seconds is an eternity—this change has added value in other ways as well.
Comment by Felix Ho posted on
Removing jQuery in the name of "security" is pretty much misleading, unless you hate all the extra work to check your existing JavaScript code with each jQuery update.
Also, it is definitely not a page-speed issue, as the main problem these days are just "bloat" images. Removing an extra 88KB from "the list" will not earn you a better score from Google.
Just admit you hate jQuery, and we can live with that.