Comparing Sitecore 8 to Sitecore 9 Over a 56k Dial-Up Connection

When was the last time you used a modern website or web app over a slow connection? What was the reason for your slow connection? Maybe a crowded Wi-Fi network at Starbucks? A flaky DSL connection at a remote client? What about a 56k dial-up connection?

If you're used to working with Sitecore over a fast connection (local LAN or high-speed connection, say, 150 Mbps), you've certainly never considered what it would be like to use Sitecore at a fraction of those speeds. I was curious what it was like to use the Sitecore Client over a slow connection, so I dialed back to the days of dial-up and loaded up Sitecore 8 and Sitecore 9. It was... interesting.

The Setup

Using two fresh, out-of-the-box instances of Sitecore (versions 8.2.5 and 9.0.0) on my local developer machine, I loaded the Sitecore Client (example: http://sc90.local/sitecore) in Chrome. I used the Network tab in Chrome Dev Tools to simulate various connection speeds, in this case, Fiber (150 Mbps) and Dial-Up (56 kbps).

Physical hardware can greatly affect Sitecore performance, so I'll be candid about my developer hardware at a high level: latest Windows 10 Pro, Intel Core i7 4800MQ processor, 32 GB memory, and fast SSD storage. Nothing about this setup throttles Sitecore for the sake of this testing.

Slowing Down Sitecore

Let's clarify the actual networking speeds that I used for testing.

The big takeaway: dial-up has just 0.0003% of the downstream bandwidth of a 150 Mbps fiber connection.

I used Chrome's network simulator to create 150 Mbps and 56 kbps speed tiers.

The Results Are In

For these tests, I warmed up each Sitecore instance's worker process, cleared my browser cache, then navigated to each of the following pages / interfaces in order:

• Login
• Dashboard
• Desktop
• Content Editor
• Experience Editor

All in all, these are pretty simple but common Sitecore interfaces. Content Authors will spend most of their time on these pages, so it gives a good baseline for comparison. Important note: I used the "Disable cache" option in Chrome Dev Tools, so each page load was fresh from the server. This is a worst-case scenario - in reality, aggresive browser caching means most Sitecore Client assets would be loaded after a few uses, and subsequent page loads would be relatively quick.

There are a few other variables that can affect load time despite network speed, so I'll call out the obvious. Database performance can greatly affect end-user experience due to server load and network latency. Using large, detailed images for the Sitecore Client backgrounds will load far slower than a solid color background. Because I used out-of-the-box installations, I left all defaults alone.

The Internet in Megabytes

To help provide some scale and context, here are the number of requests made for each page and the overall bytes downloaded:

Sitecore via Fiber

Page load times in Chrome:

Sitecore 8.2.5 on a 150 Mbps connection.

Sitecore 9.0.0 on a 150 Mbps connection.

A few takeaways:

• Most pages and interfaces load a tad bit slower in Sitecore 9, but not slow enough to make a perceivable difference in most situations.
• Sitecore said that the Experience Editor became a lot faster in Sitecore 9 - and they were right. Raw numbers don't lie: requests are down, the overall payload is smaller, and loading the Experience Editor in 2 seconds is a vast improvement over 32 seconds.
• Loading the Experience Editor with the default, out-of-the-box Sitecore welcome page does not simulate real-world usage. Most websites are going to have many more components, styling, and scripts to load, hence longer load times for real users.

Sitecore via Dial-Up (56k)

Page load times in Chrome:

Sitecore 8.2.5 on a 56 kbps connection.

Sitecore 9.0.0 on a 56 kbps connection.

Sitecore just falls apart on dial-up.

Notice the scale of these charts (maxes at 40 seconds on fiber, 400 seconds on dial-up). This isn't Sitecore's fault, but instead a stark reminder that the whole internet was once this slow. How did we tolerate such slow speeds? Pages were once measured in dozens of kilobytes, not several megabytes, so they were much easier to fit through such a narrow pipe.

A look at the Network waterfall in Dev Tools shows a lot of pending requests:

At first I used this tab to gauge network activity (which appeared to stand still many times), but I soon realized that I had to be patient. Really patient. All assets eventually loaded, but it was slow. Remember that 56 kb = 7 KB, so any file larger than 7 KB takes at least a full second to download. Multiply that by pages that weigh an average of 2 MB (roughly 2,048 KB), and waiting around is the norm when using the Sitecore Client on dial-up.

For both 8.2.5 and 9.0.0, the Experience Editor didn't even finish loading on dial-up:

Waiting... waiting... waiting...

After waiting for activity to calm down in the Dev Tools Network tab, I investigated and learned that JavaScript on the page had timed out (among a few other errors due to the slow speeds):

Our JavaScript is faster than our connection.

I'm not going to dive into JavaScript debugging for this post, so I'm making some assumptions about what's going wrong and blaming it on the slow connection.

Final Thoughts

Anyone who accesses the Sitecore Client is probably a Content Author or Editor, and is probably behind a firewall, and is probably at work - on a high-speed LAN or WAN connection. Is Sitecore capable of working over a dial-up connection otherwise? No. Sitecore is a CMS for our modern networking.

Beyond the fun of seeing a 2017 web app operate over a 1997 connection, what can we learn from this execise? While seeing some performance improvements to Sitecore 9 are obvious and welcomed, being aware that Sitecore might struggle on slower connections is still important. Imagine a scenario where a Content Author logins in over a series of netowrking choke points. He/she is remote and uses their mobile phone with a 3G connection (strike one) to connect to an office VPN (strike two). While not exactly dial-up speeds, there's a good chance that Sitecore might fall over for the Content Author.

Do you have questions, comments, or corrections for this post? Find me on Twitter: @BrandonMBruno