Response to DotNetRocks Interview Comments
Last Updated -
In case you missed it I was on Wednesday's DotNetRocks talking about web performance optimization. It was a great conversation with Carl and Richard and I wish we could have talked for a day or more.
The episode has received a few good comments so far. I wanted to take some time to respond to Tim Clark's comment here on the Blog because my response is not going to be succinct. His comment echoes common push back points I get from enterprise developers all the time. Interesting, web developers (these are developers that are not back-end developers by teade) embrace my position, which relates to the full stack comments I had in the interview.
My primary point of view is that our web applications main purpose is to engage users and solve their problems. This is the essence of user first development. Unfortunately the average developer follows a developer first approach, letting the user suffer.
Before I get into the points and responses let me give you a little background on where I am coming from. I loved jQuery, still do, but I do not use it nor have for about four years. I have even been to a jQuery conference and have been fortunate enough to foster a friendship with Dave Metvin, president of the jQuery foundation in recent years.
jQuery gave us exactly what we needed a decade ago. Since then the browsers have fixed the issues jQuery was created to solve. I like to say there are about 3 dozen APIs every web developer should know and you can do about 99% of what you ever used jQuery to do.
I do like jQuery's utility, but the size is overwhelming, larger than most full applications I build these days. So one Saturday afternoon I built dollarbill, a jQuery replacement. There are other similar libraries available, Zepto and ShoeString come to mind. Then there are sites like You Might Not Need jQuery, You Might Not Need jQuery Plugins and of course VanillaJS. There are other, more efficient options available that jQuery.
Why did I stop using jQuery? I had to start building web applications that loaded quickly on mobile devices. Loading over 3G and a mobile device can be painful if you load too much JavaScript, CSS and images. Waiting 10-20 seconds kills your effectiveness. For example we know if your page has not rendered by 3 seconds almost have of the users have left. This is true on mobile as users expect pages to render as fast if not faster than they do on their desktop. In other words there is a lot of pressure on developers and architects to deliver user friendly experiences. Sadly 99% of the web fails and is getting worse. Speed is a competitive advantage, leverage it.
If you take time to visit the HttpArchive you can see how page weight and load times have been steadily increasing. Today the average home page of the top 250,000 web sites would not fit on a high density 3.5" floppy disk. If you dig deeper you will no doubt find the average URL on the Internet is much larger than the 2MB average now being reported by HttpArchive. Just like we are getting fatter and less healthy so are our web sites.
Tim's comments are of a selfish nature, and this is not meant to be mean spirited, but to set the perspective. His comments are all focused on the developer, not the customer. Here lies the main problem. Developers and architects need to realize no matter what level we focus our development skills it is about delivering a quality user experience that builds brand loyalty and of course revenue.
In a ThoughtWorks presentation on UX principles point #2 states the following:
If you evaluate the points Tim makes they are all about the developer and being able to hire what I would classify as unqualified developers. I interview candidates for clients and find it is hard to find someone that knows any of the core APIs. They usually answer 'you just use jQuery'.
This is not limited to jQuery, Angular, React, Ember, etc all encourage developers to only know the framework and not what is really happening. Once you learn the handful of APIs and adopt some basic modular architecture techniques you see how easy it is to create large applications with very little code. You also begin to see why the proliferation of fast food frameworks is bad.
Back to why I stopped using jQuery. The applications I was tasked with back in the 2010 time frame forced me to figure out a better way. Using tools like WebPageTest allowed me to see page bloat was killing me. Further investigation revealed jQuery was killing me and I was sad. But within a couple of weeks I was able to replace everything except AJAX. I soon switched to Reqwest and never went back to jQuery.
But there was more. I felt the pressure being applied by mobile native apps. Primarily in user experience. After evaluating what most native applications were doing I felt almost all the UX components and layout features could be replicated using HTML5 and CSS across all the browsers and platforms. In particular was the page transition. This led me to create single page applications.
At the time where was nothing to reference as a SPA best practice. So I created my own architecture relying on small, task focused libraries and good organization. This caused me to create SPAjs and now a View Engine pattern and library.
I have experimented with different rendering engines, binding and push. After multiple experiences I prefer push because it is super fast. Plus 90% or more of your rendering needs are read-only scenarios. The % requiring some sort of update somewhere else in the view are very rare. My push template of choice is based on John Resig's microtemplate function. It is about 20 lines long, not 120kb.
Using micro libraries does not mean I abandon good architecture and patterns. Just the opposite, it forces me to leverage them and understand what they offer and why they are being used. Nothing is implemented blindly.
For example, I do like the Flux model, but like everything else in the React ecosystem it is a mess. If I have a moderately complex UX to build I use a simple 3kb library called Callbacks. The library does the same thing as Flux without the overhead. Oh and React is slow and full of memory leaks. Please stop using it, you are just polluting the Internet and crashing browsers all over the world. More on that another day.
After that 'early' SPA experience I can routinely build complex and large SPAs with 100kb or less JavaScript. This is the minified file size, not gzipped, which is the size quoted by Tim in his comment. My JavaScript compresses to a similar size, for the entire application. This includes libraries and application specific controllers and services. I don't say that to brag, but to give you confidence you can accomplish similar. It is always good to work toward a goal knowing someone else has done it before you.
So with that background and point of view I am going to quote some points in Tim's comment and answer them.
Use the Google CDN
I hear this all the time, use the Google CDN. The problem with that is you are assuming the end user has already referenced and retained a local cached version of the file you are using. The reality is this is not as common as you think.
First the Google and Microsoft CDNs have become fragmented. New versions of popular libraries and frameworks are released all the time. Developers are lazy and never rev their version references, etc. This has resulted in a mess of fragmentation. Where sites reference different versions. This reduces the effectiveness of these CDNs.
Another misconception is browser caching characteristics. The reality is browsers purge files from cache all the time. There is NO GUARANTEE a file is retained, even if the target resource has a far future expires header. While desktops with vast amounts of disk space tend to retain references for a long period of time mobile class devices do not. They might not even cache the resource locally at all. When I started the mobile first approach iOS would not cache a resource larger than 25kb. That limit has been lifted, but the strategy is still in place. The browser selectively caches resources based on available resources. As resources become scarce the cache is the first thing thrown overboard.
As an example my iPad is routinely telling me I am out of disk space and yes my browser cache is relatively small. Think about having 10 versions of jQuery, 3 Embers, 4 Bootstraps, etc? Chances are they are not going to be retained.
In some cases the client browser never has the option to cache resources. Some corporations intercept requests with proxy servers and other network devices. They adjust the headers and possibly cache the resources in the network. This causes all sorts of problems with changing resources, hence the practice of cache busting.
You should always add a far future expires header so your resource can be cached. But never assume the resource is cached on the client. Always work with the assumption the user is accessing your site or application with an unprimed cache.
Finally using another domain name only increases the time to load by adding another DNS resolution and another connection. Each time the browser encounters a resource on a new domain it must resolve the domain's IP address, which takes time and adds latency to the load time. Opening an additional connection is also expensive and adds additional overhead.
CDNs are a good idea and you should certainly use them for your application's assets. But do not accept false premises about their benefits. Understand how HTTP works and how browsers manage connections. Survey how many versions of different libraries are referenced and accept the fact that your library reference may not be cached on your customer's device.
Time Spent Not Shipping
Another argument I hear is using small libraries keeps you from shipping code. Hogwash I say. Ask yourself how long does it take to learn the latest fast food framework du jour? A day? A week? A month or more? My personal experience is at least a month and even then the average experience is just hacking around with code samples picked from various Stack Overflow answers till something 'works'. The typical developer does not understand why something works or even if it has negative side effects.
Let me give you another real world example. Does it take more to cook at home or go out? Sure you can go to the drive through and order something but how long does it take to drive to your favorite fast food establishment? How about driving back? What about sitting in line with your engine running? Then what is the quality of the food handed through your driver's side window?
My preference is to stay home and cook some salmon, broccoli and maybe some rice. This takes 10-15 minutes. It costs about the same as the fattening fast food meal, but takes less time. Sure I had to go to the store and get the ingredients, but I did several meals worth in the trip.
Again what is the goal of the web application? To engage and solve the customer or user's problem. Performance matters in so many ways. Fast food frameworks only cause obese web sites. They appear to solve problems quickly, but instead they take as much if not more time to implement. Developers working only against fast food frameworks learn the framework, not the technology. It takes just as much time to learn the foundational technologies as it does to learn the framework du jour. And the history of client-side frameworks tells us that this year's favorite framework (like React) will be replaced with next year's fad. So invest your resources in learning the core APIs and understanding good architecture. Know the hows and whys and you will be able to build your code faster, maintain it with less effort and your applications will load and run faster. This leads to lower TCO and higher ROI.
Bringing Developers On Board
Its easier to bring new developers on board when you build against a fast food framework. I classify this as a collective hallucination. Is there a difference between finding Angular developers and JavaScript developers? Absolutely, but you can find either. In the end which will serve you better. I argue the one that understands JavaScript and the browser. Remember Angular was last year's fast food favorite, today it is forgotten and everyone has jumped on React. This just waste learning cycles and is costly to businesses trying to 'keep up'.
An engagement I recently had was an application that had layers like the earth's strata. There was the ASP.NET WebForms and a server-centric layer of a time long since past. Just above that was the jQuery layer. It was buried beneath the Backbone and Ember layers. Last year the company added Angular to the girth. This year React. The base JavaScript was nearly 2MB before the application specific code could even be loaded. I built a quick and dirty prototype of their entire application in about 20 hours, less than 100kb of JavaScript and about 1/10 the CSS. Again a quick prototype, but it recreated that majority of their user experience.
Now ask yourself is it easier to find a good front-end developer for your application or someone tied to a fast food framework? Is it good to just blindly adopt a popular framework because it is popular?
Don't look for butts to fill your seats. Find true assets to push your application forward. Put your customer first, not your human resources department's lack of technical knowledge. Technical hiring is not easy, but you do not need to settle. Hire web developers, not general practitioners.
Technology is Not Static
It took me a few times to grok what Tim meant by not static. His point is by using jQuery you can find a quick plugin to add a feature. The problem here is you are deceiving yourself here.
First there are thousands of libraries and components available on github, the node package manager, bower, etc. I can't think of the last time I grabbed a jQuery plugin to solve anything. The reason why is they are often not well written. They are written for a time when Internet Explorer 8 was standard. This means JavaScript animations and other choices that are not so healthy.
Another problem with canned fixes like plugins is they rarely meet stakeholder requirements. Have you ever expanded a calendar icon and you see a jQuery UI calendar pop-up. How often does it's style seem out of place? Almost 100% of the time. Companies tend to have brand guidelines for a reason. Even if they don't I am sure your stake holder has something else in mind as far as look and feel. This means you now need to spend hours if not days trying to hack and bang against the quick and dirty component to make it look and behave the way the client requires. When it is done you look back and can't help but think I should have just written it from scratch, it would have been easier.
I reference libraries, tutorials, frameworks and maybe a jQuery plugin to see how something can be done. Often I incorporate the core trick into my applications, often more succinctly. Last week I posted a tutorial about key-frame animations in response to a Polymer video. If you decide to use Polymer to animate your applications you are putting yourself at a demonstrable disadvantage in both performance and coding headaches.
So to answer Tim's point sure you could possibly find a jQuery that seems to work, but the reality is you most likely didn't. Open Source code is great to find how to solve problems. Take their lead and place it in your application's architecture for best results.
Summary
I appreciate Tim's arguments, but experience and research tell me he is wrong. Just survey your favorite sites, if they are not rendered in under 1 second for you then they are not doing it right.
Many corporate engagements and discussions with industry leaders combined with my real world experience tells me using fast food frameworks is not healthy. Developers and architects have been fooled into thinking their convenience is good. Hiring solid front-end developers and adopting a solid, modular architecture gives your application the best trajectory for success.
Applications are built for end users. The front-end is especially focused on user experience. Our development habits must focus on the end user's success, not solely on developer success. Sure we need to have good practices and tools to build solid applications. However fast food frameworks are designed to make back-end focused developers feel success in an area they have little to no real experience.
I am not alone, if you follow web development thought leaders you will find many articles, presentations and other media backing up my position. One of my favorite articles is one by Peter-Paul Koch comparing the front end and the back end.
Thanks for the comments Tim. I hope my response has been insightful. Do you have a counter opinion? Ping me on twitter or comment on the DotNetRocks page. I invite dialog because it only makes us all better.