Static over Dynamic

One reason I believe made Unix and Linux are very successful is their philosophy that “Everything is a file”. But the reason that this philosophy made them successful is files are relatively static than other things (Windows registry anyone?), in my opinion.

Another example would be for certain data that changes infrequently, simply put them in a static file and do a code change is much better than save in a database. Even though a code change and re-deploy is needed when change happens, but the performance and reliability is much better. A lot of time, I’ve seen people try to boost the performance by introduce caching at all levels thinking they get the good part from both: flexibility and the performance. Well what they got is complexity since cache invalidation is the second most difficult thing in computer engineering (right after naming things) according to Phil Karlton. One interesting story that I experienced first handed: One day, we accidentally stopped a service that has been running without any issue for 6 month. We thought, OK, let’s just re-start it. But it won’t, giving errors about cannot read a url. Turns out it is trying to do a one time load of some static data from an external team. The developer did not want to write a static file so he gave the justification that this is dynamic if the other team changed the data, we just need to restart the service. The other team had no idea this service depends on this url to start. There is completely no traffic on that and they removed it together with bunch of things 5 months ago.

Lastly, I have been trying to do a dev op work whole day today. I need to start a MariaDB Galera cluster. Our company’s setup script has evolved over the years from Chef to Puppet to finally Salt. I copied a set of working salt script to setup MariaDB Galera cluster from another team, but ended up reading everything in it try to understand what are the dynamic things it is doing based on host name, configurations, etc. I really wish that we are using Docker right now, because it is a file, it is static. Pull a image is basically copy paste a file. It is much much faster and guaranteed works.

Things I value in software/application design

Below are my personal opinions. I will explain some on them in details in future posts.

  • Proper design over hacking.
    • Not the security type of hacking, for the differences, read my post here. There is a Chinese old saying: “Think three times before you act.” My experience currently is that if the developer think half way through before he starts, it will be a relative successful project. And we give this all sorts of fancy terms such as “Agile”, “Bias for action”, “RAD”. I call it stupid.
  • Static over dynamic.
    • I mentioned this preference in my previous post here. Another example I want to give is when I debug some “fancy” code, it gives me a lot of headache when everything is dynamically picked. Reading the code will give you no clue what actually happens and you have to set break points and get dirty.
  • Separation over combination (which can be easier to use short term)
    • There are a lot of “powerful” library and frameworks out there. Their feature list is long and getting longer. But in my opinion, they should “do one thing and do one thing well” (Unix philosophy). And you should also keep this in mind when you write your code.
  • Simplicity over optimization.
    • “Premature optimization is the root of all evil.” —Donald Knuth. Need I say more? Actually one thing I would like to say is that quite often, you don’t even realize you are doing premature optimization. You are simply doing nature things as a well trained engineer, cache this, minify that, etc. I say you should question everything you do all the time if it is absolutely necessary.
  • Encapsulation over extensibility
    • Don’t get me wrong, inheritance and polymorphism are powerful programming concepts. Some libraries and frameworks leverage this and you can simply implement an interface or two to use them. However, I rarely see they get used properly when build internal systems. I see all the time an interface or parent class got one and only one implementer, which only makes the debugging experience horrible because you cannot get to the real code directly from the caller.
  • Configuration over convention (No I did not get the order wrong. I mean it.)
    • Convention over configuration is a software design paradigm advocated and embraced by a lot of people. It even has its own wikipedia page. I hate it, especially those “RAD” frameworks using this as an excuse to create tons of “dark magic”. The result is poor discoverability, hard to maintain code, buggy and hard to debug. An example I have to mention is a PHP framework called Lithium. Just don’t use it.

Can we have fixed versions for all the things?

Recently, we encountered some bower packages version conflicts issues. I believe the story is that we installed a new package called angular-touch. We installed it locally on our dev environment using the following command:

bower install angular-touch –save

The –save will add the package to the bower.json file. However, for some reason, we did not get a warning about that it needs a newer version of angular (1.4.3) than what we had (1.3.6). It may just silently updated it. Only when we committed the change and the build machine start to build from scratch, the issue showed up and we got his error:

Unable to find a suitable version for angular, please choose one:
1) angular#>=1 <1.3.0 which resolved to 1.2.28 and is required by angular-bootstrap#0.12.0
2) angular#1.2.28 which resolved to 1.2.28 and is required by angular-loader#1.2.28
3) angular#1.3.12 which resolved to 1.3.12 and is required by angular-resource#1.3.12
4) angular#1.3.16 which resolved to 1.3.16 and is required by angular-mocks#1.3.16, eMenu-web
5) angular#>= 1.0.8 which resolved to 1.3.16 and is required by angular-ui-router#0.2.10
6) angular#>=1.2.10 which resolved to 1.3.16 and is required by angular-carousel#0.3.12
7) angular#>=1.0.8 which resolved to 1.3.16 and is required by ngGeolocation#0.0.7
8) angular#~1.x which resolved to 1.3.16 and is required by angular-spinner#0.6.2
9) angular#>= 1.2.23 which resolved to 1.3.16 and is required by ngCordova#0.1.15-alpha
10) angular#1.4.3 which resolved to 1.4.3 and is required by angular-touch#1.4.3Prefix the choice with ! to persist it to bower.json

Here is something I really don’t like (that’s probably also why I’m writing a blog about it): Bower, like a lot other package management systems (npm, pip, maven, etc) allows you to specify version ranges. In this example, you can see notations “>=”, “<“, “~1.x” etc. Personally, I like all the things to be fixed. My thought is that deterministic is way better than random. That is probably why I like docker a lot.

I understand why they did this though. Because when your project depends on multiple libraries (let’s say A and B), they may in turn all depends on another library (let’s say C). If A want v1.1 of C and B want v1.2 of C, you got a conflict when A may work with anything greater than v1.1. So, to make it easy, they allow library writers to specify version range and must have some smart logic to pick one of the many versions that satisfies all the requirements (or just pick the latest that satisfies). It is only when they really cannot find anything to satisfies it, they ask a human like above. However in our case, it is too late because it is the build machine installing the packages.

Now, maybe it will be very annoying if we do not have version range and developers have to manually resolve conflicts a lot. Personally I feel much better knowing what exactly is my program using but others may not care and just want to use some library easily. Then I would ask this question: why don’t we have a platform can hold multiple versions of same package and let others use whatever version they need? Maybe it is very hard and need support from the programming language level. But in my opinion, it will be awesome. We will be able to encapsulate a lot better. Yes, the final size of your program may be several times depends on average how many versions of packages you included, but disk space and memory are getting much cheaper these days.


Dependency Hell