Explain WordPress as a house to a beginner

Today I met with a non-technical person who just started to use WordPress. I used a metaphor to explain to her what WordPress is and I felt that it is pretty good and I’m going to share here.

Imagine all the domain names out there are real estate land. The website we are building for the domain name is going to be the house on the land. You can build it from scratch by write HTML manually but just like if you build a house starting with cutting down some trees for wood, it will be very slow and ugly. WordPress is like a mobile house that is already built, you just need to tow it onto the land.

After the house is in place, you need to pick an existing overall look and feel of the house. This is picking the themes of the site. Unlike the style of the house, the theme can be changed pretty easily from one to another. Just like you can paint one side of the house in a different color, you can tweak specific things by customize the theme, but requires some coding skill.

The house comes with a kitchen. By kitchen I mean the blog functionality. Using that you can create dishes (blogs) and share with the world. However, if you want more rooms (static pages), you can create them. Again, you can create manually by write html of the page but a better way to do it is use some visual editor, like just order the carpet, curtain, furnitures and put them in the room.

Tools like visual editor or other ones that helps you create blogs, or help you SEO (make your house easy to find in a crowed city) are called plugins.

Got a better analogy? Leave a comment below.

Backup strategy of this blog when it is running on docker.

So you can see that I had some backup issue recently and lost couple posts. Honestly, I still have not figured out what was the problem so I’m going to lay out my backup stack and if you can spot anything wrong, leave a comment below.

First, this blog is running on 2 dockers. One for WordPress. It just have the PHP code running, with some credentials as environment variables in the memory, so nothing need to be back up. The other docker runs the MariaDB (basically MySQL), which contains all the the data for the post, comments, users, etc. That is what I want to backup.

backup1

If you read my previous posts about docker, you know that it is mostly read only. So if you start a docker running a database on it, it will lose the data whenever you restart it. Unless you use data volume. So my MariaDB docker has a volume for the folders: /etc/mysql and /var/lib/mysql , where MariaDB will save the data. And they will stay the same when I stop/start the container.

backup2

However, I did not map the data volume to a physical folder on the host. I’m not 100% where the data volume actually is but I think if I destroy the container, the data volume will be gone. I did not map it to a physical folder because the machine is just a VM on Microsoft Azure. I don’t think there is much value mapping it and save it there since the machine can be gone easily.

What I did was to backup the data volume to Amazon S3. I originally used this dockup project and its docker image. But it has some problems with restore. So I used a fork of it here. The fork did not have a docker image, so I built it here. I created 2 services on tutum.co and tell them to mount the data volume from my MariaDB. With one click of a button, I can easily backup/restore my MariaDB data volume as a zip file onto/from Amazon S3.

backup3 backup4

But I did not want to do this manually every time I post something. (Or maybe I should have) So I set up a cron job like thing to back it up every day. I used https://github.com/sunshineo/tutum-schedule which is my fork from https://github.com/alexdebrie/tutum-schedule . What I changed was make the project run a non stop python process directly instead of supervisord. I discussed this with alexdebrie in the tutum slack channel and we all agreed that this is a good idea.

So there you are, something worked pretty well when I tested end of May. I did not post must the month of June but the backup did happen everyday. However, the backed up file never changed even after I made some post. This is a mystery to me.

Try tutum.co. You will like it.

So you tried docker a little bit, decided that it is better than whatever you had before and want to jump on it. But if you have an existing mature continues integration pipeline, you may hesitate. Because right now after you commit your code change, the rest is taken care of by your CI pipeline automatically, build, test, deploy, etc. Only when their is a problem you will be notified. However if you use docker, you find yourself need to manually type a lot of commands or start to write a lot of scripts with no pretty UI anymore. You must wonder if someone has already did this for you and the answer is Yes.

Introducing tutum.co , the one and only and best end to end solution for docker based CI pipeline. Disclaimer, I am not affiliated with them at all. I simply love their product. I switched everything I own from Heroku to here even though Heroku is free but you need to bring your own server for tutum.co . (Heroku is a great product for its time, I would have totally wrote a blog like this couple years ago if I was blogging)

I have been using tutm.co for a while. I enjoyed it all the time. But only recently I finally decided to recommend them to the world because they just solved a big problem I have. Now I think you can ignore their “Beta” flag and use them for production. (Personal opinion, not responsible for any decision you make) The problem I had was that Docker Hub sucks. I had several private repos their and I have encountered severl times that the build got backed up or completely stuck there results in the change I want to get out cannot. I don’t even make changes that frequent so I bet their actual SLA is even worse than what I experienced. It is understandable that they encounter this problem because everyone in the world is asking them to build their docker image all the time. Also they are the inventor or docker, they are system guys, probably not very experienced with service.

tutum.co solved the problem by leverage the machine of each person has, now the machine/node that I eventually will run the docker is also in charge of building the docker image. They also use the machine to run test of the docker and even though I’m not using that feature right now, believe me I will soon. And I have researched other solutions: Travis CI, Circle CI, Drone.io, some docker solutions on top of Jenkins, believe me, none of them is doing it right.

Give tutum.co a try and let me know your experience down in the comment.

REST on top of SQL. Someone please build it.

Existing similar things: (from worst to best)

http://www.espressologic.com/features-rest-api/

Not looking good. Seems to charge a lot of money and not open.

 

http://sqlrest.sourceforge.net/

Looks very crappy and not active. All XML

 

http://restsql.org/doc/Concepts.html

Looks a little less crappy. Does have a github https://github.com/restsql/restsql . And looks does have some activity.

Very barebone. Things are configured with XML. But it’s Java and can be extended. The docs and stuff makes sense. Can be used as a Java Library, but if used like that, why not use a proper ORM like Hibernate?

 

https://github.com/jeffknupp/sandman

Looks easy to use. Written in python and heavily influenced by Django clearly. Not active for a year, but the automatically inspect db using reflection is a good idea.

 

https://github.com/django-tastypie/django-tastypie

http://www.django-rest-framework.org/

Django Tastypie and Django REST framework. These two looks good but then you have to buy in Python Django. And still a lot of things to setup and code to write.

 

http://www.dreamfactory.com/

Looks good and professional. Concern about how open it is. It says Apache 2.0 Licence though, so maybe good. Very active https://github.com/dreamfactorysoftware/dsp-core Written in PHP. Seems to have much broader range, stuff like client side library etc. Support SQL, NOSQL, etc and more importantly, seems at the same time. Even have a Docker image ready.

Seems to be the best choice so far.

My requirements:

  1. REST to SQL CRUD operations.
  2. REST to SQL LFSP (I invented this List Filter Sort Pagination) operations
  3. Cross table joins. Nested return. This maybe optional in an AJAX world. But may be necessary for complicated logics. However, when you consider if this supports multiple data sources, you cannot avoid in memory join, so maybe an in memory join support is more useful. Maybe you need to query one data source and then query another based on previous result. IN clause maybe a problem here since MySQL is not very good at it and Oracle even have a 1000 limit.
  4. API version controls that supports multiple versions.
    1. Maybe you simply need multiple sets of things completely (including different databases for different versions. Let’s face it, different versions probably require different db schema) and you have another service layer on top and when a request comes in, you call call one version easily but if it is a write request, you need to convert the request to make sense for other versions and make additional requests. Still this poses a big problem for data out of sync.
    2. Maybe you have multiple tables for different versions but the transaction will update all of them. Less problematic because it is in a transaction?
  5. Automatically generate documentations.
  6. An admin UI for modifying the data
  7. CLI for the following admin stuff:
    1. Configure database connection results in configuration file. Commit to git? Maybe only the connection and configuration but not the credentials.
    2. Create new edit existing table/resource results in schema migration and data migration files. Commit to git? Or should this be in database? Or should the schema migration be in git and data migration in database.
    3. Automatically have createdAt and updatedAt ?
    4. Optionally have complete history on some table? I don’t mean SQL history. That is in a big SQL log already. But history with who modified what. Very useful for audit. How does this work when schema changes?
    5. Use existing db through reflection. But this may not be necessary. Or this could be the killer feature.
  8. Maybe a JSON REST API and admin UI for these admin work above. But keep in mind that when there is an API and UI, how can we commit to git for the changes? Maybe should learn from the versioned wordpress.
  9. ACL Access Control Level. This may exists in a completely different layer on top of what we have above. We are not going to create a mysql user for each user of the system to leverage database’s access control, that will create a lot of connections to the database. So although ACL data is saved in database but it has to be the service that make sense of it and grant or deny access.
  10. Customize business logic.
  11. Support NOSQL and other types of datasource.
  12. Support multiple and multiple types of datasource together. I definitely feels better if this is another layer on top.