VersionPress 2.0: Easy Staging

As we’re getting closer to a release of VersionPress 2.0, I’m going to blog about a couple of features that are coming as part of that release. Probably the biggest one is database synchronization which enables things like painless staging, team workflows, etc. This area is one of the biggest challenges when it comes to managing WordPress sites and I think we have a pretty interesting solution in v2.

A brief introduction

Although the sync / staging functionality is only coming out now, it was actually this very thing that got us started on VersionPress a couple of years back. My personal experience with WordPress has always been that it is a great CMS as long as you can afford to stay in a single instance scenario, but I didn’t quite know how to quickly clone the site to a safe environment, test the changes there and merge them back to the live site again. I stress the last part because that is the really tricky one.

Cloning is easy, merging is hard
Cloning is easy, merging is hard

Cloning is relatively easy as you can just copy all the files and the database, tweak some settings like the site’s URL and that’s basically it. The copy & paste approach works well here. Where it doesn’t work well, if at all, is the opposite direction. When you have some changes locally, you cannot just drop the production site and upload the testing one because what if there were new uploads in the meantime, new comments, posts, users, whatever? You would lose all that.

So you need to synchronize the two sites, or to merge them. And boy, merging is hard. In WordPress especially because a large part of the site is stored in a MySQL database and databases are generally very unfriendly to merging. They are transactional databases after all. So while WordPress generally benefits from having a database underneath, it also makes some things really hard. Which is what we’re trying to solve with VersionPress 2.0.

Easy staging

Let’s demonstrate our approach on a technique called staging. Same principles would also apply e.g. in team environments where multiple people have their own WP instances and need to synchronize the work amongst them, but staging is relatively straightforward so let’s stick with that.

The point of staging is to have a separate WordPress instance where you can test changes however you want and they will not affect the live site in any way. When happy with the result, the changes should propagate to the live site. You might even have multiple levels of testing sites, for example, dev -> staging -> live, where every two environments are synchronized the same way. Let’s use two environments, LIVE and STAGING, for the demos below.

I’ll start with a simple, default WordPress site:

staging-02-default-site

I want to experiment with some new design, and after learning my lessons in the 1.0 Walkthrough, I know it’s better done off of the main site. (Having an Undo button is nice, client not yelling at me is even nicer.) So I create a staging environment.

In VersionPress 2.0, this is done through a WP-CLI command. We are kind of in love with WP-CLI, it is an amazing technology but at the same time, we realize not everyone wants or can drop to the command line so in the future, we will have a friendly GUI for this. In v2, however, WP-CLI commands is what there is.

I’ll go to the directory where my site resides and run the wp vp clone command in it:

staging-03-clone-command

I specified that the clone should be named “staging” and that’s the bare minimum I need to do – VersionPress will then create a folder called staging for my clone, new database tables prefixed wp_staging_* (customizable) and do a couple of other things like update the wp-config.php file for me.

After the cloning is done, I have an identical site running at a new URL:

staging-04-cloned-site

I’ll now pick a new theme and update the site title:

staging-05-updated-cloned-site

Back on the live site, say that there was a new post in the meantime:

staging-06-updated-live-site

This means I cannot just blindly replace the live environment with the staging one. In the past, this would have been a point where things would get unpleasant – I would need to carefully compare both files and the database of both environments, or, in practice, maybe I’d re-do the changes manually on the live site. This might be feasible in some simpler cases but generally, it is an unpleasant manual process that leaves room for error.

With VersionPress, things are much simpler. I’ll just run the vp pull command:

staging-07-pull-command

which merges the changes from both environments:

staging-08-merged-site
Voila! Database merge in WordPress.

I find this pretty exciting because this workflow works for any kind of change, e.g., updating WordPress, installing new plugins, amending theme templates, etc. Also, the staging environment could have lived for months and contain dozens of changes, it wouldn’t matter – I would still just run the simple pull command.

This changes the difficulty or a merge from “hard, sometimes almost impossible” to “piece of cake”. I like such changes.

Conflict resolution

An important topic is conflict resolution and I’ll start by showing that VersionPress’ merging is actually quite clever and will avoid conflicts whenever possible.

I’ll go back to the initial state and start with a fresh clone. I’ll then add a new post on both LIVE and STAGING:

staging-09-concurrent-posts

Because of the way WordPress works, this will assign the same ID to both these posts:

staging-10-same-db-ids

Normally, this would be seen as a conflict in most (if not all) tools. Let’s try to merge the sites with VersionPress:

staging-11-merged-site-same-db-ids
Same database ID? No problem.

It was able to understand that the two posts are indeed unrelated and simply kept both during the merge. This behavior is extremely useful because it means that merging will be painless most of the time, which is the ultimate goal.

However, what if there is in a real conflict? Like, two people doing a conflicting change on a single piece of data in both environments?

Manual conflict resolution

In this scenario, I’m going to edit the default “Hello world!” post in both environments:

staging-12-actual-conflict

This creates a real conflict that cannot be resolved automatically. Let’s try to pull now:

staging-13-pull-detects-conflict

VersionPress was able to understand that this is in fact a conflict and prompts me to resolve it. I’ll choose the first option:

staging-14-keep-conflict-option

I am now instructed to resolve the conflict manually, and here is another nice thing about VersionPress: because we are so close to Git, the conflicts can be resolved using a standard workflow, in any tool I like. For instance, I prefer Beyond Compare for 3-way merge so will resolve the conflict there:

staging-15-conflict-resolution-beyond-compare

To commit the change, I can again use any Git client I like. I’m going to first mark the conflict as resolved in TortoiseGit:

staging-16-conflict-resolved

and then commit the changes:

staging-17-commit-resolved-merge

The Git history now looks like this:

staging-18-git-grahp-after-merge

The last step I need to do (as I was instructed above) is to run the vp apply-changes command:

staging-19-apply-changes-cmd

This synchronized the site’s database and the live site now reflects the resolved conflict:

staging-20-resolved-conflict-site

I know this is a bit geeky at the moment (again, a UI for resolving conflicts is coming in a future update) but hey, I was able to use Git to resolve a database conflict. I can’t tell you how many times in the past I wanted just that!

There’s one last thing I’d like to cover, and that is the fact that the site was under maintenance mode until I resolved the conflict. You usually don’t want this to happen on the live site, so let’s finish with a workflow that deals with that.

Pushing between clones

So far, I’ve only used a combination of vp clone and vp pull commands, but there is a third one: vp push. The updated workflow looks like this:

  1. Clone a site
  2. Do some changes there
  3. Then, standing in the clone, pull changes from the live site
  4. If necessary, resolve the conflicts
  5. Push the changes back to live

In practice, I can try to pull on the live site but if a conflict is detected, I can abort the pull:

staging-21-abort-pull

Then I’d go to the staging clone and pull from there. On this separate site where downtime isn’t an issue, I’ll choose to resolve the conflicts:

staging-22-pull-from-staging

(BTW, you can notice that I ran the pull without the --from parameter. That is because each clone remembers its origin and will pull from there.)

The actual conflict resolution is exactly the same process as I’ve shown above, so basically I’ve done this:

  1. Resolved the conflict
  2. Commited changes to Git
  3. Ran wp vp apply-changes to have them reflected on the staging site

After I checked that the merged site looks fine (still on staging), I can then push the changes to the live site:

staging-23-push-to-production

After the push completes, the live site is updated without any unnecessary downtime:

staging-24-resolved-live-site

Nirvana.

I’ll probably stop here as this should cover most of the important scenarios. Just to reiterate, we’re still working on VersionPress 2.0 and it is not entirely finished yet but if you want to play with this functionality, drop me a line at borek@versionpress.net and I’ll be able to send you the current beta. And of course, if you want to support our effort in the long run, we’ll be happy to welcome you in the Early Access Program on our homepage.

Thank you and happy cloning & merging!

37 thoughts on “VersionPress 2.0: Easy Staging

  1. I was all ears or eyes when I spotted the headline in the WPmail newsletter, but you lost me when I realised that this is WP-Cli only, it’s not an option for me at the moment, sad, very sad.
    I will come again when you have a GUI ready.

    1. I’d strongly recommend getting to grips with wpcli.

      If you are at the point where you are managing applications across different environments using version control, then you are also at the point where you should be getting used to the command line.

      There is a learning curve. But it’s worth it.

  2. Very impressed. I’ve looked at a several good local > dev > prod git workflow solutions, but they all assume there’s a single go-live, and don’t account for regular updates on an active live site.

    I’m an early supporter, and looking forward to shelving the manual update process!

  3. this looks amazing. Good work. Seems like you’re making good progress and quick too! I’ll definitely be checking this out very soon.

  4. How does staging and the command vp pull and push work if our dev wordpress site is on a different server from our production wordpress site?

      1. Thanks for responding so quickly. Our clients dev/test vs production wordpress sites are on different servers, so for now we will have have versionpress installed on both systems, and we will have to do manual copies over of changes when they occur and compare files and merge as necessary. Unless you have any other suggestion of how we can use versionpress to help us get the changes we need and move them over to the site on the other server?

        Do you have any idea when the commands will be updated? By end of 2015 or not until sometime in 2016?

        Thanks.

        1. You could push / pull between the Git repos (if your setups allow that, or use aGit server as a middle man) and then call `wp vp apply-changes` which updates the database. Quite technical at this point but should be possible.

          As for the workflows update, I cannot promise a date at the moment. Both by the end of 2015 and early 2016 are possible.

  5. I’m curious about handling of references through ID, it seems this sort of merge would break them?

    Scenario:
    Site A:
    New post, ID = 2
    New attachment, ID = 3
    On post_id=2, set postmeta related_image=3

    Site B:
    New post, ID = 2
    New Post, ID = 3

    When we merge from A to B, it should look like this:
    New post, ID = 2 (from B)
    New post, ID = 3 (from B)
    New post, ID = 4 (from A, postmeta related_image=3)
    New attachment, ID = 5

    As you can see, we lost the connection to the attachment. How is this handled in VersionPress?

  6. Borek,

    We have a scenario of 1 designer-front-end-ninja, 1 dev (me) working each one his own local environment, then a QA (aka staging) environment, and then production.

    We have tried all sort of things to figure out the database merging without any success.
    I’ve been playing around with VP the whole day, and I’m very enthusiastic about it.

    It seems to be the right tool for the project.
    I’ll keep you posted about our findings.

    Thanks a lot for the wonderful work you do.

  7. VersionPress seems quite unique, I have never heard of similar tools… especially about the database merging part.

    Do you plan to support more than just WordPress sites ?

  8. Best plug in ever!! specially with the db merge function! Congratulation Borek, you’re the man!

    I have a little issue in trying to clone the site, once I command wp-cli.phar (I can’t use the short wp command, because of my server) I got the following error

    Success: Updated remote configuration
    Success: Enabled pushing to the original repository
    Success: Enabled pushing to the clone
    Success: wp-config.php updated
    Success: Copied VersionPress
    sh: wp: not found

    It stops there, can this be because I can’t don’t use the short wp command? or it’s another error? If it’s because of the short wp command, what options do I have?

    Thanks

      1. Carlos, if you have a local .bashrc, you can create a ~/bin/ folder and source it from within the bashrc, move the wp-cli.phar there as “wp”, and then it will work for you as “wp”

        There are plenty of tutorials on how you can do this.

  9. Hello Borek,

    Thank you for this detailed tutorial for using Versionpress. As a developer, I’m always looking to improve my workflow. After reading the article, I have a question about your demo. I read that you are using a staging and live site for the demo. However it would appear that both the staging site and live site reside on the same “localhost” environment. My question – how would Versionpress work with a staging site on a localhost and the live site on a server hosted elsewhere, accessed via its domain name or IP address?

    I am fresh into the WP-CLI world so I might be overlooking something here.

    1. Hi John, this is a common request and you’re right that it’s not directly supported by VersionPress yet. But the technology is already capable of it – I’m going to write a blog post about it.

  10. “In VersionPress 2.0, this is done through a WP-CLI command. We are kind of in love with WP-CLI, it is an amazing technology but at the same time, we realize not everyone wants or can drop to the command line so in the future, we will have a friendly GUI for this. In v2, however, WP-CLI commands is what there is.”

    Has a GUI for staging been implemented in v4 beta or is it still pending?

    Thanks

    1. Hi, v4 focuses mostly on plugin compatibility and UI improvements in the single-instance scenarios.

      Staging UI is hard not because the UI would be that complex but because most people on common WP hosts would not be allowed to create staging clones of their sites, or because they want to run staging sites on different servers than production sites (both of which are reasonable demands). This problem hence will not be solved in the v4 timeframe. However, we’re working on something that will make staging really simple in a controlled environment that we’re going to introduce soon.

      Thanks,
      Borek

  11. I wont criticize you for CLI. I’m comming from the other side. Ok, so here’s my scenario. I would like to have branches in git. Two of wich (presumably dev and prod) i want to automaticly update wordpress instances on push (so i can develop on my local instance, then test it on dev and when everythign is ok push it to prod). Everything is running on my own server for git managing i’m using own gitlab. So basicly i need two things (if i got it right). Something that will firstly push (push wp vp) from the prod/dev wordpress instance to the prod/dev branch. Then push my commit to prod/dev branch, and resolve conflicts there, then call pull (wp vp pull) on the production server? Or what should be the workflow? Push my commit to production branch, Then call pull on wp instance and resolve conflicts there?

  12. Hey Borek, great work! Looks promising – how does the plugin handle complex stuff like woocommerce etc. I presume fine?

    Cheers man, keep up the good work!

  13. I cant believe how good this is ! It’s like the alpha and omega of wp worklow annoyance you’re adressing here :p Even better, it’s FOSS <3
    Can't wait to get my grip on your fantastical tool, the community thanks you 🙂

  14. I am sory that I don’t see it but where can you change to an external git repository? I would like to work on my local machine (staging) and push the the remote server hosting my page. How do I do that?

    Thanks
    Alex

    1. Hi Alex, VersionPress always commits locally, it is the same as with plain Git: `commit` is always local but you can occasionally `push` and `pull` to synchronize between your local repo and some remote one. It’s exactly the same thing with VersionPress.

Leave a Reply

Your email address will not be published. Required fields are marked *

Some HTML tags are supported