Designing an update process for busy & complicated website

Forum Moderators: phranque

Message Too Old, No Replies

Designing an update process for busy & complicated website

Seeking best practices to upgrade files and database from dev to production

mwDev

8:40 pm on May 4, 2020 (gmt 0)

I've never really worked on a project that needed version control or development environments that were any fancier than copying a local version of the files to the remote server when I was ready. But now I'm setting up something that really does need at least three environments (development, testing, and production) with the ability to collaborate among multiple developers and to push from one environment to the others, including a database.

How would such updates normally be handled with minimal (if any) downtime, especially when the database structure itself may change from one update to the next? What tools and workflows would typically be used to ensure the process goes smoothly?

Have been looking into a few options, but feeling kind of intimidated at the moment as the learning curve seems steeper than I expected it to be.

NickMNS

9:01 pm on May 4, 2020 (gmt 0)

Hello mwDev, Welcome to Webmaster World.

I use GIT for version control, and I use a VPS from one of the better known providers. The VPS allows me to clone it. So what I do is create a clone or clones as needed, set up the necessary DNS to have an accessible website (don't forget to block those that shouldn't see it eg:everybody but you and client). Then using GIT I create a branch for the changes I'm implementing, push the branch to the clone. Dev and test and when satisfied, I merge the branch and push the updated master to the production server.

Note that I don't manage my db within this framework. But you could make the db changes in your dev environment, then clone the dev environment and make that the new production server. You obviously need to be sure that the data is up-to-date in both environments, to avoid lost transactions.

If you plan on doing this a lot you may also want to look into using containers such as Docker or Kurbernetes.

phranque

2:11 am on May 5, 2020 (gmt 0)

welcome to WebmasterWorld [webmasterworld.com], mwDev!

i'm pretty sure "everyone" is using git for this these days.

graeme_p

10:59 am on May 5, 2020 (gmt 0)

Version control makes your life easier in so many ways. Once you start using it you never look back.

Git is not my favourite, but its what everyone uses so is definitely easier if you are collaborating with others. I sometimes use Fossil for things I am sure only I will work on.

If you use database migrations you version control those. You should not version control the data itself. Migrations, again, make applying changes easier and quicker.

There are ways of automating deployment (I use Fabric scripts) which means you can deploy your latest commit, run migrations and anything else required for deployment by typing one command on your local machine. Very quick and easy and less error prone.

lammert

2:04 am on Aug 13, 2020 (gmt 0)

Git only works for static objects like source code but is not useful for version control of databases. The only functioning solution I have found is to do version control of databases in the database itself with a version table which tracks the version of each other table, together with the SQL commands needed to execute for each table version increment.

The application code must check the version of each table before use and automatically issue the necessary SQL commands to upgrade the database structure whenever the actual version number of one or more tables is less than the version number expected by the application.

graeme_p

9:34 am on Aug 15, 2020 (gmt 0)

@lammert database migration tools will do most of this for you.

The one I use (which is somewhat tied to the web framework I use) will let you upgrade the database to match the current version of the code with one command, and can move backwards to early versions too (provided forward migrations have not lost data).