Development/production instances updates management

Question: Development/production instances updates management

2.3 years ago by

Hi,

I'm trying to setup a good galaxy environment for a biology institute. To do so, I need to have a development and a production instances that are clones on the structure side. I'd like to update regularly these instances (following more or less galaxy main updates) and track them with git repos.

I'm pretty sure lot of galaxy admins are trying to do that but I can't find the answer.

My idea was to have a local git repo to save the development instance (not the data). When there is a new update, I git pull it from galaxy GitHub, compare what changed in the sample files and make the appropriate changes in the config files corresponding to these samples. Then I would git push towards the production instance. To do so, I need to have a proper .gitignore file (listing especially the data files). I saw that there is a .gitignore in the galaxy GitHub repo. It's probably relevant when I want to pull the updates but I'm not sure it is regarding the transfers between the development and production instances (for example, the config files have to be tracked I guess).

I hope I'm clear...

Cheers, Mathieu

production admin galaxy local server • 919 views

ADD COMMENT • link •

modified 10 months ago by Jennifer Hillman Jackson ♦ 25k • written 2.3 years ago by mathieu.bahin • 10

2.3 years ago by

Hotz, Hans-Rudolf • 1.8k

Switzerland

Hotz, Hans-Rudolf • 1.8k wrote:

Hi Mathieu

We have a similar set up with a production and a development server. And we try to keep them as similar as possible, except for data sets and the PostgreSQL database, of course. Though, we are still on mercurial (actually, I am just in the process of switching)

We don't push from the development server to the production server. Instead, we do two independent pulls from the repository for the two servers. And, we do some etxtra steps:

we first install a new test server. This gives us the opportunity to learn about the new features of the Galaxy release.
next, we update the the code we installed newly, during the last update cycle. This will show us what we will encounter during the update (e.g. is a change of the database schema required as well).
confident, with what we expect, we update the development server. Here we can catch any problems which might be caused by our site specific modifications.
and finally we update the production server.

Hope this helps, Hans-Rudolf

ADD COMMENT • link written 2.3 years ago by Hotz, Hans-Rudolf • 1.8k

Hi Hans-Rudolf,

Thanks for your answer. Isn't it simpler to do a push from development to production instance once you've identified all the problems on the development instance? Maybe you do this way because it's difficult to identify the files to track and/or solve the conflicts?

Apart from that your procedure looks perfect and we might follow it!

Cheers, Mathieu

ADD REPLY • link written 2.3 years ago by mathieu.bahin • 10

Most of the time, there are no conflicts at all. And if, they are easy to solve.

Doing individual pulls allows you to have differences between production and a development server. Although, we try to keep them as identical as possible, the developments server is used for 'development': - new tools - new genome assembly - loging/debuging features ,etc

ADD REPLY • link written 2.3 years ago by Hotz, Hans-Rudolf • 1.8k

Ok, that makes sense!

Is the '.gitignore' file from the GitHub repo "trustable"? For example I can see in it 'universe_wsgi.ini' which seems pretty outdated now. But probably it's doesn't do any harm and match with most of the instances versions.

Other point, I don't get why the sample files (e.g. 'config/galaxy.ini.sample') are ignored (within the 'config/*' pattern). I would have imagined that when I git pull from the GitHub repo, it would update the sample files, then I would be able to check for changes between the previous sample file and the new one (if the instance is versioned also on my side). Finally, I can adapt any changes to my 'real' config file. How do you get the updates of these files then? With info on the mailing lists?

Cheers, Mathieu

ADD REPLY • link written 2.3 years ago by mathieu.bahin • 10

wrt: '.gitignore'

As I said, my last update was still done with mercurial, so I can't judge the '.gitignore' file. The reason why 'universe_wsgi.ini' is still in it, is to make sure, nothing breaks for those people who still use 'universe_wsgi.ini' instead of 'galaxy.ini', I assume.

wrt: other point

I guess this is to prevent troubles for those people who use galaxy.ini.sample instead of a copy (ie galaxy.ini).
To get the changes in galaxy.ini, I look at the fresh install I make with every update cycle

ADD REPLY • link written 2.3 years ago by Hotz, Hans-Rudolf • 1.8k

Similar posts • Search »