Question: Development/production instances updates management
1
gravatar for mathieu.bahin
2.3 years ago by
mathieu.bahin10 wrote:

Hi,

I'm trying to setup a good galaxy environment for a biology institute. To do so, I need to have a development and a production instances that are clones on the structure side. I'd like to update regularly these instances (following more or less galaxy main updates) and track them with git repos.

I'm pretty sure lot of galaxy admins are trying to do that but I can't find the answer.

My idea was to have a local git repo to save the development instance (not the data). When there is a new update, I git pull it from galaxy GitHub, compare what changed in the sample files and make the appropriate changes in the config files corresponding to these samples. Then I would git push towards the production instance. To do so, I need to have a proper .gitignore file (listing especially the data files). I saw that there is a .gitignore in the galaxy GitHub repo. It's probably relevant when I want to pull the updates but I'm not sure it is regarding the transfers between the development and production instances (for example, the config files have to be tracked I guess).

I hope I'm clear...

Cheers, Mathieu

ADD COMMENTlink modified 10 months ago by Jennifer Hillman Jackson25k • written 2.3 years ago by mathieu.bahin10
3
gravatar for Hotz, Hans-Rudolf
2.3 years ago by
Switzerland
Hotz, Hans-Rudolf1.8k wrote:

Hi Mathieu

We have a similar set up with a production and a development server. And we try to keep them as similar as possible, except for data sets and the PostgreSQL database, of course. Though, we are still on mercurial (actually, I am just in the process of switching)

We don't push from the development server to the production server. Instead, we do two independent pulls from the repository for the two servers. And, we do some etxtra steps:

  • we first install a new test server. This gives us the opportunity to learn about the new features of the Galaxy release.
  • next, we update the the code we installed newly, during the last update cycle. This will show us what we will encounter during the update (e.g. is a change of the database schema required as well).
  • confident, with what we expect, we update the development server. Here we can catch any problems which might be caused by our site specific modifications.
  • and finally we update the production server.

Hope this helps, Hans-Rudolf

ADD COMMENTlink written 2.3 years ago by Hotz, Hans-Rudolf1.8k

Hi Hans-Rudolf,

Thanks for your answer. Isn't it simpler to do a push from development to production instance once you've identified all the problems on the development instance? Maybe you do this way because it's difficult to identify the files to track and/or solve the conflicts?

Apart from that your procedure looks perfect and we might follow it!

Cheers, Mathieu

ADD REPLYlink written 2.3 years ago by mathieu.bahin10
1

Most of the time, there are no conflicts at all. And if, they are easy to solve.

Doing individual pulls allows you to have differences between production and a development server. Although, we try to keep them as identical as possible, the developments server is used for 'development': - new tools - new genome assembly - loging/debuging features ,etc

ADD REPLYlink written 2.3 years ago by Hotz, Hans-Rudolf1.8k

Ok, that makes sense!

Is the '.gitignore' file from the GitHub repo "trustable"? For example I can see in it 'universe_wsgi.ini' which seems pretty outdated now. But probably it's doesn't do any harm and match with most of the instances versions.

Other point, I don't get why the sample files (e.g. 'config/galaxy.ini.sample') are ignored (within the 'config/*' pattern). I would have imagined that when I git pull from the GitHub repo, it would update the sample files, then I would be able to check for changes between the previous sample file and the new one (if the instance is versioned also on my side). Finally, I can adapt any changes to my 'real' config file. How do you get the updates of these files then? With info on the mailing lists?

Cheers, Mathieu

ADD REPLYlink written 2.3 years ago by mathieu.bahin10

wrt: '.gitignore'

As I said, my last update was still done with mercurial, so I can't judge the '.gitignore' file. The reason why 'universe_wsgi.ini' is still in it, is to make sure, nothing breaks for those people who still use 'universe_wsgi.ini' instead of 'galaxy.ini', I assume.

wrt: other point

I guess this is to prevent troubles for those people who use galaxy.ini.sample instead of a copy (ie galaxy.ini).
To get the changes in galaxy.ini, I look at the fresh install I make with every update cycle

ADD REPLYlink written 2.3 years ago by Hotz, Hans-Rudolf1.8k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 169 users visited in the last hour