Question: Obtaining a clone of "Main"
1
gravatar for Jeremy Leipzig
2.7 years ago by
United States
Jeremy Leipzig50 wrote:

How would one go about getting the current Main instance running locally?

main • 1.1k views
ADD COMMENTlink modified 2.5 years ago by fsainsbu0 • written 2.7 years ago by Jeremy Leipzig50
5
gravatar for Jeremy Leipzig
2.7 years ago by
United States
Jeremy Leipzig50 wrote:

**Updated to reflect recent changes to galaxy-cloudman-playbook**

Hi Jeremy,

A script to do just that is available with the galaxy-cloudman-playbook

https://github.com/galaxyproject/galaxy-cloudman-playbook/tree/master/roles/galaxyprojectdotorg.cloudman-galaxy-setup/files/scripts

This works with a standard local installation. You can skip all the ansible stuff and just run the install script using the instructions below

Start with a fresh Galaxy installation and database

Modify Galaxy

curl https://raw.githubusercontent.com/galaxyproject/galaxy-cloudman-playbook/master/roles/galaxyprojectdotorg.cloudman-galaxy-setup/files/galaxy/shed_tool_conf.xml.cloud > config/tool_conf.xml 

Uncomment and alter this line in galaxy.ini

tool_config_file = config/tool_conf.xml,config/tool_conf.xml.main

Configure and Run Galaxy Cloudman Playbook

git clone https://github.com/galaxyproject/galaxy-cloudman-playbook.git
cd galaxy-cloudman-playbook/roles/galaxyprojectdotorg.cloudman-galaxy-setup/files/scripts
pip install bioblend
ln -s ../../../../files/shed_tool_list.yaml.cloud .

configure the shed_tool_list.yaml.cloud with your Galaxy url and admin user API (obtained from your Galaxy instance web app)
python install_tool_shed_tools.py -t shed_tool_list.yaml.cloud

A couple caveats:

If this the app server is running Galaxy from a directory make sure the IP in your conf file ends in a slash e.g. 10.30.4.241/galaxy/

You will need to enable API access, so if you are using external authentication like LDAP an explicit Apache or nginx section might be required to allow this.

<Location "/galaxy/api">                                                                                                                                                         
    Satisfy Any                                                                                                                                                       
    Allow from all                                                                                                                                                     
</Location>

Expect a ton of InsecureRequestWarnings. There are "operation not permitted" errors even if you run this as root.

Put your shed_tool_conf.xml and shed_tool_data_table_conf.xml in version control since these will be altered by the script.

 

ADD COMMENTlink modified 2.7 years ago • written 2.7 years ago by Jeremy Leipzig50
2

To have tools placed into proper tool panel sections, I've created a template `shed_tool_conf.xml` file that contains all the tool section IDs I use. Here's an example: https://github.com/galaxyproject/galaxy-cloudman-playbook/blob/master/roles/galaxyprojectdotorg.cloudman-galaxy-setup/files/galaxy/shed_tool_conf.xml.cloud

You can then add that file to the list of files in `galaxy.ini` under `tool_config_file` and Galaxy will then modify that one vs. others you may have listed (e.g., https://github.com/galaxyproject/galaxy-cloudman-playbook/blob/master/group_vars/all#L133).

Ideally, I feel the Toolshed should give you an option to create a a tag section if it does not already exist or use an existing one but that's a different story.

ADD REPLYlink written 2.7 years ago by Enis Afgan590

I tried the script. It installed most of the tools. However, the tools were not in the right sections, but spread separately. 

I cloned galaxy from the master branch, and uncomment few lines about tools and users. Is there other settings might affect this?

Thanks. 

ADD REPLYlink written 2.2 years ago by xunshengbu20
0
gravatar for Jennifer Hillman Jackson
2.7 years ago by
United States
Jennifer Hillman Jackson23k wrote:

Hello,

You can start here: http://getgalaxy.org

The instructions continue from there for how to set up a production instance. Tools are obtained from the Tool Shed. A local will take significant resources to do the same types of analysis that are set up Main, but this is something you may already know.

If the server is for your own use (or a smaller group), then a cloud version could be a better solution: http://usegalaxy.org/cloud

All choices:
http://wiki.galaxyproject.org/BigPicture/Choices

A list of other groups that have set up public Galaxy instances:
http://wiki.galaxyproject.org/PublicGalaxyServers

And there is active deployment and development discussion going on at the galaxy-dev at galaxyproject dot org mailing list:
http://wiki.galaxyproject.org/MailingLists

Best, Jen, Galaxy team

ADD COMMENTlink written 2.7 years ago by Jennifer Hillman Jackson23k

I have a galaxy instance but I want it to look exactly like Main without spending all my time reordering the tool panel. Why can't I get a copy of the `integrated_tool_panel.xml` used in Main?

ADD REPLYlink modified 2.7 years ago • written 2.7 years ago by Jeremy Leipzig50
0
gravatar for fsainsbu
2.5 years ago by
fsainsbu0
Australia
fsainsbu0 wrote:

Hi Jen and Enis

I have tried to follow this but have run into a number of issues:

I could not find an example so used shed_tool_list.yaml.cloud

wget https://github.com/mdshw5/cloudman-image-playbook/blob/master/scripts/shed_tool_list.yaml.cloud

running works, up to a point.

rerunning it shows

python install_tool_shed_tools.py -t shed_tool_list.yaml.cloud^M

(1/206) Tool all_cufflinks_tool_suite already installed. Skipping...^M^M

(2/206) Tool all_vcftools already installed. Skipping...^M^M

.

.

164/206) Installing tool suite_gatk_1_4 from devteam to section gatk (TRT: 0:00:04.112586)^M^M

        Tool installing.^M^M

        Tool suite_gatk_1_4 installed successfully (in 0:00:20.647678) at revision 3e6a4fcbf335^M^M

(165/206) Installing tool suite_gops_1_0 from devteam to section bxops (TRT: 0:00:25.878629)^M^M

        * Error installing a tool (after suite_gops_1_0)! Name: 0:00:08.197327,owner: devteam, revision: c82347870270, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}^M^M

(166/206) Installing tool suite_samtools_0_1_18 from devteam to section samtools (TRT: 0:00:35.193715)^M^M

        * Error installing a tool (after suite_samtools_0_1_18)! Name: 0:00:05.967991,owner: devteam, revision: 71d378848982, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}^M^M

(167/206) Tool suite_samtools_0_1_19 already installed. Skipping...^M

..

.

(202/206) Installing tool weblogo3 from devteam to section plots (TRT: 0:00:47.618287)^M^M

        Tool installing.^M^M

        Tool weblogo3 installed successfully (in 0:00:24.286027) at revision a10f9ed463ae^M^M

(203/206) Tool weightedaverage already installed. Skipping...^M^M

(204/206) Tool windowsplitter already installed. Skipping...^M^M

(205/206) Tool xy_plot already installed. Skipping...^M^M

(206/206) Tool data_manager_fetch_genome_all_fasta already installed. Skipping...^M^M

Skipped tools (201): [u'all_cufflinks_tool_suite', u'all_vcftools', u'analyze_covariates', u'annotation_profiler', u'bamleftalign', u'bam_to_sam', u'basecoverage', u'bcftools', u'best_regression_subsets', u'bowtie2', u'bowtie_color_wrappers', u'bowtie_wrappers', u'bwa_0_7_10', u'bwa_wrappers', u'categorize_elements_satisfying_criteria', u'ccat', u'change_case', u'clustalw', u'cluster', u'column_maker', u'complement', u'compute_motif_frequencies_for_all_motifs', u'compute_motifs_frequency', u'compute_q_values', u'concat', u'condense_characters', u'convert_characters', u'correlation', u'count_covariates', u'count_gff_features', u'coverage', u'ctd_batch', u'cuffcompare', u'cuffdiff', u'cufflinks', u'cuffmerge', u'cut_columns', u'depth_of_coverage', u'dgidb_annotator', u'divide_pg_snp', u'dna_filtering', u'draw_stacked_barplots', u'dwt_cor_ava_perclass', u'dwt_cor_avb_all', u'dwt_ivc_all', u'dwt_var_perclass', u'dwt_var_perfeature', u'emboss_5', u'emboss_datatypes', u'express', u'fasta_clipping_histogram', u'fasta_compute_length', u'fasta_concatenate_by_species', u'fasta_filter_by_length', u'fasta_formatter', u'fasta_nucleotide_changer', u'fasta_to_tabular', u'fastqc', u'fastq_combiner', u'fastq_filter', u'fastq_groomer', u'fastq_manipulation', u'fastq_masker_by_quality', u'fastq_paired_end_deinterlacer', u'fastq_paired_end_interlacer', u'fastq_paired_end_joiner', u'fastq_paired_end_splitter', u'fastq_quality_boxplot', u'fastq_quality_converter', u'fastq_quality_filter', u'fastqsolexa_to_fasta_qual', u'fastq_stats', u'fastqtofasta', u'fastq_to_fasta', u'fastq_to_tabular', u'fastq_trimmer', u'fastq_trimmer_by_quality', u'fastx_artifacts_filter', u'fastx_barcode_splitter', u'fastx_clipper', u'fastx_collapser', u'fastx_nucleotides_distribution', u'fastx_quality_statistics', u'fastx_renamer', u'fastx_reverse_complement', u'fastx_trimmer', u'featurecounter', u'filter_transcripts_via_tracking', u'find_diag_hits', u'flanking_features', u'freebayes', u'freebayes_0_9_14_8a407cf5f4', u'freebayes_wrapper', u'gatk2', u'generate_pc_lda_matrix', u'get_flanks', u'getindelrates_3way', u'getindels_2way', u'gi2taxonomy', u'gmaj', u'hgv_fundo', u'hgv_hilbertvis', u'histogram', u'indel_realigner', u'indels_3way', u'intersect', u'join', u'lastz', u'lastz_paired_reads', u'lca_wrapper', u'lda_analysis', u'linear_regression', u'logistic_regression_vif', u'macs', u'maf_cpg_filter', u'merge', u'merge_cols', u'microsatellite_birthdeath', u'microsats_alignment_level', u'microsats_mutability', u'mine', u'multispecies_orthologous_microsats', u'mutate_snp_codon', u'ncbi_blast_plus', u'partialr_square', u'pearson_correlation', u'pgsnp2gd_snp', u'picard', u'pileup_interval', u'pileup_parser', u'plot_from_lda', u'poisson2test', u'principal_component_analysis', u'print_reads', u'quality_filter', u'rcve', u'realigner_target_creator', u'remove_beginning', u'sam2interval', u'sam_bitwise_flag_filter', u'sam_merge', u'sam_pileup', u'sam_to_bam', u'samtool_filter2', u'samtools_flagstat', u'samtools_mpileup', u'samtools_phase', u'samtools_rmdup', u'samtools_slice_bam', u'samtools_sort', u'scatterplot', u'show_beginning', u'show_tail', u'sicer', u'snpeff', u'snpfreq', u'split_paired_reads', u'ncbi_sra_toolkit', u'substitution_rates', u'substitutions', u'subtract', u'subtract_query', u'suite_fastx_toolkit_0_0_13', u'suite_samtools_0_1_19', u't2ps', u't2t_report', u'table_annovar', u'table_recalibration', u'tables_arithmetic_operations', u'tabular_to_fasta', u'tabular_to_fastq', u'tophat', u'tophat2', u'tophat_fusion_post', u'trimmer', u't_test_two_samples', u'variant_annotator', u'variant_apply_recalibration', u'variant_combine', u'variant_eval', u'variant_filtration', u'variant_recalibrator', u'variant_select', u'variants_validate', u'varscan_version_2', u'vcf2pgsnp', u'vcf_annotate', u'vcf_extract', u'vcf_filter', u'vcf_intersect', u'vcftools_annotate', u'vcftools_compare', u'vcftools_isec', u'vcftools_merge', u'vcftools_slice', u'vcftools_subset', u'vt_variant_tools', u'weightedaverage', u'windowsplitter', u'xy_plot', u'data_manager_fetch_genome_all_fasta']^M^M

Errored tools (2): [u'suite_gops_1_0', u'suite_samtools_0_1_18']^M^M

All tools listed

 

In the monitoring  in admin: yellow around the cloning ones?

Monitor installing tool shed repositories

 
   
Name Description Owner Revision Installation Status

get_flanks

Get flanks returns flanking region/s for every gene

devteam

a72f0decd7b3

Cloning

suite_gops_1_0

Metarepository for the gops tool suite.

devteam

c82347870270

New

fastx_nucleotides_distribution

Draw nucleotides distribution chart

devteam

63d6e2daad48

Cloning

data_manager_gatk_picard_index_builder

data_manager_gatk_picard_index_builder

devteam

700f2df51eb0

Cloning

table_recalibration

Table Recalibration on BAM files

devteam

53dd1bfced54

Cloning

unified_genotyper

Unified Genotyper SNP and indel caller

devteam

66dd4d4c1743

Cloning

cluster

Cluster the intervals of a dataset

devteam

d5677eecbad4

New

basecoverage

Base Coverage of all intervals

devteam

1e6a9e97fa41

Cloning

samtools_slice_bam

Slice BAM by provided regions

devteam

74a8d2d60258

New

samtools_slice_bam

Slice BAM by provided regions

devteam

68ba55e96489

Cloning

join

Join the intervals of two datasets side-by-side

devteam

e56b47dce68a

New

intersect

Intersect the intervals of two datasets

devteam

5b3c6135a982

New

coverage

Coverage of a set of intervals on second set of intervals

devteam

1e864693a1c0

Cloning

samtools_rmdup

Remove PCR duplicates

devteam

7e92b2a53aab

Cloning

samtools_mpileup

MPileup SNP and indel caller

devteam

973fea5b4bdf

New

sam_to_bam

Convert SAM format to BAM format.

devteam

c73bf16b45df

New

samtools_rmdup

Remove PCR duplicates

devteam

fe83e6f8e65e

New

samtools_flagstat

Provides simple stats on BAM files.

devteam

a3dd61e7bec1

New

package_tophat_1_4_0

Contains a tool dependency definition that downloads and compiles version 1.4.0 of tophat.

devteam

0fea0c8cc6cf

Cloning

package_weblogo_3_3

Contains a tool dependency definition that downloads and installs version 3.3 of the WebLogo application.

devteam

648e4b32f15c

Cloning

pileup_interval

Pileup-to-Interval condenses pileup format into ranges of bases

devteam

a110f9d6ae24

Cloning

complement

Complement intervals of a dataset

devteam

d958d5a0d1e8

Cloning

subtract

Subtract the intervals of two datasets

devteam

5bc2dacbe729

Cloning

data_manager_sam_fasta_index_builder

Builds sam indexes using the new data table format.

devteam

d6b6db69d9bf

Cloning

Can this be fixed? If so how?

 

Thanks in advance.

ADD COMMENTlink written 2.5 years ago by fsainsbu0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 83 users visited in the last hour