Question: Galaxy : error with htcondor
1
gravatar for Daphné
3.0 years ago by
Daphné • 0
France
Daphné • 0 wrote:

Hello,

I use galaxy jobs with htcondor with job_conf.xml file. I get an error when i launch a job in galaxy. Do you have an idea ?


  • Error in log file

galaxy.jobs.runners.condor ERROR 2015-12-10 17:14:53,600 (6985) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

  • job_conf.xml

<?xml version="1.0"?>
<job_conf>
    <plugins>
        <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
        <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner"/>
    </plugins>
    <handlers default="handlers">
        <handler id="main"/>
        <handler id="handler0" tags="handlers"/>
        <handler id="handler1" tags="handlers"/>
    </handlers>
    <destinations default="condor">
        <destination id="local" runner="local"/>
        <destination id="condor" runner="condor"/>
    </destinations>
</job_conf>

Best regards.

software error galaxy htcondor • 1.1k views
ADD COMMENT • link • modified 2.9 years ago • written 3.0 years ago by Daphné • 0
0
gravatar for Jennifer Hillman Jackson
3.0 years ago by
United States
Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

As a first pass solution, what happens when you execute with a job_conf.xml including just this content?

https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster?highlight=%28htcondor%29#Condor

Thanks, Jen, Galaxy team

 

ADD COMMENT • link written 3.0 years ago by Jennifer Hillman Jackson ♦ 25k
0
gravatar for Daphné
3.0 years ago by
Daphné • 0
France
Daphné • 0 wrote:

Hi,

Yeah, I have an error because the handler is missing.

galaxy.jobs ERROR 2015-12-11 08:21:38,167 Problem parsing the XML in file /opt/bin/galaxy/config/job_conf.xml, please correct the indicated portion of the file and restart Galaxy. Job configuration file defines no valid handler elements.
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 134, in __init__
    self.__parse_job_conf_xml(tree)
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 197, in __parse_job_conf_xml
    raise ValueError("Job configuration file defines no valid handler elements.")
ValueError: Job configuration file defines no valid handler elements.
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 65, in paste_app_factory
    app = galaxy.app.UniverseApplication( global_conf=global_conf, **kwargs )
  File "/opt/bin/galaxy/lib/galaxy/app.py", line 85, in __init__
    self.job_config = jobs.JobConfiguration(self)
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 141, in __init__
    raise config_exception(e, job_config_file)
Exception: Problem parsing the XML in file /opt/bin/galaxy/config/job_conf.xml, please correct the indicated portion of the file and restart Galaxy. Job configuration file defines no valid handler elements.

And when I add only the "main" handler, I have the same error that before.

galaxy.jobs.runners.condor ERROR 2015-12-11 08:25:30,697 (6986) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

With the level log in "DEBUG"

galaxy.jobs DEBUG 2015-12-11 09:06:37,088 (6989) Working directory for job is: /opt/data/galaxy-data/database/job_working_directory/006/6989
galaxy.jobs.handler DEBUG 2015-12-11 09:06:37,130 (6989) Dispatching to condor runner
galaxy.jobs DEBUG 2015-12-11 09:06:37,275 (6989) Persisting job destination (destination id: condor)
galaxy.jobs.runners DEBUG 2015-12-11 09:06:37,279 Job [6989] queued (148.612 ms)
galaxy.jobs.handler INFO 2015-12-11 09:06:37,318 (6989) Job dispatched
galaxy.tools.deps DEBUG 2015-12-11 09:06:37,606 Building dependency shell command for dependency 'samtools'
galaxy.tools.deps WARNING 2015-12-11 09:06:37,606 Failed to resolve dependency on 'samtools', ignoring
galaxy.jobs.runners DEBUG 2015-12-11 09:06:37,731 (6989) command is: python /opt/bin/galaxy/tools/data_source/upload.py /opt/bin/galaxy /opt/data/galaxy-data/database/tmp/tmptGd1We /opt/data/galaxy-data/database/tmp/tmpstpGXX 11022:/opt/data/galaxy-data/database/job_working_directory/006/6989/dataset_11022_files:/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat; return_code=$?; python "/opt/data/galaxy-data/database/job_working_directory/006/6989/set_metadata_kdxdgu.py" "/opt/data/galaxy-data/database/tmp/tmptGd1We" "/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy.json" "/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_in_HistoryDatasetAssociation_10398_BAIzaX,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_kwds_HistoryDatasetAssociation_10398_WopDFL,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_out_HistoryDatasetAssociation_10398_uoSjGJ,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_results_HistoryDatasetAssociation_10398_N2V7n2,/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_override_HistoryDatasetAssociation_10398_ap1MC9" 5242880; sh -c "exit $return_code"
galaxy.jobs DEBUG 2015-12-11 09:06:37,774 fail(): Moved /opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat to /opt/data/galaxy-data/database/files/011/dataset_11022.dat
galaxy.datatypes.metadata DEBUG 2015-12-11 09:06:37,882 Cleaning up external metadata files
galaxy.datatypes.metadata DEBUG 2015-12-11 09:06:37,918 Failed to cleanup MetadataTempFile temp files from /opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_out_HistoryDatasetAssociation_10398_uoSjGJ: No JSON object could be decoded
galaxy.jobs.runners.condor ERROR 2015-12-11 09:06:37,924 (6989) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

I tried with only the "local" configuration and it works well.

 

ADD COMMENT • link modified 3.0 years ago • written 3.0 years ago by Daphné • 0
0
gravatar for Daphné
2.9 years ago by
Daphné • 0
France
Daphné • 0 wrote:

Hello,

I tested with SGE, and it works well. This problem is only with condor.

I think the configuration is bad, but i don't see the error.

Daphné

ADD COMMENT • link written 2.9 years ago by Daphné • 0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 171 users visited in the last hour