Question: Galaxy : error with htcondor
1
gravatar for Daphné
3.0 years ago by
Daphné0
France
Daphné0 wrote:

Hello,

I use galaxy jobs with htcondor with job_conf.xml file. I get an error when i launch a job in galaxy. Do you have an idea ?


  • Error in log file

galaxy.jobs.runners.condor ERROR 2015-12-10 17:14:53,600 (6985) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

  • job_conf.xml

<?xml version="1.0"?>
<job_conf>
    <plugins>
        <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
        <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner"/>
    </plugins>
    <handlers default="handlers">
        <handler id="main"/>
        <handler id="handler0" tags="handlers"/>
        <handler id="handler1" tags="handlers"/>
    </handlers>
    <destinations default="condor">
        <destination id="local" runner="local"/>
        <destination id="condor" runner="condor"/>
    </destinations>
</job_conf>

Best regards.

software error galaxy htcondor • 1.1k views
ADD COMMENTlink modified 2.9 years ago • written 3.0 years ago by Daphné0
0
gravatar for Jennifer Hillman Jackson
3.0 years ago by
United States
Jennifer Hillman Jackson25k wrote:

Hello,

As a first pass solution, what happens when you execute with a job_conf.xml including just this content?

https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster?highlight=%28htcondor%29#Condor

Thanks, Jen, Galaxy team

 

ADD COMMENTlink written 3.0 years ago by Jennifer Hillman Jackson25k
0
gravatar for Daphné
3.0 years ago by
Daphné0
France
Daphné0 wrote:

Hi,

Yeah, I have an error because the handler is missing.

galaxy.jobs ERROR 2015-12-11 08:21:38,167 Problem parsing the XML in file /opt/bin/galaxy/config/job_conf.xml, please correct the indicated portion of the file and restart Galaxy. Job configuration file defines no valid handler elements.
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 134, in __init__
    self.__parse_job_conf_xml(tree)
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 197, in __parse_job_conf_xml
    raise ValueError("Job configuration file defines no valid handler elements.")
ValueError: Job configuration file defines no valid handler elements.
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 65, in paste_app_factory
    app = galaxy.app.UniverseApplication( global_conf=global_conf, **kwargs )
  File "/opt/bin/galaxy/lib/galaxy/app.py", line 85, in __init__
    self.job_config = jobs.JobConfiguration(self)
  File "/opt/bin/galaxy/lib/galaxy/jobs/__init__.py", line 141, in __init__
    raise config_exception(e, job_config_file)
Exception: Problem parsing the XML in file /opt/bin/galaxy/config/job_conf.xml, please correct the indicated portion of the file and restart Galaxy. Job configuration file defines no valid handler elements.

And when I add only the "main" handler, I have the same error that before.

galaxy.jobs.runners.condor ERROR 2015-12-11 08:25:30,697 (6986) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

With the level log in "DEBUG"

galaxy.jobs DEBUG 2015-12-11 09:06:37,088 (6989) Working directory for job is: /opt/data/galaxy-data/database/job_working_directory/006/6989
galaxy.jobs.handler DEBUG 2015-12-11 09:06:37,130 (6989) Dispatching to condor runner
galaxy.jobs DEBUG 2015-12-11 09:06:37,275 (6989) Persisting job destination (destination id: condor)
galaxy.jobs.runners DEBUG 2015-12-11 09:06:37,279 Job [6989] queued (148.612 ms)
galaxy.jobs.handler INFO 2015-12-11 09:06:37,318 (6989) Job dispatched
galaxy.tools.deps DEBUG 2015-12-11 09:06:37,606 Building dependency shell command for dependency 'samtools'
galaxy.tools.deps WARNING 2015-12-11 09:06:37,606 Failed to resolve dependency on 'samtools', ignoring
galaxy.jobs.runners DEBUG 2015-12-11 09:06:37,731 (6989) command is: python /opt/bin/galaxy/tools/data_source/upload.py /opt/bin/galaxy /opt/data/galaxy-data/database/tmp/tmptGd1We /opt/data/galaxy-data/database/tmp/tmpstpGXX 11022:/opt/data/galaxy-data/database/job_working_directory/006/6989/dataset_11022_files:/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat; return_code=$?; python "/opt/data/galaxy-data/database/job_working_directory/006/6989/set_metadata_kdxdgu.py" "/opt/data/galaxy-data/database/tmp/tmptGd1We" "/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy.json" "/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_in_HistoryDatasetAssociation_10398_BAIzaX,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_kwds_HistoryDatasetAssociation_10398_WopDFL,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_out_HistoryDatasetAssociation_10398_uoSjGJ,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_results_HistoryDatasetAssociation_10398_N2V7n2,/opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat,/opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_override_HistoryDatasetAssociation_10398_ap1MC9" 5242880; sh -c "exit $return_code"
galaxy.jobs DEBUG 2015-12-11 09:06:37,774 fail(): Moved /opt/data/galaxy-data/database/job_working_directory/006/6989/galaxy_dataset_10398.dat to /opt/data/galaxy-data/database/files/011/dataset_11022.dat
galaxy.datatypes.metadata DEBUG 2015-12-11 09:06:37,882 Cleaning up external metadata files
galaxy.datatypes.metadata DEBUG 2015-12-11 09:06:37,918 Failed to cleanup MetadataTempFile temp files from /opt/data/galaxy-data/database/job_working_directory/006/6989/metadata_out_HistoryDatasetAssociation_10398_uoSjGJ: No JSON object could be decoded
galaxy.jobs.runners.condor ERROR 2015-12-11 09:06:37,924 (6989) failure preparing job script
Traceback (most recent call last):
  File "/opt/bin/galaxy/lib/galaxy/jobs/runners/condor.py", line 93, in queue_job
    self.write_executable_script( executable.job_file, script )
AttributeError: 'str' object has no attribute 'job_file'

I tried with only the "local" configuration and it works well.

 

ADD COMMENTlink modified 3.0 years ago • written 3.0 years ago by Daphné0
0
gravatar for Daphné
2.9 years ago by
Daphné0
France
Daphné0 wrote:

Hello,

I tested with SGE, and it works well. This problem is only with condor.

I think the configuration is bad, but i don't see the error.

Daphné

ADD COMMENTlink written 2.9 years ago by Daphné0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 171 users visited in the last hour