Question: UnicodeEncodeError: latin-1' codec can't encode character
0
gravatar for snehalbpatil87
19 months ago by
snehalbpatil8730 wrote:

Hello:

I am running simple Galaxy custom tool with R at the back end. It gives me following error:

'latin-1' codec can't encode character u'\u2018' in position 326: ordinal not in range(256)

Does anyone faced this error? Some of the blogs suggests to add Default-ENCODING ="latin-1" in /lib/galaxy/util.py.

But I dont have util.py file in my galaxy.

Thanks Snehal

unicode R • 1.1k views
ADD COMMENTlink modified 18 months ago • written 19 months ago by snehalbpatil8730
0
gravatar for vladimir.gritsenko
19 months ago by
Tel Aviv University
vladimir.gritsenko90 wrote:

Unicode character 2018 is a left quotation mark. My (wild) guess is that you're trying to process a text file which was saved in fancy text editor, or on a Mac (which really likes adding unicode-only quotations marks even when no one is asking it to).

I see two options:
1) Pinpoint the offending text and make sure it's not outputting any funny Unicode characters (how to do this depends on who the offender is exactly - if it's just an input text file, you can open it in a text editor and edit it manually).
2) Change your default encoding to UTF-8. This is dangerous, and this assumes that all galaxy tools down the road will be able to handle unicode, which is unlikely.

As a useful aside, to understand unicode and text encoding, here's an excellent tutorial: http://www.joelonsoftware.com/articles/Unicode.html

ADD COMMENTlink written 19 months ago by vladimir.gritsenko90
0
gravatar for Dannon Baker
19 months ago by
Dannon Baker3.6k
United States
Dannon Baker3.6k wrote:

I've tried to reproduce this with the 16.01 and 16.04 releases and am unable to do so using using a simple python tool which outputs a big block of utf-8 text. What version of galaxy are you using? And can you give more of the error traceback, if available?

If you'd like me to take a deeper look, you could also share the offending tool and I can try to reproduce and fix it that way.

ADD COMMENTlink written 19 months ago by Dannon Baker3.6k
0
gravatar for snehalbpatil87
19 months ago by
snehalbpatil8730 wrote:

Hello Dannon and vladimir:

Thank you for responding to my question.

I am working on the custom tool which runs this command at the back end:

Rscript --vanilla --verbose /home/snehal/Galaxy/galaxy/tools/extraTools/classify_simple.R --project pulldown_mc --inFile /home/snehal/Galaxy/galaxy/database/files/001/dataset_1956.dat --outFile /home/snehal/Galaxy/galaxy/database/files/001/dataset_1988.dat

If at run it directly on red hat server it works fine . But in Galaxy it gives me below error:

galaxy.jobs.runners.local ERROR 2016-04-27 11:58:29,235 Job wrapper finish method failed
Traceback (most recent call last):
  File "/home/snehal/Galaxy/galaxy/lib/galaxy/jobs/runners/local.py", line 129, in queue_job
    job_wrapper.finish( stdout, stderr, exit_code )
  File "/home/snehal/Galaxy/galaxy/lib/galaxy/jobs/__init__.py", line 1264, in finish
    self.sa_session.flush()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 150, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2004, in flush
    self._flush(objects)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2122, in _flush
    transaction.rollback(_capture_exception=True)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2086, in _flush
    flush_context.execute()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 373, in execute
    rec.execute(self)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 532, in execute
    uow
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 170, in save_obj
    mapper, table, update)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 673, in _emit_update_statements
    execute(statement, multiparams)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1335, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
  File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 158, in execute
    query = query % db.literal(args)
  File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 265, in literal
    return self.escape(o, self.encoders)
  File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 203, in unicode_literal
    return db.literal(u.encode(unicode_literal.charset))
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2018' in position 326: ordinal not in range(256)
galaxy.datatypes.metadata DEBUG 2016-04-27 11:58:29,311 Cleaning up external metadata files

I checked input and output files.It doesn't have any quotation marks. I have v15.07-85-g66082f7 version.

Thanks Snehal

ADD COMMENTlink written 19 months ago by snehalbpatil8730
0
gravatar for Dannon Baker
19 months ago by
Dannon Baker3.6k
United States
Dannon Baker3.6k wrote:

Ahh, I misunderstood at first, and thought it was the Galaxy framework itself throwing this error, but it's actually an echo from the database.

Is this a production server, or is it something that'd be ok to tinker with? Basically, for some reason the database encoding is set to latin1 instead of utf8, so it can't handle the characters the tool generates (which would be stored in the database as the dataset peek, error text, etc).

ADD COMMENTlink written 19 months ago by Dannon Baker3.6k
0
gravatar for snehalbpatil87
19 months ago by
snehalbpatil8730 wrote:

Hello Dannon:

I changed my mysql database to utf8 and utf8_unicode_ci . But I ma still getting that error..Do i need to change that in any of the galaxy configuration file..

Thanks Snehal

ADD COMMENTlink written 19 months ago by snehalbpatil8730
0
gravatar for snehalbpatil87
18 months ago by
snehalbpatil8730 wrote:

did anyone face similar problem?

ADD COMMENTlink written 18 months ago by snehalbpatil8730
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 116 users visited in the last hour