Browse libraries as admin results in unicode error

Question: Browse libraries as admin results in unicode error

2.4 years ago by

Dear all,

when I want to browse the data libraries in the admin panel, I get a generic error on the screen. The log files gave some more information. the main error seems to be:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

The full log of this error is attached below. I implemented the fix mentioned in this post: An error adding datasets to a New data library, but the error persisted. Funnily enough I can access the libraries via Shared Data/Data Libraries from the main screen. Since it seems to be an unicode error, it might have something to do with that I am located in Germany, although the system is set up with enUS settings.

Does anybody have a solution for this problem?

Thanks a lot in advance,
Thomas

The log files show the following:

    127.0.0.1 - - [22/Jun/2016:18:01:48 +0200] "GET /galaxy/library_admin/browse_libraries HTTP/1.1" 500 - "http://127.0.0.1/galaxy/admin" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0"
Error - <type 'exceptions.UnicodeDecodeError'="">: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
URL: http://127.0.0.1/galaxy/library_admin/browse_libraries
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 151 in __call__
  app_iter = self.application(environ, sr_checker)
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/paste/recursive.py', line 85 in __call__
  return self.application(environ, start_response)
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py', line 640 in __call__
  return self.application(environ, start_response)
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/base.py', line 126 in __call__
  return self.handle_request( environ, start_response )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/base.py', line 184 in handle_request
  body = method( trans, **kwargs )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 90 in decorator
  return func( self, trans, *args, **kwargs )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/library_admin.py', line 151 in browse_libraries
  return self.library_list_grid( trans, **kwd )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/helpers/grids.py', line 309 in __call__
  kwargs=kwargs )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/webapp.py', line 811 in fill_template
  return self.fill_template_mako( filename, **kwargs )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/webapp.py', line 825 in fill_template_mako
  return template.render( **data )
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/mako/template.py', line 445 in render
  return runtime._render(self, self.callable_, args, data)
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/mako/runtime.py', line 829 in _render
  **_kwargs_for_callable(callable_, data))
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/mako/runtime.py', line 864 in _render_context
  _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File '/home/galaxy_production/galaxy-dist/.venv/local/lib/python2.7/site-packages/mako/runtime.py', line 890 in _exec_template
  callable_(context, *args, **kwargs)
File '/home/galaxy_production/galaxy-dist/database/compiled_templates/base.mako.py', line 61 in render_body
  __M_writer(unicode(next.body()))
File '/home/galaxy_production/galaxy-dist/database/compiled_templates/grid_base.mako.py', line 79 in render_body
  __M_writer(unicode(self.load()))
File '/home/galaxy_production/galaxy-dist/database/compiled_templates/grid_base.mako.py', line 107 in render_load
  __M_writer(unicode( h.dumps( self.get_grid_config( embedded=embedded, insert=insert ) ) ))
File '/home/galaxy_production/galaxy-dist/database/compiled_templates/grid_base.mako.py', line 293 in render_get_grid_config
  value = column.get_value( trans, grid, item )
File '/home/galaxy_production/galaxy-dist/lib/galaxy/web/framework/helpers/grids.py', line 374 in get_value
  return escape(value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)


CGI Variables
-------------
  CONTENT_LENGTH: '0'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5'
  HTTP_CONNECTION: 'Keep-Alive'
  HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a012e1a42fb1d069863744fff67e957800879219c061b9ef432565c4a06fde1d8'
  HTTP_HOST: '127.0.0.1'
  HTTP_REFERER: 'http://127.0.0.1/galaxy/admin'
  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0'
  PATH_INFO: '/library_admin/browse_libraries'
  REMOTE_ADDR: '127.0.0.1'
  REQUEST_METHOD: 'GET'
  SCRIPT_NAME: '/galaxy'
  SERVER_NAME: 'localhost'
  SERVER_PORT: '8080'
  SERVER_PROTOCOL: 'HTTP/1.1'


WSGI Variables
--------------
  application: <paste.recursive.RecursiveMiddleware object="" at="" 0x7f705fce84d0="">
  is_api_request: False
  paste.cookies: (<SimpleCookie: galaxysession="c6ca0ddb55be603a012e1a42fb1d069863744fff67e957800879219c061b9ef432565c4a06fde1d8">, 'galaxysession=c6ca0ddb55be603a012e1a42fb1d069863744fff67e957800879219c061b9ef432565c4a06fde1d8')
  paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'="">]
  paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object="" at="" 0x7f705fce8bd0="">
  paste.httpserver.proxy.host: 'dummy'
  paste.httpserver.proxy.scheme: 'http'
  paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object="" at="" 0x7f706d5d5d10="">
  paste.recursive.forward: <paste.recursive.Forwarder from="" galaxy="">
  paste.recursive.include: <paste.recursive.Includer from="" galaxy="">
  paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from="" galaxy="">
  paste.recursive.script_name: '/galaxy'
  paste.throw_errors: True
  request_id: '9ff04b08389211e6a811782bcbaba08d'
  webob._parsed_query_vars: (GET([]), '')
  wsgi process: 'Multithreaded'
------------------------------------------------------------

software error galaxy • 826 views

ADD COMMENT • link •

modified 2.4 years ago by Martin Čech ♦♦ 4.9k • written 2.4 years ago by thomas.draebing • 0

2.4 years ago by

Martin Čech ♦♦ 4.9k

United States

Martin Čech ♦♦ 4.9k wrote:

Hi Thomas,

what version of Galaxy do you have? The original problem was the local date was being presented in non-unicode format (like French or possibly German). The fix you mentioned should correct it. Did you restart your Galaxy?

Martin

ADD COMMENT • link written 2.4 years ago by Martin Čech ♦♦ 4.9k

Hi Martin,

thanks for the quick response. I am using version 16.01. That should be up-to-date with the bitbucket-repo. I restarted the galaxy service and also the workstation after applying the fix, but that did not seem to work.

But there is also a difference to the problem described at this thread (enter link description here). The original post from Vladimir stated that he could actually create a library as admin, but just could not add files. I am not able to access the data libraries screen for admins at all. Thus creating libraries is so far not possible (in the admin interface, using the standard user interface it works). And since the problem described in the github bug report spoke of abbreviations of French dates having non-ASCII characters, this shouldn't be the case here, since the system and Galaxy are set up in English. Although of course external files could be the cause, but why are the libraries accessible through the non-admin way.

Thus I guess that the same fix would have to be applied at a different command somewhere.

Best, Thomas

ADD REPLY • link written 2.4 years ago by thomas.draebing • 0

Hi Thomas, first thing we should clarify is that there are two data library interfaces. One is under Shared Data/Data Libraries - this is fairly new implementation and it encapsulates features for both users and admins (showing them only the actions they are allowed to do). You can surely create and manage libraries through that interface if you are an admin. It also has better handling of unicode. The second interface is under the Admin tab in the menu - this one is a legacy implementation of library administration interface that is left in place because it offers some rare functions that are not implemented in the new UI. This interface has not been touched in a while and is not tested much.

I am not sure what would be causing the unicode errors, could it be your database locale creating nonascii dates? Do you actually have no libraries on your Galaxy right now? (check DB please)

ADD REPLY • link modified 2.4 years ago • written 2.4 years ago by Martin Čech ♦♦ 4.9k

Hi Martin,

as mentioned before the Shared Data/Data Libraries interface works fine. The problem only exists for the Admin interface. I was just used to use the Admin interface so far, but will switch to the new interface now. The Admin interface will then probably anyway fully deprecated in the next updates I guess? So this bug is not really being a big issue. Anyway, I checked the date format of the database (which is a postgres 9.5 database). It looks as follows: yyyy-mm-dd HH:MM:SS.xxxxxx. Thus consists only of ASCII characters. I searched the whole database for non-ASCII chars with the following commands:

sudo pg_dump --inserts -U <galaxy-user> <galaxy-db> > a.tmp
grep --color='auto' -P -n "[\x80-\xFF]" a.tmp

I could not find any non-ASCII characters that way.

Thanks again, Thomas

ADD REPLY • link written 2.4 years ago by thomas.draebing • 0

Thanks for the help tracking this, I am running out of ideas :/

Please try using the new UI and let me know if you cannot accomplish something. The admin interface for libraries will disappear in some future release (I think the last big feature missing is moving folders and datasets around). Future library improvements are listed here: https://github.com/galaxyproject/galaxy/issues/1177 .

ADD REPLY • link written 2.4 years ago by Martin Čech ♦♦ 4.9k

Thanks for looking into this! If I should find the fault at one point, I will post it.

ADD REPLY • link written 2.4 years ago by thomas.draebing • 0

Similar posts • Search »