I have configured my SWIFT object storage and the galaxy.ini / object_store_conf.xml files per the documentation; however, I continue to get S3 response errors:
tool_shed.tool_shed_registry DEBUG 2016-07-11 12:00:20,908 Loading references to tool sheds from ./config/tool_sheds_conf.xml.sample tool_shed.tool_shed_registry DEBUG 2016-07-11 12:00:20,909 Loaded reference to tool shed: Galaxy Main Tool Shed galaxy.objectstore.s3 DEBUG 2016-07-11 12:00:20,934 Object cache dir: database/object_store_cache galaxy.objectstore.s3 DEBUG 2016-07-11 12:00:20,934 job work dir: database/job_working_directory_swift galaxy.objectstore.s3 DEBUG 2016-07-11 12:00:20,935 Configuring Swift Connection galaxy.objectstore.s3 DEBUG 2016-07-11 12:00:20,958 Bucket not found, creating s3 bucket with handle 'galaxy1' galaxy.objectstore.s3 ERROR 2016-07-11 12:00:20,960 Could not get bucket 'galaxy1', attempt 1/5 Traceback (most recent call last): File "/home/galaxy/galaxy/lib/galaxy/objectstore/s3.py", line 190, in _get_bucket self.conn.create_bucket(bucket_name) File "/home/galaxy/galaxy/.venv/local/lib/python2.7/site-packages/boto/s3/connection.py", line 621, in create_bucket response.status, response.reason, body) S3ResponseError: S3ResponseError: 401 Unauthorized None
I looked at the connection.py and s3.py files and cannot see where the standard S3 connection (to AWS) is over ridden by SWIFT connection settings. I verified I can connect to my SWIFT object store and bucket with another app.
Anybody have ideas here? Thanks.
Another question I have regarding SWIFT object storage - does anybody know what version of authentication is being used? v1, 2,3?
Does the cloud you're using have EC2/S3 APIs enabled? Galaxy's implementation for using Swift object store relies on boto library, which is an AWS native library so everything goes via those API endpoints. Using the native OpenStack APIs is not available at the moment. With that, I believe it's using v2 auth.
As it happens - I was able to get my object store (EMC ECS) connected via the S3 API. After reviewing the python scripts I realized boto was being used, so I set the http port for the ECS's S3 API - 9020 - and it works. Here's a sample of my object store config file:
<object_store type="swift" id="primary" order="0"> <auth access_key="galaxy" secret_key="S3 secret key"/> <bucket name="galaxy" use_reduced_redundancy="False" max_chunk_size="250"/> <connection host="IP address" port="9020" is_secure="False" conn_path="" multipart="True"/> <cache path="database/object_store_cache" size="1000"/> <extra_dir type="job_work" path="database/job_working_directory_swift"/> <extra_dir type="temp" path="database/tmp_swift"/>
Regarding the support of SWIFT, I assume you need the python client for Swift installed also, no?
Glad to hear you've made progress. There's no need for a Swift client to be installed since only boto is being used.