Question: FTP upload issue at http://usegalaxy.org - FAQs, help, troubleshooting

15 months ago by

auradunbine • 20 wrote:

Hi! galaxy masters:

I have tried FTP upload about 2 month(July~Aug), and it is still not working. I sure my Filezilla is default setting(passive mode) and I also tried WINSCP, another FTP software, as follow the tutorial page. Both are fail to connect the galaxy server!! I use galaxy for 7 years, but I cannot do anything during these 2 months! Please help me!! Thanks a lot!! Below is the error message:

Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.56:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,56,118,199).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing
Status: Disconnected from server
Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.56:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,56,118,221).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing

upload ftp galaxy • 2.0k views

ADD COMMENT • link •

modified 14 months ago by Jennifer Hillman Jackson ♦ 25k • written 15 months ago by auradunbine • 20

Hello - Some networks have difficulty connecting after the recent security changes to the FTP service (firewall issues). A solution is in progress and we expect it to be ready to test very soon. More updates then. Jen, Galaxy team

ADD REPLY • link written 15 months ago by Jennifer Hillman Jackson ♦ 25k

Hi, Jen: Your assistance in this matter is deeply appreciated!! I'm looking forward the solutions!!

ADD REPLY • link written 15 months ago by auradunbine • 20

Update: The full correction is still in progress as a priority fix.

For an alternate description of how FTP upload is currently failing plus a workaround for loading large data (Upload via URL) please see: https://biostar.usegalaxy.org/p/24648/

ADD REPLY • link written 15 months ago by Jennifer Hillman Jackson ♦ 25k

Thank you for all your assistance and I really appreciate your help in resolving the problem! I have tried both passive and active FTP with "require explicit FTP over TLS", but it's still not working. I'm sure my ftp configuration is following as the instruction. Below is the error message:

Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.60:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,60,119,199).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing
Status: Disconnected from server
Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.60:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,60,118,138).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing

ADD REPLY • link written 14 months ago by auradunbine • 20

This is the error message that is produced when an Implicit FTPS connection is made. Only Explicit FTPS is working at the moment. If you are using Filezilla, follow these instructions and connect with File > Site Manager > The site you set up for usegalaxy.org. Try passive and active mode as described in yesterday's update. If using another client, the setup can vary but should be easily found with a google. Or, consider asking a local admin for help.

The instructions in the Hub for using default passive mode with FTP/Filezilla are not valid at the moment. Avoid "Quick Connect".

ADD REPLY • link modified 14 months ago • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

14 months ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Update 2017-09-05 1:15 EST

Our admin Nate has made some changes but they are not complete yet. Use the first solution for now. We won't close the ticket out until the new configuration changes are fully in place.

Explicit FTPS: Working with either active or passive FTP. Which to use depends on whether the site allows active or not - just try one and then the other. If you are not sure how to configure your client for Explicit FTPS, see the Galaxy hub here or follow these general Filezilla instructions.

Implicit FTPS: NOT working, we are in the process of setting this up for people whose firewalls prevent the use of Explicit FTPS.

Thanks everyone for reporting the problems and for your patience! Jen

ADD COMMENT • link modified 14 months ago by Nate Coraor ♦ 3.2k • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

14 months ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Update 2017-09-07

Both Explicit and Implicit FTPS are now functional.

Thanks, Jen & the Galaxy team

ADD COMMENT • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Hi Jen:

It seems that the new setting is still not working to us! I also tried to connect the galaxy server by some public wifi (without any firewall setting), it is not working, either. So I sure that the connection problem may not caused by the local firewall setting. Below is the latest error message:

Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.56:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,56,117,157).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing
Status: Disconnected from server
Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.56:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,56,117,163).
Command:    MLSD
Error:  Connection timed out after 20 seconds of inactivity
Error:  Failed to retrieve directory listing

ADD REPLY • link written 14 months ago by auradunbine • 20

It looks like this was an explicit FTPS connection, could you try implicit FTPS and see if this has any effect for you? You can verify whether it's making an implicit connection by observing the port number, which should be 990, e.g.:

Status: Connecting to 129.114.60.56:990

ADD REPLY • link written 14 months ago by Nate Coraor ♦ 3.2k

I tried it, but...........

Error message:

Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.56:990... Error: Connection timed out after 20 seconds of inactivity Error: Could not connect to server Status: Waiting to retry... Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.56:990... Error: Connection timed out after 20 seconds of inactivity Error: Could not connect to server Status: Disconnected from server Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.56:990... Error: Connection timed out after 20 seconds of inactivity Error: Could not connect to server Status: Waiting to retry... Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.56:990... Error: Connection timed out after 20 seconds of inactivity Error: Could not connect to server

ADD REPLY • link written 14 months ago by auradunbine • 20

Is the Windows firewall enabled on your system? Can you temporarily disable it while testing this connection?

ADD REPLY • link written 14 months ago by Nate Coraor ♦ 3.2k

I just tested connections to usegalaxy.org, MacOS and Filezilla, and the FTP functionality is working as expected.

FAQ: https://galaxyproject.org/ftp-upload/

FTP, Filezilla > Settings, set at passive mode (saved), using "Quick connect" (implicit) form setting screenshot
FTP, Filezilla > Settings, set at active mode (saved), port 990 specified, using "Quick connect". Passive could also be set for this connection type. (implicit) form setting screenshot
FTP, File > Site Manager set up for usegalaxy.org as a site, using "Connect" on that form (explicit) form setting screenshots (there are two)

If these do not work, what is your OS? Do you want to share a screenshot of the connection type that matches one of the above and is not working?

ADD REPLY • link modified 14 months ago • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

It is working now!! Both explicit and implicit are successfully connected!!
Many thanks!!!!!

Status: Resolving address of usegalaxy.org
Status: Connecting to 129.114.60.60:21...
Status: Connection established, waiting for welcome message...
Response:   220-usegalaxy.org FTP
Response:    
Response:    This FTP server allows users to upload large data to the Galaxy server at:
Response:    
Response:      https://usegalaxy.org/
Response:    
Response:    Your login uses the same credentials as your Galaxy login - email address and
Response:    password.
Response:    
Response:    Once files are successfully uploaded, the list of files available for importing
Response:    in to a history will be displayed in the regular Upload tool.  For assistance,
Response:    please contact:
Response:    
Response:      galaxy-bugs@galaxyproject.org
Response:    
Response:    Files uploaded via FTP and not imported to a history within three days will be
Response:    deleted.
Response:   220 ProFTPD 1.3.5d Server usegalaxy.org FTP (Explicit SSL/TLS on port 21)) [::ffff:129.114.60.60]
Command:    AUTH TLS
Response:   234 AUTH TLS successful
Status: Initializing TLS...
Status: Verifying certificate...
Command:    USER auradunbine@gmail.com
Status: TLS/SSL connection established.
Response:   331 Password required for auradunbine@gmail.com
Command:    PASS ********
Response:   230 User auradunbine@gmail.com logged in
Command:    SYST
Response:   215 UNIX Type: L8
Command:    FEAT
Response:   211-Features:
Response:    EPRT
Response:    EPSV
Response:    MDTM
Response:    SSCN
Response:    TVFS
Response:    MFMT
Response:    SIZE
Response:    PROT
Response:    LANG bg-BG;en-US;es-ES;fr-FR;it-IT;ja-JP;ko-KR;ru-RU;zh-CN;zh-TW
Response:    CCC
Response:    PBSZ
Response:    AUTH TLS
Response:    MFF modify;UNIX.group;UNIX.mode;
Response:    REST STREAM
Response:    MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
Response:    UTF8
Response:   211 End
Command:    OPTS UTF8 ON
Response:   200 UTF8 set to on
Command:    PBSZ 0
Response:   200 PBSZ 0 successful
Command:    PROT P
Response:   200 Protection set to Private
Status: Connected
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/" is the current directory
Command:    TYPE I
Response:   200 Type set to I
Command:    PASV
Response:   227 Entering Passive Mode (129,114,60,60,119,122).
Command:    MLSD
Response:   150 Opening BINARY mode data connection for MLSD
Response:   226 Transfer complete
Status: Directory listing successful

ADD REPLY • link written 14 months ago by auradunbine • 20

Great news, so glad! :)

ADD REPLY • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

But I found there is still a problem for some computers, unavailable connection seems not due to firewall or OS. I don't know why....? (firewall closed, OS is Win10 or Win7)

Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.60:21... Status: Connection established, waiting for welcome message... Status: Initializing TLS... Status: Verifying certificate... Status: TLS connection established. Status: Logged in Status: Retrieving directory listing... Command: PWD Response: 257 "/" is the current directory Command: TYPE I Response: 200 Type set to I Command: PASV Response: 227 Entering Passive Mode (129,114,60,60,117,85). Command: MLSD Error: Connection timed out after 20 seconds of inactivity Error: Failed to retrieve directory listing Status: Disconnected from server Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.60:21... Status: Connection established, waiting for welcome message... Status: Initializing TLS... Status: Verifying certificate... Status: TLS connection established. Status: Logged in Status: Retrieving directory listing... Command: PWD Response: 257 "/" is the current directory Command: TYPE I Response: 200 Type set to I Command: PASV Response: 227 Entering Passive Mode (129,114,60,60,119,57). Command: MLSD Error: Connection timed out after 20 seconds of inactivity Error: Failed to retrieve directory listing

ADD REPLY • link written 14 months ago by auradunbine • 20

Just provided some "unavailable" connected information. OS: Win 10 (firewall closed)

Setting: FTP, File > Site Manager set up for usegalaxy.org as a site, using "Connect" on that form (explicit)

Error: The data connection could not be established: ETIMEDOUT - Connection attempt timed out

FTP, Filezilla > Settings, set at active mode (saved), port 990 specified, using "Quick connect". Passive could also be set for this connection type. (implicit)

Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.56:990... Status: Connection attempt failed with "ETIMEDOUT - Connection attempt timed out", trying next address. Status: Connecting to 129.114.60.60:990... Status: Connection attempt failed with "ETIMEDOUT - Connection attempt timed out". Error: Could not connect to server Status: Waiting to retry... Status: Resolving address of usegalaxy.org Status: Connecting to 129.114.60.60:990... Status: Connection attempt failed with "ETIMEDOUT - Connection attempt timed out", trying next address. Status: Connecting to 129.114.60.56:990... Status: Connection attempt failed with "ETIMEDOUT - Connection attempt timed out". Error: Could not connect to server

ADD REPLY • link modified 14 months ago • written 14 months ago by auradunbine • 20

Thanks for the extra info about the problem. We are reviewing and will get back soon, Jen

ADD REPLY • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Is there a network administrator at your site who you can contact to help debug this issue from your side? I'm unable to reproduce this locally and it seems like something - either on the host or on that particular network - is blocking connections.

In the case of explicit FTPS, it succeeds in connecting on the control channel (port 21), but attempts to switch to passive mode - the PASV command. This instructs the FTP server on our side to create a random port (in our case, this will be between 30000 and 31000) for your client to connect to for data transfers. FileZilla attempts to get a directory listing from the data port with MLSD but times out when attempting to connect to the data port.

There could be multiple reasons for this, for example, there was a case we debugged a few weeks ago where the version of the FTP client (lftp) was too old to support TLS session reuse, which we require. A client upgrade fixed this. However, if you are using a recent version of FileZilla, this should not be an issue.

Another reason is that some firewalls inspect traffic to (remote) port 21 and watch for the response to the PASV command so they can allow outbound traffic from the client to the data port returned by PASV. Since the connection becomes encrypted after issuing AUTH TLS, the firewall cannot do this and thus connections to the data port fail.

As for the implicit FTPS connection, something is blocking your connection to the FTPS control channel (port 990). Some firewalls may only allow outbound connections to a certain list of ports, which may not include 990.

ADD REPLY • link written 14 months ago by Nate Coraor ♦ 3.2k

Please log in to add an answer.

But I found there is still a problem for some computers, unavailable connection seems not due to firewall or OS. I don't know why....? (firewall closed, OS is Win10 or Win7)

Setting: FTP, File > Site Manager set up for usegalaxy.org as a site, using "Connect" on that form (explicit)

Error: The data connection could not be established: ETIMEDOUT - Connection attempt timed out

FTP, Filezilla > Settings, set at active mode (saved), port 990 specified, using "Quick connect". Passive could also be set for this connection type. (implicit)

Similar posts • Search »