cleanup_dataset doesn't free space

Question: cleanup_dataset doesn't free space

13 months ago by

n.goue • 50

n.goue • 50 wrote:

Hi, I set up a CRON to handle cleaning on our local Galaxy 17.05 server.

xxx/galaxy/config/galaxy.ini -d 10 -r -1
xxx/galaxy/config/galaxy.ini -d 10 -r -2
xxx/galaxy/config/galaxy.ini -d 10 -r -3
xxx/galaxy/config/galaxy.ini -d 10 -r -4
xxx/galaxy/config/galaxy.ini -d 10 -r -5
xxx/galaxy/config/galaxy.ini -d 10 -r -6

It looks working as it says for histories for example :

Processing history id 11 (Unnamed history)

Purging history id 11

Processing history id 13 (Unnamed history)

Purging history id 13

Processing history id 15 (Unnamed history)

Purging history id 15

Processing history id 16 (Unnamed history)

Purging history id 16

Processing history id 17 (Unnamed history)

Purging history id 17

Processing history id 18 (Unnamed history)

Purging history id 18

Purged 6 histories.

Elapsed time: 0.288854122162

For dataset I get a message saying : "No such file or folder "

Removing disk, file xxx/galaxy/scripts/my_scripts/database/files/000/dataset_994.dat

Error, dataset file has already been removed: [Errno 2] Aucun fichier ou dossier de ce type: 'xxx/galaxy/scripts/my_scripts/database/files/000/dataset_994.dat'

Purging dataset id 994

Removing disk, file xxx/galaxy/scripts/my_scripts/database/files/000/dataset_991.dat

Error, dataset file has already been removed: [Errno 2] Aucun fichier ou dossier de ce type: 'xxx/galaxy/scripts/my_scripts/database/files/000/dataset_991.dat'

Purging dataset id 991

Purged 116 datasets

Freed disk space: 15757221243

Elapsed time: 5.19438910484

However, if I look at the local server disk space, no space is free. Do I miss something ? I notice that in https://github.com/galaxyproject/galaxy/tree/dev/scripts/cleanup_datasets there are other scripts like admin_cleanup_datasets.py Should I include them at some point?

Thanks,

administration galaxy • 377 views

ADD COMMENT • link •

modified 13 months ago • written 13 months ago by n.goue • 50

13 months ago by

Nate Coraor ♦ 3.2k

United States

Nate Coraor ♦ 3.2k wrote:

When you run the cleanup scripts, your working directory should be the root of the Galaxy directory, e.g.:

cd xxx/galaxy; . ./.venv/bin/activate; python ./scripts/cleanup_datasets/cleanup_datsets.py [args]

If your file_path in config/galaxy.ini is set to a relative path, it will be expected to be relative to the Galaxy root directory.

ADD COMMENT • link written 13 months ago by Nate Coraor ♦ 3.2k

13 months ago by

n.goue • 50

n.goue • 50 wrote:

Hi Nate,

I tried relative and absolute paths with the same result. If I do df -h before and after running the script (outside my CRON), no space is freed. Talking about file_path, I didn't modified it, and the default one in my config/galaxy.ini is #file_path = database/files. What is the best practice, should I change it ?

Thanks!

ADD COMMENT • link written 13 months ago by n.goue • 50

Hi, actually it did some free space, 2Mb, But log said

Purged 137 datasets Freed disk space: 2446911937 Elapsed time: 25.5015599728

Nothing compared...

Would you have other hints ?

Thanks, N.

ADD REPLY • link modified 13 months ago • written 13 months ago by n.goue • 50

Please log in to add an answer.

Similar posts • Search »