It is not quite clear what the problem is, but I am guessing that the job is failing for memory?
The memory needed to execute is a job is distinct from the memory in your account quota.
Input problems can sometimes contribute to failures. This is how to check and how to get direct 1-1 help if you cannot determine the problem (with a bug report, if working at http://usegalaxy.org or can reproduce the problem there).
Thanks! Jen, Galaxy team