Almost certainly this is due to your internet connection speed. The transfer to/from the public Main Galaxy instance at http://usegalaxy.org goes over a production network. We could probably find out the specs for Main, but up/down speeds are always a range and when moving data between two sites the up/down rate of both matters. Not very helpful for your situation since it won't impact you local.
Looking at your own network and the down rate will lead the to answer for file loading on your local (you'll know your own speed and can decide if upgrading is worth it, if an option). Export speeds will be a calculation that includes your up rate. This is true for any data provider access or other external data transfer downloads/uploads. There are many ways to check your network speed - google "speedtest" - your provider may offer a specific tool or there are general ones.
To my knowledge, local memory would NOT have an impact here, but perhaps that will be amended once others see this post.
Good luck, Jen, Galaxy team