Archive for December, 2008

bbcp for fast network copies

Friday, December 5th, 2008

This is not strictly database or MySQL related, however DBAs often deal with large data sets so others may find it useful.

I recently discovered bbcp, a utility to copy files efficiently over a network. It has the ability to use parallel network streams and specify window sizes, which can dramatically increase the transfer rates. It copies via the ssh protocol, so it fits nicely into common security environments.

(more…)