ClusterMonkey.net: OK, OK, so there are plenty of File System articles on Cluster Monkey. But, from an administrators perspective, no good discussion of cluster administration can continue without coming to that thorniest of issues, file systems and I/O. In the long history of parallel computing, I/O in most cases could have stood for Ignored/Oops! instead of Input/Output. In the more recent history of Beowulf clusters, I/O has finally received some quality attention by some quality people, but there is still no "silver bullet" file system, and providing the right kind of storage for the right job remains one of the biggest headaches for cluster administrators.

Once again, this topic could fill many columns. So, in this column, we're not going to dive into the complexities of choosing your parallel file system, but rather discuss from the system administrators perspective when you should use a parallel file system at all, and when some other option is appropriate.

In truth, the file system itself isn't the whole issue. The design of the entire I/O subsystem in your cluster, hardware and software is a critical issue. Should you use disk-less nodes? Should you use multiple servers with SAN attached storage? These are important questions, and perhaps we'll get to them in the future, but for this month, let's assume a garden-variety cluster, look at the trade-offs, and hopefully we'll get more sophisticated over time.

Read more...