SysAdmin to SysAdmin: Parallel I/O for clusters using PVFS
Posted by Kenneth Farmer, Wednesday December 15 2004 @ 09:19AM EST
Linux.com: When you're trying to share files and perform I/O-intensive operations across 100+ nodes of a Beowulf cluster, the old model of a central NFS file server handling every client request begins to break horribly. What many cluster admins do instead is limit NFS to distributing home directories across the cluster, and using some form of parallel I/O model for doing "real work."
Of course, cluster administrators and users alike crave the benefits of an NFS-like system; admins
(in many cases) want logins only to the head node of the cluster, and users want to have all of their output in one place so they don't have to collect it from the various local disks of the individual compute nodes. However, it has been proven time and again that NFS is ill-suited for the kind of pounding it would take in a large, I/O-intensive cluster environment. The answer to this problem has come from researchers, open source projects, and the private sector, in the form of a parallel I/O model.
IDC: Appro Xtreme-X Supercomputer Blade Solution
Analysis of the Xtreme-X architecture and management system while assessing challenges and opportunities in the technical computing market for blade servers. Video - The Road to PetaFlop Computing
Explore the Scalable Unit concept where multiple clusters of various sizes can be rapidly built and deployed into production. This new architectural approach yields many subtle benefits to dramatically lower total cost of ownership.
SpyderByte.com
SpyderByte.com
