Gary Montry

The LCI Conference focus this year is on big clusters. Not necessarily on raw performance per se, but on every other factor required to acquire, host, provision, maintain, and achieve scalable performance for systems as a whole.

The first two keynotes set the tone by describing the perils and pitfalls of installing huge systems and getting them to perform. Even after a few years all of the pieces don’t necessarily play together well enough to meet the original design objectives. Horst Simon began the first day with an excellent philosophical discussion about the current state of high performance computing (HPC), hardware architecture, and the political atmosphere surrounding the drive to assemble the worlds’ first petaflop machines. He noted that even though we have started construction of a petaflop computer, there are presently only two general-purpose machines in the world capable of 100+ teraflops on the Linpack benchmark.

This was a perfect segue from the opening keynote Monday evening by Robert Ballance of Sandia National Laboratory about the difficulties of assembling Red Storm and getting it to perform. Even though Sandia has years of experience building and maintaining some of the largest supercomputers in the world, Red Storm turned out to be a unique experience for them. Why? Because it was much bigger than anything they had previously built. So the old saw in computing, “if it’s 10x bigger, it is something entirely new,” still holds, and we should not expect a petaflop machine to come together quietly at this moment in HPC time.

One interesting observation which Horst made in his talk is that programming a 100,000+ core machine using MPI is akin to programming each transistor individually by hand on the old Motorola 68000 processor, which of course had only 68,000 transistors. That wasn’t so long ago to most of us, and his point is that we can’t grow too much more in complexity unless we have some new software methodology for dealing with large systems.

The discussions generated by his comments never really addressed the fact explicitly that we are going to need new compiler technology sooner rather than later to handle the complexity. Neither MPI or OpenMP are the answers by themselves.

The rest of the talks on day one had a heavy emphasis on parallel I/O systems, and the difficulties of getting them to scale on large cluster systems. The problem here is that some of the tests can take so long (Laros, SNLA) that the production system would be unavailable for unacceptable periods of time. So I/O system administrators are forced to do simulations of the I/O systems on smaller development configurations. Presently, it seems that scalable I/O systems are limited to about one KiloClient (my term) for single-process/single-file I/O scenarios. Forget about it if you’re talking about shared-file I/O. I think this is still pretty darn good progress, but the performance variability of these I/O systems is large, and it appears that their performance is very sensitive to a huge number of environmental parameters. Repeatability seems to be somewhere over the HPC horizon.

Day two we will have several presentations on software, new systems, and roadrunner (LANL’s one Petaflop system). It should be a lot of fun.