The Power of Linux

When the Association of Moving Image Archivists needed a solution to the film preservation crisis-half of all film before 1950 is lost and the rest is fading fast-it gathered tech-savvy researchers from across the country to solve the problem.

A collaboration with the Library of Congress, the National Science Foundation and three universities led to the Moving Images Collection http://mic.imtc.gatech.edu/index.php, a site that brings together thousands of groups that preserve, archive and consume moving images.

This required the most open and standard technologies available. A team of technologists from the University of Washington, Rutgers University and Georgia Institute of Technology chose Linux on POWER as the foundation for the project.

'I've always had good success with AIX* on POWER,' says Jim DeRoest, assistant director of computing and communications at the University of Washington and coprincipal investigator for the NSF grant.

'Because of the NSF grant, cost was an issue, so we went with SUSE LINUX and a variety of open source and free tools, such as Postgre SQL and IBM* Directory Server, all running on IBM pSeries* machines.' Once the pSeries servers at the University of Washington and Rutgers University were installed, the value of open standards began to shine through.

'We had no experience with Linux before this installation,' says Grace Agnew, associate university librarian at Rutgers and the principle investigator for the NSF grant. 'But we were pleasantly surprised at how easy it was. We had the servers up and running within a week. It was trivial.'

Linux on POWER collaborations like this are cropping up all over the world. The combination of cost, performance, ease of migration and open standards make Linux on POWER the choice for a growing number of developers. Brian Connors, vice president of Linux on POWER technologies for IBM, says since Linux on POWER was introduced in the third quarter of 2003, its growth has doubled every quarter.

Connors emphasizes that Linux is a relatively young platform compared with the elder statesman UNIX*. According to IDC, Linux held only 16 percent of the 2003 server market, compared with about 40 percent for UNIX. Connors points out that those numbers are misleading because many enterprises use UNIX on a primary partition with Linux on secondary partitions-a configuration that IDC counts as a UNIX server.

Still, IDC predicts that by 2008, Linux will close the gap (even as they currently count server operating systems, capturing 30 percent of the server market. Connors says IBM has several strategic initiatives to help close that gap, both in hardware and software. 'Some of the enhancements in POWER5* make up for the immaturity of Linux versus UNIX,' he says.

'And the IBM Linux Tehnology Center has been instrumental in contributing to the 2.6 kernel, which will allow Linux to scale up to 16 full processors. When you add POWER5's subprocessor partitioning capability, it brings Linux closer to UNIX.'

Recent Linux on POWER announcements will accelerate the closing of the gap. 'With the advent of the recently released IBM BladeCenter JS20 models, we expect Linux on POWER to be a strategic investment for a growing number of customers,' Connors says.

'The combination of POWER processorsand the 2.6 Linux kernel will help Linux on POWER close the gap with enterprise-level hardware.'

Operating system cost wars

The cost issues in Linux's favor are hard to quantify. Though distributions like SUSE and Red Hat aren't free, most of the Linux tools are free and open. For the MIC project, for example, it represents tens of thousands of dollars in initial cost savings.

But some studies claim that the total cost of ownership for Linux is higher than it is for competitors such as Windows NT because of the higher cost of qualified labor. The studies suggest that because Windows NT administrators are more prevalent than Linux administrators, NT administrators will be cheaper and easier to recruit, thus lowering server costs over time.

Douglas Welsh disputes those studies. Welsh is a lecturer and senior professional technical staff member for the department of molecular biology at Princeton University. The department is host to a sophisticated genetic analysis database accessed by researchers all over the world through the Web-running Linux on POWER. Welsh, the project manager for the database, says one of the main benefits to running Linux is readily available talent.

'Linux is now the standard for teaching computer science and programming at the university level,' he says. 'The people we hire have been trained and are familiar with Linux systems. That makes it cheaper and easier to administer Linux over time.'

Some studies, including one as early as July 2002 conducted by the Robert Frances Group (RFG), confirm Welsh's position. The RFG study shows that, on average, one Linux administrator can administer significantly more servers than NT administrators can.

Taking a host of variables into account-initial cost, labor cost, administrators per server processor and relative security-the RFG study concluded that the three-year cost of a 100,000-hit processing unit was significantly different among three systems: Sun Solaris costs $561,520, Windows costs $190,662 and Linux costs $74,475. The study was conducted on x86 systems to control for hardware variables.

Chip performance wars

The abundance of inexpensive x86 hardware makes it difficult to compare the costs of POWER5 systems to x86 systems such as the IBM xSeries* servers. Where POWER shines is in the workloads it can assume per processor. With its mature 64-bit architecture, POWER5 compares favorably with Sun SPARC on performance, making comparisons to x86 costs similar to apples and oranges. Connors says the overwhelming market share advantage for 64-bit processors goes to POWER.

'Sixty-four bit POWER has been around for five generations, whereas other x86 64-bit processors, such as AMD's Opteron, are in their first generation,' he says. 'There are a lot of ways this maturity plays out. For example, Opteron only scales to two-way.

'A better comparison for Opteron is HP's Itanium 2. Opteron is the clear choice there,' he adds. 'Itanium is dead. They merely took what's been there for 15 or 20 years and added some extensions. That's why HP only shipped maybe 10,000 units, whereas we've shipped millions of POWER chips in its eight years on the market.'

When you consider how much more work a single POWER5 processor can handle versus a single x86 processor, you begin to see savings in server consolidation and performance improvement. Welsh, who upgraded his Princeton genetics analysis database from 32-bit x86 to 64-bit POWER4* processors, says the performance jump was 'probably an order of magnitude.'

When you add in multithreading, subpartitioning and other performance enhancements in POWER5, it becomes clear that POWER5 is in another league altogether. 'The end result is a two-by performance improvement over Linux on 64-bit x86,' Connors says. 'And while SPARC continues to lose ground, POWER grows 34 percent year after year.'

Migrating Penguins

The Princeton genetics database and the MIC project have one thing in common: The technologists leveraged expertise in other platforms or architectures to seamlessly move into Linux on POWER. DeRoest's team at the University of Washington applied AIX on POWER expertise to Linux on POWER.

Agnew's team at Rutgers applied Sun Solaris on SPARC expertise to Linux on POWER. And Welsh's team at Princeton applied Linux on x86 expertise to Linux on POWER. The central theme is that Linux is close enough to UNIX and runs on enough common platforms that it makes for an easy migration path.

Welsh says the upgrade from Linux on x86 to Linux on POWER was transparent. 'One of our main goals was to get a platform that ran a 'standard' version of Linux (Red Hat, SUSE, etc.) rather than a customized, off-the-beaten-track version,' he says. 'That was one of the primary reasons we chose IBM. From our developers' perspectives, the technology is the same, just the box is different.'

On the MIC site, collaboration drives much of the move to open standards. The site consists of a union catalog and an archive directory, which together comprise the largest database of moving images in the world. The union catalog, under development at the Rutgers site, brings together hundreds of thousands of records of moving images-films, videos, newsreels and the like-with a description of the owners of those archives.

It enables archive owners to write the metadata descriptions of their film or tape resources. The directory, under development at the University of Washington site, is a Lightweight Directory Access Protocol (LDAP) technology that allows users to search and retrieve the information at the union catalog, whether or not they can access the actual archives.

In addition, the Georgia Tech development focuses on the Web portals and other front-end elements of the site. Agnew says a main driver in the choice of Linux on POWER was to pave the way for future work on the project.

'For a project like MIC that could potentially scale into 3,000 different organizations with multiple archives or collections with millions of individual listings and millions of hits per month, we needed something that could scale,' she says.

'But we also wanted something that we could easily migrate and replicate, if necessary.' Jane Johnson, MIC project manager and the overseer of the project for the LoC, says, 'I continue to be amazed by how well the project goes. The standards-based approach has enabled this to run as smoothly as it has.'

A cardinal rule of collaboration is to establish common ground before getting down to business. Power Architecture runs on just about every platform, from Nintendo game cubes to Apple Macs to midrange servers to large supercomputer clusters. Linux runs on everything from embedded systems to mainframes.