Basics of an efficient storage structure for Microsoft Exchange

In a very short time, e-mail technology has become a communication backbone for most of today's companies. In many cases, end users will now notice an outage of the e-mail system much sooner than the failure of other business-critical applications. IT service departments in particular can attest to that.

Users tend to store a vast number of inbound and outbound electronic messages and their attachments forever, and this underlines the increasing use of e-mail systems as a pool of information that plays a vital role for businesses.

The information in this pool needs to be managed efficiently, protected, and recovered if errors occur or data is lost. At the same time, the solutions deployed must not be tied to specific storage hardware in any way. Instead, for reasons of cost, they have to give users the maximum amount of freedom to adjust to new storage and network environments.

The best-known and most widespread messaging system is probably Microsoft Exchange. Messages and attachments of public and private e-mail folders are stored in Exchange's own message databases. Transaction logs are also kept and managed, as is typical for databases. Because of the volume of data involved, Microsoft Exchange is a very storage-intensive application, and it stands and falls with the underlying storage infrastructure. An efficient storage infrastructure must meet three key requirements:

• Manageability
• Availability
• Recoverability

Manageability
Storage administrators need to keep a close eye on their Exchange infrastructure and be notified at an early stage of any changes in the growth pattern of database files and transaction logs. Intelligent trend analyses of past and present Exchange capacities point out deviations from previous growth patterns on the spot and enable storage administrators to respond proactively to such changes - either by migrating e-mail attachments via hierarchical storage management (HSM) interfaces or by increasing the storage capacity.

Of course, it must also be possible to configure additional storage units in online mode without impacting the running Microsoft Exchange environment. Automatic detection systems for Exchange resources tell administrators about available physical storage systems.

From these, administrators can then generate logical volumes, i.e. virtual disks, in keeping with their capacity, performance and resilience requirements. These logical volumes can even extend across the boundaries of heterogeneous storage systems to ensure that they are big, fast and resilient enough to back up the Exchange information.

A storage configuration wizard can handle many tasks automatically and rule out operating errors from the very start. Such wizards not only guide users through the configuration process but also suggest the optimal storage layout - for example the size or RAID level of Exchange volumes.

In large Exchange environments comprising several servers, storage administrators should no longer have to carry out these configuration steps for every single server. Once a configuration has been defined for one Exchange server, it can simply be replicated on all other servers in the infrastructure. This ensures that identical configurations are set up, and makes it easier to manage them.

A configuration wizard for Exchange is available to configure new storage systems and integrate them in the Exchange storage infrastructure. It suggests a storage layout for the Exchange volumes for the specific environment.

Availability
Optimum availability of the storage infrastructure is based, on the one hand, on advanced data protection technologies and, on the other hand, on organisational measures that ensure that Exchange operations are not impeded while necessary management work is in progress.

Data protection technologies
It is vital to minimise planned downtime and prevent unplanned downtime, so this aspect has to be taken into account early - when logical volumes are created for Exchange. Software functions are available to generate resilient volumes with a RAID 5 layout or a mirror concept (RAID 1 or RAID 1+0) even from unintelligent physical disks and JBODs (Just a Bunch of Disks).

Redundant volumes compensate for the outage of individual physical disk systems in the selected storage layout. However, such situations mean that the storage infrastructure is more susceptible to other errors, which could then bring the entire Exchange application to a standstill. Hot sparing/hot relocation can provide additional security until the defective disks are replaced.

Hot spares are active but empty physical disks. An original disk with errors is replaced dynamically by a hot spare disk, and the formerly redundant data structure is re-established. This procedure, called hot relocation, reassigns the data blocks between the physical disks in a way that is completely transparent to users, and ensures constant data protection and redundancy despite the failure of a component. The original state can be restored when the defective disk has been repaired.

Changed capacity, data protection or performance requirements can nevertheless make it necessary to enlarge or reduce the volumes for Exchange, or to switch from one storage layout to another. Intelligent software solutions for volume management now do this online. Switching the storage layout to a multiple mirror, for example, is not only advantageous in terms of availability but also in terms of enhanced performance. Read accesses are then split up evenly across all parts of the mirror.

Organisational aspects - backup via off-host servers
With backup windows becoming ever smaller, backing up Exchange information can be a real challenge. It is no longer possible to back up large Exchange systems with conventional technologies in environments that need to be up and running non-stop - 24 hours a day, 7 days a week - without considerable effort and a negative impact on availability and performance.

The best way of backing up large Exchange systems is the off-host backup method. Off-host means that another computer that is independent of the Exchange server takes over the backup job while the Exchange application continues running normally. The off-host server must not access the original data in parallel to the Exchange server, though, because this would significantly degrade performance. Furthermore, a consistent backup level could not be guaranteed in such a case because new messages keep arriving.

The snapshot procedure, which freezes the Exchange data as a copy at a certain point in time, avoids these problems. For obvious reasons, this procedure is also called a point-in-time copy. The third mirror of an Exchange volume is generally used for snapshots. A hot snapshot function ensures that Exchange can continue working normally.

Special verification procedures guarantee the data consistency of snapshots. This part of the mirror is then detached from the Exchange server and made available to the off-host backup server. The snapshot volume ought to be on an external disk system so that the backup server can access it via a SCSI or Fibre Channel SAN connection. This server then backs up the frozen Exchange copy in peace without affecting the Exchange server.

The snapshot volume could theoretically be deleted once the backup has been completed, but there is generally no point in doing so as it would take a lot of system resources and time to set it up again completely for the next backup.

The snapshot volume is therefore reattached to the original data set as a multiple mirror, and is synchronised with the current data. To avoid complete resynchronisation it is advisable to deploy an intelligent fast resynch technology as provided by some advanced snapshot solutions. A data change map is used to ensure that only modified data is written to the snapshot volume. This saves a great deal of time compared with complete volume synchronisation, and makes sure that all parts of a mirror match completely.

Snapshot and off-host procedures are ideal for backing up Exchange databases. An off-host server backs up a detached mirror and thus reduces the load on the Exchange application server.

Recoverability
Point-in-time copies are not only the basis for off-host backups of the entire data set but are also used for fast recovery of corrupted or lost data without the time-consuming process of loading the last backup from tapes.

While the Exchange database is running, a scheduler controls the automatic creation of multiple point-in-time copies on online storage media at defined intervals. In the event of a logical error in Exchange, administrators initiate the recovery process on the basis of the point-in-time copy using the last correct data set before the error occurred. Then the transaction logs are automatically applied to guarantee data integrity.

Users should be aware that this quick recovery procedure cannot replace daily backup to tape. Still, it is an excellent additional way of reducing the loss of data and the time needed to recover it during critical business periods without jeopardising operations.