E-mail explosion

The average e-mail is growing faster than you can say "supersize me". Your mail server may already have trouble digesting big attachments, but the problem is only going to get worse. While today's typical e-mail weighs in at a mere 20KB, analysts predict attachment types such as video will make 50M to 200MB messages common in the next few years. A steady diet of mammoth messages can overwhelm mail servers, delay delivery and increase the size of already large in-boxes, which slows back-up and restore operations.

"To manage an e-mail system, 50 to 75 percent of the cost is in the labour," says Michael Osterman, founder of Osterman Research, a company that analyzes messages. That chore is likely to consume more of your IT department's time unless you get mushrooming messages under control. A growing number of products can help you get a grip on expanding e-mail systems, including e-mail archiving software, caching appliances that store large file attachments and third-party file download services.

The trouble with traditional solutions

E-mail came to life as a system for short, temporary messages, but the in-box has evolved into the primary home of crucial business information for many people. In-boxes are already packed with large presentations and PDF files, but video will cause the real explosion. Video clips will become common in the next two years as people create video on their PCs, digital cameras and even their mobile phones, says David Via, an analyst at Ferris Research.

"The e-mail problem is not going away," Via says. "IT needs to think about it strategically."

The blight of bloated in-boxes is nothing new for market researcher Millward Brown. Many employees had 1GB mailboxes, and some even surpassed 3GB, according to Kean Millward, CTO of the company. "People were complaining on a daily basis that e-mail was slow," he says. "We suffered Exchange server outages and had trouble restoring in a timely fashion due to the sheer size of the user mailboxes." Instituting quotas for the size of in-boxes and file attachments won't solve the problem, as Millward found. Employers pay a productivity cost when users regularly spend too much time on mail tasks, such as combing through and deleting messages to comply with a mailbox size limit.

His organization unsuccessfully tried limiting employees in the UK and Europe to 10MB attachments. "This left people asking, 'What do I do when I need to send a file to a client?'" he says. "Our client-service people needed a good method for shipping large files around." These employees occasionally used third-party file download sites for attachments, but worried about security, he says. Some staff had to ask clients if they had an FTP site. US employees, who were not limited to 10MB attachments, caused problems when trading files with overseas colleagues. Plus, some executives had to be exempt from the rules.

Millward had to install hardware to solve the problem.

Perhaps you've considered using an FTP server to manage e-mails with large attachments. But that requires users to change their e-mail sending behaviour, and what is more important, FTP will eat up too much IT staff time.

"FTP is hard to manage," Osterman says. "An FTP server is just a junk drawer of messages. It's easier to control deletions with an archival system. You should have a retention schedule and a deletion schedule [for e-mail and attachments]. If you keep documents for one year, at one year and one day, the documents go."

Page Break

Archiving options

E-mail archiving software addresses not only storage headaches, but also compliance and discovery needs for many companies. "A court may order you to provide all the e-mails relating to a particular topic for several years," Osterman says. "You can't really do this with back-up tapes. The cost of finding and retrieving the e-mails would be extraordinary."

Construction company Webcor Builders installed Veritas Enterprise Vault e-mail archiving software in late 2002 for both reasons.

"We were having performance problems with our Exchange servers, says Gregg Davis, Webcor's CIO.

"We couldn't upgrade them fast enough. We had e-mail slowdowns too. The server would become unresponsive, and we'd have delivery delays." Everyday documents like building specs and contracts weren't getting any smaller. Simultaneously, regulatory requirements regarding documentation became stringent, Davis says. The Veritas software solved both problems, he says, taking the archived e-mail off the mail server, and offering search features necessary to round up particular e-mails when the company faced regulatory scrutiny or litigation.

Webcor's Exchange mail server is now holding its own, though the company has grown from 70 employees to close to 400. "We pretty much stopped having to expand our Exchange server," Davis says. Enterprise Vault "increased the pure efficiency of the thing".

Initially, Webcor was running Exchange 2000. The company rolled out Exchange 2003 between August and October 2005. Just before the rollout, Webcor upgraded Veritas Enterprise Vault to the newest available version to ensure compatibility.

Be realistic about your mail-server expectations, Davis says. "Exchange doesn't shrink. If your database is at 20GB and you deploy this, you'll stay at 20 instead of going to 25. You manage the growth."

When shopping for an e-mail archiving system, choose one that doesn't require users to change behaviour, Davis recommends. "Some other solutions require the users to do something with folders, categories or arranging mail a certain way," he says. "With Enterprise Vault, all you notice is an icon."

Veritas pricing depends on number of users and add-on modules; for mailbox archiving directly from users' in-boxes, it starts at $US14,719 for as many as 500 users and the base server licence. Davis spent about $20,000 initially and pays about $4000 per year in maintenance fees for Enterprise Vault and some add-on software modules.

Think carefully about the kind of searches your company will need to do on the archived e-mail. You want full-text indexing, which means you can search the message title, header info, message body and any attachments.

Archived e-mail that's not easily searchable could save space on your Exchange server now, but cause a major headache later, says Melanie Koch, controller and operations officer at ChemOne, a wholesaler and distributor of dry chemicals.

"We had a retention policy that no e-mails were deleted," Koch says. "We realized any discovery costs would be huge. From a business standpoint, we couldn't find all the documents we needed. Our Exchange server was overwhelmed, but just upgrading the Exchange server wouldn't solve the problem."

ChemOne turned to ZipLip, which makes a variety of mail archival and attachment management software: Koch installed ZipLip's Unified E-mail Archival Suite last October, choosing the product largely for the search features. For example, ChemOne sends many internal reports and documents around that have similar or identical names, so the company needed to easily narrow e-mail searches to particular date ranges, Koch says

ZipLip's products comply with Multipurpose Internet Mail Extensions (MIME), another factor to consider. If you choose an archiving solution with a proprietary format, then part ways with your archiving vendor a few years later, it could spell big trouble.

In addition, open standards such as MIME can prove key when companies acquire or merge divisions that may not be using the same mail and archiving solutions.

Page Break

Clever caching appliances

A new crop of caching appliances from Accellion and Intradyn can also help you manage the large-attachment problem. These boxes typically plug into the network (in the demilitarized zone) with little configuration and don't require much ongoing maintenance. They store large attachments and automatically delete them at the times you specify.

When a user sends a message with a large attachment, the caching appliance automatically strips out the attachment and stores it. The e-mail recipient gets a message, as usual, but with a clickable link that allows download of the file. So the Accellion device means much smaller messages -- for example, at 10K or 12KB, instead of 10MB. The box occupies no space on the company's mail server. And when the mail server crashes, there's less to restore.

Restore and regular back-up times will go down, says Millward, who solved his company's attachment dilemma by installing two Accellion Courier File Transfer Appliances in 2005. One device resides in the company's England location and one in the US. "We've had no crashes of the mail server since installing the Accellion appliance." He says. "Back-up times are smaller."

After a straightforward configuration by in-house IT staff, Millward set the boxes supporting 4500 employees to autodelete the large files after eight days. This automation is one of the main reasons Millward chose the Accellion product. It also integrates with existing Lightweight Directory Access Protocol e-mail system, so users' regular e-mail accounts and logons provide access to the Accellion box, he says. Millward spent $15,000 to $20,000 on the boxes.

As for maintenance, "It's virtually hands-off," he says, noting only a few software tweaks have been required, and Accellion did them remotely. The company is in the process of migrating to Exchange 2003 and has an eye on e-mail archiving software. The company's headquarters site will soon begin a CommVault pilot program for 500 users.

How do you know when a caching appliance is the right answer? If you deal with large attachments and don't need to keep them forever, a caching appliance is a plug-and-play solution. If you do need to retain, organize and search the e-mails, you'll want to use the caching appliance in concert with e-mail archiving software.

Accellion boxes cost $5000 and up, depending on processing memory and hard drive options. An enterprise version is designed specifically for multiple-site installs; replication capability allows multiple offices to have access to the same data on all the appliances. Millward went with the regular version. When examining the cost of any caching product, consider whether you'll also need to budget time and money for consulting or integration work.

The more attachments your company sends, the more value you will net from a caching appliance. For example, graphic design, architecture and engineering companies that deal with huge attachments day in and day out will see plenty of bang for the buck, Osterman says. "You're going to get a fairly significant value in a company like that. You have a parallel infrastructure."

Page Break

Download drop-offs

Third-party file download services (also known as virtual staging servers) offer an alternative strategy for dealing with giant attachments. Users sign up for these services, then "drop off" files that would otherwise be e-mailed as attachments; the e-mail recipient gets a message with a link that allows retrieval of the attachment. For example, Dropload.com provides a quick, basic and free service of this kind. The files (of any type, up to 100MB) go away after seven days.

On the downside, you will not have a record of attachments and will not be able to retrieve them for regulatory or legal needs if you choose this kind of solution. But it's easy and convenient for a smaller business, or as a stopgap choice for certain documents. You certainly can't beat the price.

Another option, Sendthisfile.com, delivers free accounts for individuals, with a few caveats. For example, you and your recipients would see some third-party advertisements, and e-mails must be picked up within three days. For a more professional image, an enterprise plan for business customers lets you customize the look and feel of the browser page that e-mail senders and recipients will see, and includes access to administrative reports detailing transfers. This also offers more-sophisticated management options, such as deleting files after a specified number of days.

Exchange 12 anticipation

As for the item at the hot spot in the e-mail explosion, the Exchange mail server, most companies have upgraded to the current version, Exchange 2003. The next version, Exchange 12, due in late 2006 or early 2007, could help manage e-mail problems.

The upgrade will be based on 64-bit technology, which allows more addressable memory and larger memory caches than today's 32-bit version. This should significantly reduce the I/O traffic travelling to and from the Exchange server's storage system, says Microsoft's Ray Mohrman, Exchange technical product manager.

Current Exchange servers are typically bound by traffic, not disk space. I/O traffic reduction should help administrators utilize more of a server's storage space. Another possible benefit is the ability to pack more user in-boxes onto one server. Exchange 12 requires hardware that supports 64-bit technology, as well as a 64-bit operating systems.

The 64-bit technology should also improve Exchange's performance with e-mail caching appliances, which are typically memory intensive, Mohrman says. Exchange 12 also will let Outlook users archive e-mail to SharePoint sites.

No matter which approach you choose, find a strategy for dealing with the great e-mail expansion before it gets any worse, Millward says. Rest assured, he says, users will continue to push your e-mail system with larger and larger messages: One of his users recently sent a 1.8GB file.