Two press releases caught my eye this week that aren’t exactly earth-shattering, but got me thinking about the way the storage market is changing and widening.

First, SanDisk revealed that its flash cards are recording footage of an excursion to Everest by a three-member climbing team sponsored by Dell, Windows Vista, MSN and MSNBC. Here’s a media gallery of the chilly-looking expedition so far.

Then there was also an announcement from RAID, Inc. of its compact Razor RAID array using 2.5-inch SAS drives, billed as “ideal for small spaces such as cockpits, tanks, submarines and other civilian applications with specific space constraints.” The ‘cockpits’ idea got my imagination going.

Between flash memory, with fewer moving parts and power requirements, and small-form-factor hard disks, not to mention the continued increase in content we store digitally, enterprise-level data storage is worming its way into unheard-of environments. As such, many in the industry have been predicting an increasing focus on edge devices, mobile computing environments and the mobile workforce for the storage market. Hopefully enterprise storage managers are paying attention to these new frontiers while architecting storage at headquarters.

Also, since it’s Friday, and who couldn’t use a laugh? Check out this priceless Gizmodo post on an internal Microsoft sales video that recently made its awkward YouTube debut. Key line: “You’ve gotta wonder how, in a company the size of Microsoft, there’s not a single person who [can] step up and say “Hey, you know what? This Vista music video we’re making for the sales department, complete with a cheesy Bruce Springsteen impersonator and horrible music, damages the dignity of not only everyone involved in its production, but everyone who watches it.”

I love listening to NPR. I listen to, watch and read many news sources, but I find the stories they choose and the nuances they bring to their reporting refreshing. I was listening to NPR this morning when a very rare thing happened–I heard someone being interviewed that I’ve interviewed before myself. It’s not often that IT industry news makes a mainstream general-purpose broadcast, so I paid close attention.

The pundit in question was Rob Enderle, a technology analyst I interviewed last month when EMC acquired Pi. After hearing his brief comments on the current state of the US economy and how he predicts it will affect technology innovation in Silicon Valley, I called him up myself and dug a little deeper into the matter with him.

Read more »

The storage market is a vibrant one right now, and as social networking concepts like blogs become more popular in the corporate world, the storage industry has a lively, varied blogosphere to match. Below are examples of some of the more interesting commentary I’ve seen lately, in case you missed them.

Read more »

Or so says Pliant Technology, a new company that just received $8 million in Series A funding. It’s comprised of former execs from storage companies including Maxtor, Quantum, Fujitsu and Seagate.

The cast of characters is as follows:

• Jim McCoy, Chairman - Co-Founder of Maxtor and Quantum
• Amyl Ahola, CEO - Former CEO of TeraStor, vice president at Seagate and Control Data
• Mike Chenery, President/Founder - Former vice president of advanced product engineering at Fujitsu
• Doug Prins, Founder/Chief Architect - Former consultant for Fujitsu, Emulex, and Q-Logic
• Aaron Olbrich, Founder/CTO - Formerly at Fujitsu and IBM

And that’s just about all we know in detail right now about Pliant. I spoke with McCoy this week about the announcement of funding; he said the company has decided to come out of stealth now, but has been working on perfecting the solid-state drive for the last two years.

The new company is aiming to improve the solid-state drive with its products, which are due out by the end of this year, with alpha and beta testing scheduled beginning this summer. Pliant’s drives will perform better than current flash drives, “closer to what the DRAM people have,” McCoy claims. The drives have also been “designed for a 24×7 operating environment, with error rates equal to or better than hard drives.” Specifically, the drives are going to tackle an issue McCoy says has been a dirty secret in the solid-state game: read disturb, a phenomenon in which reading data from one portion of a flash drive causes degradation in nearby bits.

Existing solid-state vendors have tried to address this problem, as well as issues with write endurance, using error correction codes (ECCs). But according to McCoy, ECC is not enough. “ECCs are a minimal starting point,” he said. “By themselves, they are not sufficient.”

If that gets you all wound up about the state of solid state, though, you’re going to have to wait to find out how exactly Pliant plans to build a better mousetrap. The specifics of its technical approach are “confidential at this point,” said McCoy.

Will the new and improved Pliant drives be able to do anything about the acquisition costs that are keeping many users away from solid-state drives right now? “There won’t be much of a price penalty over other [SSD products],” McCoy said, which I’ll take as a no. McCoy did point out that long-term, solid state is more cost-effective than over-provisioning hard drives.

The problem is, users rarely start from scratch; many will have over-provisioned hard drives already, and would need to start by adding very expensive SSDs on top of already very expensive assets. “Customers are reaching the end of possible performance with hard drives,” McCoy countered. “And new systems [like EMC’s Symmetrix] are going to start going out with a combination of drives.”

According to research from IDC, performance and mobility-related requirements will propel SSD revenues from $373 million in 2006 to $5.4 billion in 2011, a 71% CAGR. And I’ve heard many in the industry lament that while the capacity of spinning drives has been going up continually, the ability to get data off those drives faster is not keeping pace. Something will obviously need to change.

Meanwhile, the answer to the question of exactly how Pliant’s products propose to be a catalyst in that equation remains in stealth for now.

Some of you may have heard of Cleversafe, until now an open-source research project working to develop the prototype of a system that would automatically spread data over geographically dispersed grids while encrypting it.

Cleversafe has been making slow but steady progress over the last year and a half or so and have been keeping me updated. Their concept, is an interesting one: a way to automate the “chunking” of data over geographically dispersed nodes through new algorithms that also make each chunk of data unreadable, essentially combining primary storage with disaster recovery and data security all in one go, as our friends across the pond would say.

So far, Cleversafe has launched itself as an open-source project, invited developers to play with the Dispersed Storage Network (DSNet) prototype, and signed up 14 internet service provider (ISP) partners to pilot the service. This spring, those partners will begin to sell some actual software and hardware to go with the pie-in-the-sky concept.

The new products, which will be generally available May 31, include a storage node, called the Cleversafe Slicestor; a storage router, called the Cleversafe Accesser; and a software management console called the Cleversafe Manager. Each Slicestor will hold 3 TB raw in a 1U pizza box. There is no formal restriction on the number of Slicestors and Accesser nodes in one grid, but the first products will be offered in groups of 8 and 16 nodes, with a 4:1 ratio of storage to router nodes recommended. The nodes can be kept in a single rack in one location or distributed globally. Cleversafe says its business model will be to offer its grids directly to enterprises, as well as ISPs and managed service providers who can offer Cleversafe storage as an online or hosted service.

This is the kind of stuff that really intrigues me in the storage market–the kind of stuff that makes me envision Conan O’Brien with a flashlight under his chin singing “In the Year 2000…” The futuristic stuff. As a general, all-around nerd, it’s interesting to me to talk to the people planning the next generation of technology, to learn what the challenges are and what goals their sights are set on. The Cleversafe concept is a particularly interesting one to me given the global-scale DR challenges we’re beginning to face.

When we chatted about it last week, though, Taneja Group founder Arun Taneja tempered my enthusiasm with the reminder that future products are just that: in the future, and the proof is in the pudding. “At the concept level I’ve never had any issue with Cleversafe,” he said. “But while the concept is interesting, provability will take a long time.” Cleversafe must show its product can support multi-tenancy environments reliably, without mixing up data chunks, and must show that its performance and ability to recover data are what it says they are.

And while some of the deepest innovations in technology are happening around storage, Taneja also reminded me that the market for storage products remains more conservative than most. “Even if Cleversafe can prove that this is the best thing since sliced bread, the GMs, Fords and Pepsis of the world would have to test something like this for years before they’d trust it,” he said.

So we might not be looking at The Storage Internet ™ anytime soon. But I’m going to keep watching.

I can hardly call myself a storage geek–I don’t know a MAC address from a Macintosh and couldn’t operate a CLI with a gun to my head. So it’s rare I take a personal interest, the way Tory does, in most of the products or trends I cover.

The one exception to that is the idea of digital preservation. This is probably because, unlike a true storage geek, I don’t have to worry about trying to fix a machine that’s broken or trying to throttle my service engineers. So I have time on my hands to think about the long-term future of data, data storage, and what we’re going to do with all the important records that are currently being converted from physical format to digital. Spinning disk still has nothing on a cave painting, data tapes have nothing on an acid-free paper book, and in 100 years, we might have an unprecedented historical problem: how to preserve our culture and our information for future generations.

That’s the kind of thing you don’t have to be able to architect a storage fabric to be affected by. Every living person has a vested interest in how the human race will pass on knowledge and information over the long haul.

The problem is, those of us with the time to think about this stuff aren’t the ones who know how to answer that question, and the ones with the know-how are too busy putting out day-to-day fires in their data centers to worry about how it’s all going to work when they’re long gone.

And who says it’s their (your) responsibility anyway? Shouldn’t institutions like the National Archives be the ones worrying about it? Shouldn’t the storage vendors be the ones developing the right media for long-term storage?

As of this week, there’s finally a publicly-funded consortium at least trying to find the answer to those questions about digital preservation, all leading up to the biggest mystery of them all: Who’s going to pay for it?

The consortium, known as the Blue Ribbon Task Force on Digital Preservation, was launched by the National Science Foundation and the Andrew W. Mellon Foundation in partnership with the Library of Congress, the Joint Information Systems Committee of the United Kingdom, the Council on Library and Information Resources, and the National Archives and Records Administration. The consortium, headed up by academics from the San Diego Supercomputing Center, will attempt to bring together testimony from a variety of sources — consumer and enterprise, vendor and end-user — to arrive at a sustainable economic model for digital preservation.

The group has been funded for a two-year project. The first year of the project, according to Francine Berman, Director, San Diego Supercomputer Center and High Performance Computing Endowed Chair, UC San Diego, and co-Chair of the Blue Ribbon Task Force, will produce a report on “a survey of what we know.” The initial report will feature case studies and opinions from experts in digital preservation, and is expected to appear by the end of 2008 or early 2009. By 2010, the task force hopes to have a second report suggesting an approach to digital preservation that’s the most cost-effective and logistically feasible for the most people.

It’s all a little loosey-goosey, Berman admitted, saying, “These are open questions.” So far the group doesn’t have much idea what its direction will be. Alternatives for economic models that will be taken into consideration include an iTunes-like pay-per-use model; a privatized model relying on corporations to finance preservation; or a public-goods model that preserves digital records the same way public parks are preserved, through a collective public trust.

Further complicating matters, “there won’t be a one-size-fits-all solution to the digital preservation question,” according to Berman. Consumers will be concerned with preserving family photos, for example, which will be an entirely different process from preserving corporate and government records. Preserving digitally-recorded works of art and multimedia files will be yet another issue to resolve.

Personally, I’m a little reluctant to put much stock in a government study until I see it produce actionable results, and as a taxpayer I’m not nuts about the number of studies my hard-earned dollars go to that just tell us things we already know. But in this case, I’m just happy someone’s thinking about it. And maybe getting others to start thinking about it a little more, too.

Raising awareness is another goal for the task force, Berman confirmed. “My dry cleaner knows what global warming is, and could also probably give you a basic definition of the human genome,” she said. “What we’re looking for is that same level of understanding about digital preservation, which also affects us all.”

Even my friends who don’t normally follow the storage business are atwitter over an Engadget report that Buffalo has unleashed a 100 GB behemoth flash drive upon the world. Geeks everywhere are probably salivating to take the thing apart (yes, I’m looking at you, Tory) … unfortunately, they’ll have to wait. The catch is that Buffalo is only releasing the product for now in its home country of Japan.

According to company reps, the $1,000 asking price for the credit-card sized USB accessory makes it less than cost-effective to import right now. (If you just can’t get enough flash memory, there are 64 GB monsters roaming North America.)

The Engadget comments section also contains an interesting discussion of the merits of such a large flash drive. In the Engadget screenshot, the card looks like a  behemoth, but the post says it’s about the size of a credit card. Still, it launched a spirited discussion that I think asks some pertinent questions, namely, “would it not be more practical to just buy a $300 travel drive?”

At this juncture, and at this price point, certainly. But Moore’s law waits for no man, and the price of a 100 GB card will come down. Hence the other questions that this announcement begs: at what capacity and price point does a mechanical drive become more practical than a solid state drive? How will that equation change over time? It’s something we in the storage market are going to have to examine more closely in the coming year.

Hitachi GST is back at it again this week with another update to its disk drives, this time with a redesign of its desktop SATA and PATA drives for power efficiency. Hitachi claims the updates to its silicon on the new Deskstar P7K500 drive can reduce the drive’s power consumption by up to 40 percent–or down to as low as 6 watts when active and 2 to 3 watts while idle.

The new specs were accomplished in a couple of different ways, one of which is the use of a new system on a chip model for power modules, and changing the power regulator on each drive from a linear architecture to a switched one. The moves were made with the new Energy Star 4.0 spec for PCs released in July, which allots a “budget” of 50 watts in idle mode for the whole system while idle, of which it’s estimated 8.3 go to the disk drive. With the new 250 GB version of the Deskstar, Hitachi is claiming a draw of 3.6 watts in idle mode, and 4.8 watts for the 320, 400 and 500 GB models.

This savings won’t necessarily make a dent in anyone’s home electric bill, according to Lee Johnson, 3.5-inch Product Marketing Manager for Hitachi. “But with the additional watts left over, PC makers can use that added wiggle room to design PCs with more RAM, more features on the motherboard, or a higher processor clock speed,” she said.

Hitachi plans to add similar power-savings technology to its enterprise-class drives, but IDC’s John Rydning says that may not necessarily be practical–nor lead to significant cost savings in enterprise disk systems.

“At the enterprise level there’s not a lot of impact on the overall system by reducing idle drive power draws,” he said, noting that turning drives completely off through MAID is the way the enterprise is headed. “But if you’re a large enterprise organization with hundreds or thousands of PC workstations, this might make a difference.”

Not a day goes by that I do not hear from yet another storage or server vendor that their offering, whatever it is, is green. This mania started about a year ago in earnest. Prior to that, the green movement was pretty much restricted to organizations outside of the computer industry. So, what is really going on? What has caused every software and hardware company to suddenly formulate a green message.

I have a theory, and you heard it here first. I think there is a fundamental grassroots movement towards green that has started in the U.S. and it is picking up momentum like no other I have seen in thirty years. This moment is more powerful than the Presidential elections and other important matters facing the country. It is bigger than Exxon and Mobile. It is bigger than GM and Ford. For years, the debate has been raging about global warming. No matter which side of the debate you place yourself, the green movement has begun. And because now it is becoming fashionable, every company in every industry will feel the need to do something “green.” I believe we are now in the phase 1 of this movement. In phase 1, each company takes stock of what they have in their product line and extracts what they can of a green message. Granted most, if not all of these companies had never thought of any of their products in terms of green before. Not in product development and not in marketing. Of course, good design practices prevailed and many resulted in lower power usage or smaller packaging but they were hardly ever viewed within the context of green. So, in Phase 1, what we are seeing is a recasting of the company message incorporating green.

I see it every day. Sometimes I laugh when I see a storage company twisting and turning its message to incorporate green. Even to the point that more than one company has stated to me that they are so green that even their logo has green in it. Give me a break. The logo was done years ago when green was equated to the color of a person’s face when they saw a ghost.

But I frankly don’t care.

I am thrilled just to see the storage companies participate in the green movement. So what if 75% of what I see today is recasting of a message relating to an older product. So what if Manhattan’s energy and space crunch started the ball rolling. I think once the company is committed to the green message they will design their next product accordingly. They can’t escape it. That is why I believe the green movement will have a genuine impact in the next five years. Let the companies play the game. Play along with them. Give them slack for now. Because once they are in, they are in. I love it.

Before you think that there is nothing real in the products today let me restate something. There are technologies that have hit the market in the past three years that are making a serious green impact. Data deduplication is one such technology. It has hit the market on the secondary storage side first, that is, applied to backup/restore and archiving markets. When used in appropriate ways, one can reduce the amount of disk required by a factor of 20. No matter which way you look at it, 1 TB of storage uses a lot less power and requires a lot less cooling than 20 TB. Thin provisioning is another good example. I chose these examples to illustrate a point: it is not simply hardware technologies that deliver green. In fact, at Taneja Group we believe software will play a huge part in the greening of storage and servers. Not to say that hardware wouldn’t play a role. Look at Copan’s MAID technology, for instance. Or IBM and HP’s blade server technology. New techniques for airflow through racks, nanotechnology and new data center designs will all contribute. But, we believe the impact of software technologies will dominate, especially with installed hardware.

Green will soon become a competitive advantage. Because of the financial implications, real change will occur. Soon even the Mobiles and the Exxons will have to yield to the pressure. That is how strong grassroots movements are. I believe the time is here. And I couldn’t be happier.

Note: Recently Taneja Group wrote a Technology In Depth paper on this topic. If you would like a copy please send a request through www.tanejagroup.com.

IBM has reported two discoveries in its ongoing work in nanotechnology, both of which have implications for data storage of almost unimaginably tiny proportions.

According to a Reuters report, scientists at IBM reported yesterday that they have determined how to move the magnetic orientation of an atom, a key step toward using atoms as tiny storage devices. Each atom has a magnetic field that needs to be stabilized somehow before it can be used as the basis for a storage system where “bits” are not magnetized particles, as they are today, but atoms themselves.

IBM scientists in Zurich also announced that they have successfully “switched” the polarity of molecules, another key to computing on an atomic level. Currently, computer systems rely on the ability to “flip” magnetic particles to represent the ones and zeros of binary code. Being able to do the same with molecules, or even atoms, could eventually lead to microscopic computers, and breakthroughs in density on the order of 30,000 movies on an iPod, according to the researchers.

IBM’s not the only organization currently working on theoretical physics that’ll give you an ice-cream headache. Research revealed in July from the University of California Santa Cruz could lead to similar breakthroughs in the stabilization of magnetic fields on conventionally-constructed disk drives, the better to prevent data corruption. Industry experts pointed out that this research is most likely to be used for near-term density breakthroughs.

This research also represents a crucial piece of the puzzle for IBM’s atomic equations — figuring out the polarity of the individual bits or atoms is only half the battle. How those atoms or molecules or bits behave as a group on the surface of a drive is also a hurdle to be overcome before you can carry around a whole Blockbuster Video in your iPod shuffle.