Case Study – Can Storage Ever Be Installed and Forgotten?

Can Storage Ever Be Installed and Forgotten?

INTRODUCTION
Even the most stable and reliable storage systems still require some degree of maintenance. Many may believe that firmware updates, software updates and replacement of failed storage components can never be avoided. Really? This does not have to be so, here is a storage system example that has been running for four years without any need for maintenance or unexpected downtime.

Back in 2017, Toshiba installed a ZFS storage system supporting storage infrastructure needs of the “Technology Experience Lab” at NTT Global Data Centers. Since then, the storage system has proven its outstanding reliability: except for a scheduled 10 minutes of downtime to install some additional ZFS features, the system runs steadily without component failures in any of the 60 HDDs, SSD, controllers, power supplies, fans, cables or other components.

TECH LAB EXPLORING NEW ARCHITECTURES
The Technology Experience Lab provides data center managers and their teams with the room and infrastructure to explore innovative approaches for architecting their servers and storage. It enables businesses to evaluate the efficacy of private or hybrid clouds, distributed architectures, and alternative approaches to deliver IT services in a low-risk environment. In addition, its community of users provides support and ideas through meetups, boot camps, webinars, and hackathons.

Server and Storage in the Technology Experience Lab
(Copyright: © NTT – Global Data Centers)

In total, 102 TB of net storage were desired, requiring 240 TB of raw storage, utilizing the ZFS-based JovianDSS software of Toshiba’s partner Open-E. This software had proven to offer high availability, no single point of failure, and high-flexibility, providing consistent snapshots and instant restore when required. The hardware to support it would need to be reliable and high-performance to take full advantages of the software, supporting several iSCSI block storage targets ranging in size from 10 TB to 40 TB, plus some shared file folders.

EXTENSIVE PLANNING WAS KEY FACTOR
The planning stage was the core for the long-term success of the final storage implementation. Toshiba often undertakes research into server implementation in its laboratories. It has resulted in close relationships with a broad range of component and software suppliers coupled with a deep understanding of what works well. Leveraging this knowledge, the team was able to recommend a hardware architecture that worked with Open-E JovianDSS and had proven itself in other projects.

Inside the Technology Experience Lab
(Copyright: © NTT – Global Data Centers)

RELIABILITY & OUTSTANDING PERFORMANCE
To form the 102 TB of net storage, the team selected Toshiba’s 4 TB, 3.5″ SAS Enterprise Capacity Drives (MG04SCA40EA). With their MTTF (mean time to failure) of 1,400,000 hours and a non-recoverable error rate of just ten errors per 1016 bits read, they were ideal for achieving the reliability required. Performance was not ignored either. The 7,200 rpm drives achieved a Zpool read performance rating of 12.9 x single disk and 8.5 x single disk for writes. For the ZFS write logs and reach caches, reliable 10 DWPD SAS Enterprise SSDs with 1.6 TB storage from KIOXIA (formerly Toshiba Memory) were selected.

A significant factor in HDD failures is heat, so the HDD enclosure chassis had to be closely reviewed before selection. The team selected AIC’s J4060-01 Dual Expander, 12 GB/s, 60 bay toploader JBOD. The 1400 W 1+1 hot-swap redundant 80+ Platinum power supply ensured electrical efficiency while also fulfilling the reliability requirements. The JBOD also features four 80 x 38 mm hot-swap fans. System testing showed that the coolest and warmest drive temperature difference was just 4°C, confirming that the correct JBOD had been selected. Connectivity to the JBOD was provided by the Microchip Adaptec® RAID Adapter ASR-8885 with 8 internal and 8 external ports, run in HBA mode. This model was highlighted as a top choice back in 2017.

Finally, the storage server and JBOD location in the Technology Experience Lab, at the Frankfurt 1 Campus of NTT Global Data Centers, was the last piece of the reliability strategy. Its 60,000 m2 of space offers exceptional network connectivity and its power is derived from two separate European power supply networks that reach directly to the server and drive enclosure. Physical security is ensured, while storage reliability is supported by their highly efficient cooling system.

Technology Experience Lab in Frankfurt
(Copyright: © NTT – Global Data Centers)

NO FAILURE IN FOUR YEARS
Until now the system runs failure-free for four years and the plan is to continue running the system to explore its reliability further – even when the supplier warranty ends for most of the enterprise-grade hardware components in one year. Based on field experience with Toshiba Enterprise HDDs, the team expects that the storage has many more years of failure-free functionality to offer.

Over the course of its operation, the storage solution has handled a multitude of proof-of-concept activities in the Technology Experience Lab, much to the satisfaction of all involved. “The Technology Experience Lab benefits a lot from this storage solution, and we are truly impressed by its continuous stability. Set up within our highly available data centers, I can’t imagine a more reliable way of running enterprise storage. We’re looking forward to many more years of failure-free operation”, said Dominik Friedel, Business Development Manager of NTT Global Data Centers.

SUCCESS STORY OF ALL PARTNERS INVOLVED
Having reviewed how the storage is used, with countless snapshots and scrubbing, and iSCSI block targets being connected and disconnected, week in, week out, the software has proven to be the optimal choice too. “I’m proud to see how well this installation works after all these years, but I am also sure that this is not all this setup can do. Excellent performance and reliability are ensured by the ZFS and Linux-based system in combination with first-class hardware, so it can last for decades. And that’s not just a supposition; Open-E has been developing ZFS data storage systems for over 20 years and we have similar, long-lasting installations like this in use at customers all over the world”, said Krzysztof Franek, CEO of Open-E GmbH.

Krzysztof Franek, CEO of Open-E GmbH

Perhaps most critical to the success of this project has been the collaboration. Deep expertise in each field is essential, but is not enough in isolation. Only by bringing the best of what technology has to offer together, and exploring its capabilities and options, it’s possible to successfully fulfil such demanding requirements. “We are exceptionally proud to have the partner network of Open-E, KIOXIA, AIC, NTT and Microchip. Even if your Toshiba Hard Disk Drives are incredibly reliable, they still need a reliable system built of many components to deliver real end-user value”, added Rainer Kaese, Senior Manager Business Development, Storage Products Division, Toshiba Electronics Europe GmbH.

Rainer W. Kaese, Senior Manager Business Development,
Storage Products Division, Toshiba Electronics Europe GmbH