UCL Case Study - TrueNAS - Welcome to the Open Storage Era

MULLARD SPACE SCIENCE LABORATORY AT UNIVERSITY COLLEGE LONDON (UCL)

London, United Kingdom
https://www.ucl.ac.uk/mssl
Astrophysics, Research, Science

THE STORAGE CHALLENGE

University College of London’s Department of Space and Climate Physics (Mullard Space Science Laboratory – MSSL) is a world-leading research organization and the UK’s largest university-based space research group. MSSL delivers a broad, cutting-edge science program, underpinned by a strong capability in space science instrumentation, space-domain engineering, space medicine, systems engineering and project management.

MSSL’s scientific research ranges from cosmology and the study of extragalactic objects, to studies of the solar system, the Earth, and humans working and living in space. MSSL continues to research and develop the next generation of space instrumentation, and since its inception in 1966, MSSL has taken part in more than 35 satellite missions and over 200 rocket experiments.

MSSL demands the most from their storage. Requirements range from handling tens of millions of surveys at 64KB to space probe datasets at 10GB+ each and just about everything in between. MSSL needed to replace its aging clustered SAN file servers.
The clustered SAN had been chosen for reliability and failover, but distributed lock management led to degradation in throughput and latency. In addition to user impact, the inadequate performance had severely impacted MSSL’s large backup requirements and threatened business continuity should large-scale data recovery be needed.

WHY MSSL CHOSE TRUENAS®

When MSSL began to look for a new storage solution they were seeking a solution with better latency, higher performance, improved file-serving reliability, and the ability to scale well with the rest of the file-serving system. Having experience with large disk setups, MSSL found that many of the types of solutions they had used in the past still had significant file-serving performance limitations that would be difficult to overcome; they were looking for a proven commercial storage solution that met their needs.

Alan Brown, the Networking and Unix Manager at MSSL for over 15 years, first encountered ZFS on Solaris but became even more interested as ZFS was ported over to the Linux and BSD-based operating systems. Alan continued to follow the development of ZFS and OpenZFS (the Open Source variant of ZFS). He recalled previous experiments with ZFS where massive amounts of spinning disk storage were coupled with SSD read cache and backed by write caching, reducing the seek load on mechanical drives dramatically. These features would be very helpful with some common MSSL processing patterns which repeatedly read the same files.

Alan decided to research how different storage solutions were leveraging the power of the OpenZFS file system and how it compared with ZFS. After browsing through products from many different vendors, Alan was happy to find that TrueNAS by iXsystems is built on OpenZFS and can process a massive level of read activity. TrueNAS caches reads of the same file, allowing repeated access with next to no performance degradation. This would be a huge improvement over the older non-cached system MSSL was using and would alleviate one major bottleneck. Alan also reviewed SAN-clustered file systems but found that they still typically have poor latency performance due to internodal voting requirements. Alan also found that OpenZFS would eliminate the periodic offline file system checking needed in Ext4, GFS2, etc. and the associated disk non-availability

The OpenZFS file-system is well known among storage experts as one of the most resilient copy-on-write file systems available. It uses data integrity checksums and replication of important blocks to ensure that your data is safe and secure against various types of common failures. MSSL concluded that TrueNAS was a natural fit for their storage requirement because of the proven track record of the ZFS file system and iXsystems’ reputation for producing quality engineered products.

“ The most compelling extra feature we found with TrueNAS was that it’s based on OpenZFS. Unlike traditional RAID systems, when TrueNAS detects an error from a hard drive it not only corrects it but simultaneously writes the correction back to the hard disk. This is a valuable feature that mostly goes unnoticed but is critically important in large storage systems.”

Alan Brown

Conclusion

MSSL deployed a TrueNAS storage array with High Availability Unified Storage Appliance with over 200 TB of storage. The TrueNAS storage array enables MSSL to grow to well over 1PB. As their data grows, MSSL can easily upgrade their storage controllers, enabling them to scale to a massive 5PB, while meeting their data bandwidth and IOPS requirements.

After implementing their TrueNAS storage array, MSSL has seen a real world improvement in simultaneous data throughput by up to 50 times. Even with this improvement in throughput, MSSL is estimating that they have headroom for a ten-fold increase in traffic.

TrueNAS was a solid choice for MSSL because it provides stability and resilience beyond what is commonplace in the enterprise storage industry. TrueNAS allows MSSL to have centralized laboratory storage for millions of files that MSSL curates, increased reliability, and the scalability MSSL needs as their repository grows. iXsystems provides MSSL with an extra spares kit for their TrueNAS storage array as well as next-day hardware support if they need assistance.

Requirements

Scalable infrastructure to grow from 200 TB to over 4.5 PB
Protect over Millions of videos and audio files
Ensure public availability of data while keeping systems secure

THE STORAGE CHALLENGE

The JFK Library Foundation has a long term plan to digitize and store millions of documents and large amounts of videos, film, and audio recordings. Not only is the material vast, but it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types and sizes. The library required a high-end NAS system capable of sharing with a variety of systems as the digitization process is completed.

Performance aside, the platform had to be accessible to various thirdparty platforms, while maintaining data integrity. The storage had to be capable of being managed independently from any digital asset management software, so that the library’s independent services could easily access the data. Admins, users, and staff all require varying levels of access to the storage infrastructure to ensure against data loss from theft or human error. Data itself then needed to be incredibly secure, so that high resolution, uncompressed media can remain error-free from the time it is stored.