Swarm Object Storage Empowers NEP with Performance & Reliability Combined
Case Study
Media & Entertainment

DataCore Swarm Empowers NEP CDN and Delivers Exceptional Performance and Reliability

Download Case Study

From the Oscars to the Olympics, NEP provides the technology, the people and know-how to enable its clients to produce the world’s biggest live and broadcast events. NEP’s unparalleled worldwide network gives them the largest and deepest pool of outsourced broadcast engineering and production talent in the industry, providing solutions around the globe for Remote and Cloud Production, Studio Production, Video Display, Host Broadcasting, Post-Production Media Asset Management, Video on Demand, Digital Media Services, Playout and Uplink Communication.

NEP’s solutions are developed by award-winning in-house integration teams led by Gerbrand de Ridder, NEP’s Head of R&D and Lead System Architect of NEP, The Netherlands.

The initiatives are supported onsite and in real time by highly trained technical engineers who deliver customized solutions, superior service (24/7) and flawless execution to clients at every stage of the production cycle.

Gerbrand de Ridder’s team designed and deployed NEP’s CDN (Content Distribution Network) in the Netherlands, leveraging a large scale-out object storage platform with DataCore Swarm software.

The CDN back-end is a common use case for object storage but, in this particular case, NEP found a clever way to use object storage as a key element in delivering a great balance between performance and reliability of the system.

The Challenge

NEP realized that the shift by the media and entertainment industry towards digital content production and consumption demanded them to rethink their CDN strategy. The challenge was to discover a solution that met the growing needs of their 650,000+ accounts – all utilizing different consumption methods.

In addition, they needed an object storage solution that provided flexibility and scalability to meet their goals in terms of capacity, cost, throughput, growth and reliability, coupled with full hardware independence. With a combination of old and new hardware systems, it is critical for the hardware to be compatible in the same cluster.

The Solution

After a thorough evaluation process that included several vendors, DataCore Swarm was selected for its infrastructure flexibility and an attractive licensing model. Since the installation, NEP has grown to be an influential partner of DataCore, who listens to needs and feature requests while developing new and innovative file access methods such as SwarmFS and FileFly.

Even though a large part of data accessed at NEP is performed via API, there are still legacy application components that require file-based access – a common scenario across use cases.Swarm also demonstrated its ability to scale not only in capacity but also in performance. NEP has several versions of each piece of content to serve different types of devices and bandwidth.

As a result, the object store has copies of hundreds of thousands of files of varying sizes which, when not cached, must be streamed directly to the end user’s device.

Despite the complexity of streaming content of multiple sizes, Swarm provides impressive throughput, and has continued to improve after implementation of further infrastructure optimizations.

A large Swarm cluster configuration is the key and active component of NEP’s CDN platform. Originally deployed to store all available content, given its durability and reliability – now the least-viewed videos are usually streamed directly from Swarm before being cached on the edge network appliances if demand increases.

With more than 200 nodes from two different hardware vendors, the flexibility and scalability of Swarm object storage, paired with the software-defined approach has allowed NEP to build a platform that is perfectly tailored to its requirements.

We chose DataCore Swarm because it was the most flexible storage platform and is a key component to deliver the high-end quality video service NEP aims for.

Gerbrand de Ridder NEP

The Benefits

Internal benchmarks at NEP have shown that their multi-PB capacity with more than 200 active nodes is capable of delivering sufficient throughput that has made the DataCore Swarm cluster a key component in the delivery of the high-end quality service that NEP has always aimed for.

The system, even though large and critical for NEP’s business, does not require a full-time Storage Administrator – another major cost saving.

“Swarm provides good per TB licensing as well as continuous improvements in existing and new features.”

Gerbrand de Ridder

Even demanding system upgrades, such as capacity expansion activities, are performed with minimal user intervention with Swarm automatic load balancing. Object storage brought several benefits when compared to other storage technologies.

Swarm’s flexible erasure coding is used extensively by NEP to obtain the best level of space efficiency and robust data protection at the same time. Traditional RAID was eliminated, as it represented a significant threat of data loss as long rebuild times continue to grow with the increasing capacities of modern disk drives.

Infinite Scalability

Ability to rapidly scale the Swarm cluster with hundreds of nodes, and dozens of clusters as required – from a few 100 TBs to multiple PBs and beyond.

Unparalleled Flexibility

NEP has gained complete hardware independence with no vendor-lock-in with Swarm’s migration in-place technology.

High Throughput for Content Delivery

Swarm enables content delivery with parallel I/O processing delivering high throughput and performance at scale and allows end-users to seamlessly stream videos directly from the content archive

Distributed Access with Multi-Protocol Support

Using a combination of S3 and HTTP(S) protocols and APIs, Swarm ensures anytime, anywhere content access for NEP’s 650,000+ user base served by the CDN.

Greater Efficiency and Lower TCO

Leveraging a self-healing and self-managing software-defined architecture, Swarm reduces administration effort and automates data management.

Robust Data Protection

Replacing traditional RAID with flexible erasure coding and replication, Swarm guarantees the reliability of NEP’s media services and can be customized to alternate data protection techniques on the same hardware. Immutability, encryption, and Legal Holds, further enhance security measures.

Hardware Agnostic

Swarm doesn’t require any special hardware adaptations and can work with any standard x86 server hardware.