Evolution Of Distributed Live Production

  • By Adrian Pennington

Evolution Of Distributed Live Production

Contributing Editor Adrian Pennington examines how live production is evolving in these ever-challenging times...

Over the past few months, the media industry has adapted to fewer or no on-site staff and the need to produce from the home. This has accelerated the migration towards distributed production solutions, which depend on connected technologies.

But this situation has also generated a lot of buzzwords and hyperbole. There is a need for clarity about the practical implementation of remote distributed live productions. We’re also now seeing end users moving away from the ‘quick fixes’ brought on by the rapid change of situation at the beginning of last summer, towards more sustainable solutions with long term investments, powered by 5G, AI, and IP.

According to Sony, the ongoing situation has exacerbated the need for live production companies to take the long and wide view. “An holistic approach to their production workflow is necessary,” says Norbert Paquet, Head of Live Production Solutions, Sony Professional Solutions. “This technological reset rests on their wider business transformation goals. With this in mind, pragmatic steps to shift into the future include assessing existing technical capabilities and people’s skills and setting these against the ambitions of the company. This will highlight the uptooling and upskilling needs.”

Paquet says that new technologies such as IP, Cloud, AI or 5G will be a necessary part of that uptooling process, but that success and Return on Investment will only come if the implementation is pragmatic and bedded in the reality of live production workflow and operations. 

“Ultimately that implementation will rely on people working together in a collaborative way to shape the new puzzle - and this is where, as much as technology, partnering with the right people will be crucial to success,” he says. “This is one of the many reasons Sony and Nevion have come together, proposing a holistic approach to live production, which brings together people and technology.”

Remote productions reduce the cost of a traditional OB by saving on travel and accommodation for staff and the rental of large OB trucks. In some cases, it also speeds up creation of the video content.

“Remote technologies will not lose their relevance after the pandemic,” says Igor Vitiorets, CTO, Slomo.TV. “Remote production is gradually transforming into a standard workflow.”

For Slomotion replays at live events, Slomo.tv have implemented a remote operator workstation technology. The operator, using a standard workstation connected to a special Linux driven box, controls the OB van server via the internet from anywhere. “Using H.264 proxy growing files recording to cloud-based storage, our servers can also provide quick editing for creation of the summary,” explains Vitiorets. “This technology is in demand for live TV shows. However, a high-speed and stable internet connection with low Ping remains the main precondition for the viability of distributed live production.”

LTN Global led the remote production trend by investing in a “cutting-edge” 30,000-sq-ft global production facility in Kansas City, Missouri. Centralised production facilities with full-time dedicated and experienced staff give access to state-of-the-art technology and industry-leading professionals to all industry players, levelling the playing field. 

“It’s now crucial that media and entertainment companies invest in long-term solutions that leverage the remote production capabilities and consistently deliver the highest broadcast quality,” asserts Mike Burk, General Manager, LTN Create. “In doing so, these workflows lead to cost savings and personnel efficiencies.”

LTN Global has also pioneered standardised home kits that can be dispatched to any location to enable high-quality, consistent production. Burk says, “Reliable and scalable production solutions that ensure broadcast-level quality are critical success factors and should be part of any media organization’s technology stack.”

Pooled resources based on software-defined processing devices such as Lawo’s V__matrix for video related processing and A__UHD Core for audio related processing already allow for decentralized, private cloud-type live operation.

“A variety of distributed production workflows can be mapped to these in a highly efficient way utilizing a resource-, schedule- and network-aware control system like VSM,” says Lawo’s Wolfgang Huber. “Lawo’s early adoption of the ST2110, AES67/RAVENNA IP standards for its entire product portfolio, and its reliance on the Ember+ protocol were the foundation for refinements released in the early days of the first COVID-19 lockdown wave in 2020. 

WAN-networked, distributed resource pooling, Mix Kitchen scenarios as well as remote factory acceptance and maintenance confirmed our long-term product strategy.”

Lawo point out that back in 2018, NEP Australia implemented a continent-wide remote production setup for sports telecasts that leveraged Lawo IP infrastructure. Only months later, other Lawo customers like Norway’s TV2, Belgium’s DPG Media turned remote production into a permanent vs. mobile strategy. 

“We expect that virtualization and efficient resource utilization in a dynamically scalable and geographically distributed environment will become the new normal. We are working towards simplifying the user experience in an increasingly challenging environment.”

Riedel Communications says it identified the enormous potential of distributed live production years ago and have been investing in associated technologies ever since. 

“The goal is to reduce the ecological and economical footprint for both us and our customers,” explains Carsten Voßkühler, Senior Project Manager. “Our ROC (Remote Operations Center) is a key result of these efforts, so when the pandemic struck there was no need for a hasty reaction and quick fixes as we had networks, products and solutions for remote workflows already in place.”

He points to the groundbreaking distributed production ‘Around the World in 80 ms’. For this virtual live event, Riedel’s ROC enabled all audio, camera and lighting operators to work remotely and in real time across three continents. 

“In the future, 5G will be a key driver for these productions, because quite often, our ROC productions have to rely on public internet or LTE for signal transmission. 5G networks will not only provide more bandwidth and lower latency, but an unprecedented degree of flexibility, making things possible that we didn’t dare dream about a couple of years back!”

The adoption of standards-based IP, as a more flexible and scalable replacement for SDI, alongside the greater use of software, is key to reducing costs and enabling more automation in live productions. 

“Grass Valley is unique in delivering our IP Dynamic Blueprint, which includes purpose-built, on-premise IP-connected devices managed and controlled by the GV-Orbit orchestration layer,” says Marco Lopez, the company’s General Manager for Live Production. “The LDX 100 native triple-speed UHD and IP camera platform and the IP-enabled K-Frame XP video switcher for full raster UHD-4K processing are examples of this IP Blueprint offering.”

In addition to providing the tools to stream content over standard IP infrastructures, Grass Valley enables production or even work from home operations with its cloud SaaS platform, GV AMPP (Agile Media Processing Platform). Lopez says it allows operators to build and maintain services, scale applications and capacity up or down as required, and only pay for what they use.  Moreover, they can do so from anywhere worldwide, dramatically reducing the amount of operational personnel on location.

Launched at the beginning of the Covid-19 outbreak as a way to highlight the best practices in the industry, EVS’ Live Production Anywhere initiative brings together a series of recommendations designed to transform existing production systems into more flexible production environments that are better adapted to meet the demands of today and tomorrow.

The company also surveyed customers in the latter half of the year which revealed the top three issues facing production crews. These are: “managing increasingly complex and ambitious workflows,” “delivering more value with less spend on equipment,” and “improving productivity.” 

“Interestingly, customers no longer appear to be worried about working with remote team members, proving the resilience of the industry in adapting to this new landscape and also suggesting that remote working isn’t as daunting as many feared,” it says.

Production engineers, in particular, find themselves under additional pressure to adapt their workflows and processes and rethink the way they deliver live events. EVS has moved to provide a range of programs designed to support production engineers and recognize the vital role they play. For instance, over 1200 professionals signed up to the EVS Cerebrum GO online training course.

“As we navigate this pandemic, production engineers need a simple way of working remotely, without sacrificing their ability to easily control and monitor all of their devices from different vendors,” says Nestor Amaya, VP Solutions Architect. “Many have had to perform heroics throughout 2020, and as we look ahead to next year, it’s vital that they are supported with a control system that enables them to perform without the need for acrobatics in this new normal.”

Stay-at-home orders also mean that many staff are not in the studio or onsite in an OB truck, which is a catalyst for virtual operations. Back-office workflows have migrated to off-site locations, but broadcast operations have depended on equipment being on-site for 100 years.

“Forward-thinking broadcasters are virtualising at least some of their infrastructure, but is this really a choice?” quizzes John Schur, President, Solutions Group, Telos Alliance. 

“Virtualisation is not only a production reality for 2021, but a requirement.”

He makes the point that virtualisation doesn’t necessarily mean cloud. Virtual instances of real time products can often run on COTs servers; either on-premise or in the cloud, simplifying issues such as redundancy, replacement units and spares, wiring and scaling up additional workflows and systems for special events as needed.

“From production to editing to streaming delivery, virtualisation enables sports broadcasters to create ‘studios’ anywhere, utilising data centres and other cloud-based tools to put on events,” he says. “Comms, monitoring, mix control, encoding, transcoding and more, for remote production workflows are all possible and often have the benefit of SaaS business models to boot. SaaS opex business models allow the scalability and flexibility of on-the-fly compute resource utilisation to be paid for as needed.”