Virtual Audience Concepts

A live audience is a key part of the atmosphere of any entertainment show, but the COVID-19 pandemic and new social distancing rules mean it will be quite some time before we see a studio audience again.  The TV and Entertainment business will bounce back, and very quickly the most creative minds in the industry have turned their focus on finding ways to keep productions running while keeping talent, crews and contributors safe.  The most difficult challenge is bringing in a studio audience – hundreds of people, from across the country, gathered together for a few hours in very close quarters.   Currently the risks and logistics involved are insurmountable.

Since 2013, Kinetic Pixel have been leaders in the area of video calling and remote contributions, with ground-breaking projects such as Embarrassing Bodies and Last Commanders demonstrating the breadth of experience and high standards we can achieve with a small budget and the right expertise.  Coupling this with our world-class 3D graphics, automation and interaction systems, we’ve been able to apply our experience to finding low-cost ways of bringing in large numbers of contributors into a show using consumer video calling tools that many millions of people are now using on a daily basis.

The video you can see above is the first result of this experimentation.  It’s a little rough around the edges, but we think it certainly demonstrates some concepts that haven’t been seen before.  Taking a live zoom call and feeding into both our graphics engine, and a set of computer vision and machine learning algorithms we’re developing allows us to manipulate the standard ‘grid’ layout of a group video call in ways that have never been seen before.

This demo shows Zoom’s virtual background feature repurposed to provide a primitive chroma-key, allowing us to mask out each caller rather than constraining them to their little rectangle.  This is a stylistic decision – sometimes it’s better to embrace the medium, and keep those frames so the audience at home can still relate to what they are seeing. Our systems are flexible enough to allow for these, and many other, options.

Our algorithms determine the optimum framing for each caller.  Individual level and colour correct could easily be added to allow some grading of the disparate video sources from varying qualities of webcam, devices and lighting conditions.  Facial recognition can ‘fingerprint’ each caller, ensuring that should they drop out of the call and come back in later, we can still map them to the same place in the audience.  Further machine learning techniques can detect if there’s even a person in the frame, and if not, automatically exclude them till they return.

Once this processing has been done, the callers can be manipulated individually; built into a representation of audience seating;  Mapped into groups or teams;  Or simply arranged in something more interesting and creative than a grid of faces.  Typically most video calling software such as Zoom or Skype have incredibly low latency, so sending Program Audio back to the callers will ensure you get quick reactions to the key moments and funniest gags in your show.

This video was generated in real-time, using a live Zoom call and Ventuz 6 3D graphics engine. We’re only showing a small number of callers because we are a small team.  The solution we’re demonstrating can scale to hundreds, and it is possible to do so without compromising on the visual fidelity of each caller.

Contact us at for more information on how we can adapt this demo for your production.

 
 

Like us on Facebook

Find Us

Address

Kinetic Pixel Limited

Unit C Caerphilly Business Park

Caerphilly

CF83 3ED

Email: info@kineticpixel.tv

Phone : +44(0) 2920 098 920