Reality in All Directions

With the popularity of virtual reality surging, the potential uses of the technology widen as more applications are conceived to capitalize on its immersive nature.

But like many emerging digital innovations, one major roadblock stands in the way of its widespread adoption: the sheer amount of data that must be processed for an effective user experience.

The data generated by 360° video is a case in point. It often exceeds the bandwidth available for rapid transmission, leading to the all-too-familiar choppy output or the interruptions caused by buffering. To compensate, more sophisticated hardware can be purchased, or more precious bandwidth can be allocated — neither of which is a satisfactory solution for the host of the video, or for the consumer.

UC Santa Barbara electrical and computer engineering professor Kenneth Rose and his research group are poised to make things smoother. With a $40,000 gift from the company InterDigital Communications, through its charitable affiliate the Signal Foundation for Wireless Innovation, Inc., Rose and his team will investigate methods of compression to decrease both the volume of data and the time needed to process it.

“Our objective is to develop enabling technology in terms of compression, since the amount of data involved in 360 video is extensive, which as it stands would make many application scenarios impractical,” Rose said. In contrast to conventional video technology, which is projected on a flat plane, 360° video is projected on a spherical plane that encloses the viewer. And that presents unique challenges.

“Usually, video compression — as in MPEG standards — leans heavily on prediction, essentially predicting the next frame from the previous frame, based on estimating the motion that occurs between frames,” Rose said. “This is straightforward in standard video coding by assuming that motion consists locally of simple translations.” In standard video compression, the receiver also sends small corrections to the video prediction, resulting in relatively little data in transit.

“In 360 the ‘frame’ is effectively the image on an entire sphere and we need to estimate how everything moved from an available reconstructed frame or sphere to the frame in the next time instant,” Rose continued. “Proper modeling of the motion on the sphere would lead to accurate prediction and would hopefully yield considerable reduction of the data needed to provide high-quality reconstructed 360 video.”

In doing so, the aim is that 360° video becomes more accessible for more applications, and more users, without the need for extremely high bandwidth connections or storage requirements. Aside from the rapid adoption by social media and the gaming community for entertainment, the immersive video can be used in virtual reality applications geared toward business, therapy, education and even tourism.

“Improving compression efficiency is fundamental to cost reduction in all parts of the virtual eco-systems,” said InterDigital video research manager Yan Ye, director at InterDigital, who leads the company’s video technology development efforts. “Storage reduction at the host site and on the device reduces cost, bandwidth reduction during transmission improves speed, even processing power can be potentially reduced, if the technology has reasonable complexity.”