Real-time Transmission of High Definition (HD) 3D
Video and HD Audio in Gigabit-LAN
Principle Investigator: Professor Raymond Yeung, IE Department
Co-Investigators: Professor Chinlon Lin, Mr. Alan Lam (CO I),
and
Mr. Ka Kui Choy (ACO), IE Department
Vision
With the advances in electronic appliances and gigabit networks, we believe that real-time transmission of HD 3D (stereoscopic) video will be of high demand and will prevail in the next generation Internet, especially in the areas of medicine, product design, and digital entertainment. However, with the current multimedia technologies, we are still facing lots of challenges in HD 3D video real-time transmission. This project will focus on the difficulties of the deployment of HD 3D video real-time transmission and try to work out state-of-the-art solution to overcome these difficulties.
Why HD, 3D, and Real-time Transmission
As digital TV will be available as early as 2007/08, more and more HD products are released in the market. It is projected that HDTV telecasting will soon become a demand in HK, in a way very much like color TV replacing black-and-white TV in the 1970s. The real-time HD transmission technology will become a need within few years.
Under the trend of globalization, collaboration among different regions will be commonplace. Sharing 3D images and videos among partners at different remote sites for such purposes as architecture review, manufacture design, or remote medical diagnosis, is desired. Furthermore, 3D display will also be a key component in future digital entertainment.
The
medical field has high demand on 3D HD video transmission for remote surgery or
diagnosis. However, due to the bandwidth and equipment limitation, it can
currently be done only in the 2D SD format. Robotic surgical systems are also
becoming popular because of its precision and minimum invasion. To operate such
a system, the surgeon needs to have a clear and 3D view. And the 3D video
transmission needs to be in real-time because a split second delay during a
blood vessel cut can be fatal in any operation. Real-time 3D video transmission
also plays an important role in future network game industry.
Existing Problems
1. High bandwidth of HD video
For 8bits 4:2:2 1080i HD Video, the video bandwidth for uncompressed video is around 1.69Gbps. Although gigabit connection is widely available today, it still cannot fulfill the requirement for transmitting 2D HD Video
2. No solution for HD video real-time transmission
Data compression by MPEG-2 TS or H.264 can help to solve the HD video high bandwidth problem. However, they need high-end server support (i.e., very expensive hardware card, high processing power CPU, and large amount of RAM) which is not available in conventional PCs or electronic appliances. In addition, these video coding formats hardly address the problem of real-time transmission.
3. Lack of effective real-time transmission protocol for Audio Engineering Society (AES) or multi-channel audio
Most AES are for streaming only. So far, there is no real-time transmission protocol for any AES or multi-channel audio.
4. Lack of video CODEC for HD 3D video
There exists no video CODEC for 3D video or real-time HD video transmission. Without an effective video CODEC for HD 3D video, real-time HD 3D video transmission will be extremely expensive.
5. Auto-stereoscopic display on heterogeneous devices
More and more electronic appliances can now display Auto-Stereoscopic 3D images, such as 3D LCD monitor or cellular phone. However, there is no effective algorithm or standard to display a single source 3D image on these heterogeneous devices.
Objectives
In this project, new standards and protocols will be developed to address the problems in real-time transmission of HD 3D video.
a. Develop a new CODEC for capturing and trans-coding the 3D HD Cameras.
b. Develop a new video format standard that accommodates 3D video and HD audio.
c. Develop a new compression algorithm that strikes a balance between compression rate and coding complicity.
d. Develop a new transmission protocol for 3D HD video.
e. Develop a new 3D display algorithm for Auto-Stereoscopic display on heterogeneous devices.
Status and Progress
Prototype
The following prototype has been built to demonstrate the
feasibility of real-time 3D HD video transmission on computer network.
Transmitting
real-time 3D HD Video on different platforms
Now developing new technology to transmit real-time 3D HD Video on different platforms via Gigabit computer network.
Collaboration
with Department of Surgery, CUHK
3D video of minimally invasive surgical operation has been captured from DaVinci Robotic Surgical System and has been successfully telecasted on computer network in real-time.
DaVinci
Robotic Surgical System Real-time
3D video transmission
This project has supported the International Robotic Surgery Symposium 2007 organized by CUHK Surgery Department. Surgical operation 3D video was telecasted in real-time on computer network during live demonstration sessions. The real-time 3D video transmission from Robotic Surgical System “Da Vinci” in this Symposium is a showcase to demonstrate such an application of real-time HD 3D video transmission and distribution for next-generation telemedicine for biomedical research, training, and education as well as health-care. With further advances in high-speed broadband optical access networks worldwide, such HD 3D video transmission and distribution over very long distances may make this an even more useful system for tele-medicine across the continents.
This project has also helped HK Trade Development Council to demo 3D Minimally Invasive Surgery Video in Medical and HealthCare Fair 2007. We have also given a seminar and demo of “3D HD Video Distribution for Robotic Surgery” in a HKIE and SPIE activity.
Conducting
3D Video Conference with
3D video conferences have been conducted with
