WO2011087356A2

WO2011087356A2 - Video conferencing using single panoramic camera

Info

Publication number: WO2011087356A2
Application number: PCT/MY2010/000235
Authority: WO
Inventors: Hock Woon Hon
Original assignee: Mimos Berhad
Priority date: 2010-01-15
Filing date: 2010-10-29
Publication date: 2011-07-21
Also published as: WO2011087356A3

Abstract

A video teleconferencing system is provided herewith, wherein the video teleconferencing system is able to transmit data in three different modes, based on the users preference.

Description

Video Conferencing Using Single Panoramic Camera

Field of Invention This invention relates to the field of video teleconferencing. Background of Invention

In a conventional (audio-only) teleconferencing system, one of the major drawback is the difficulty in determining which one of the participant present on the other side is speaking or interjecting words. Voices of the participants are only identifiable based on the quality of their sounds such as pitch and accent. In the event that there is a silent party present on the other side, his or her presence can go undetected altogether. While, some existing video teleconferencing systems may provide the visual viewing of participant on the other side, these systems are not advantageous when applied in a conference room setting. This is because the video conferencing systems are usually set up with all the participants facing a common visual display apparatus such as projector screen. This creates a scene of a 'theater' rather than a conference. Other alternatives are where each participant is seated in front of his or her individual visual display such as a laptop screen, which is equipped with a camera to capture the image of the person.

However, this method requires multiple screens to be available and in the event of a large number of participants, this method may not be practical. Therefore, there arises a need for a video teleconferencing method which will enable users present and in a remote location to view all present participant, identify the active speaker and also utilize only one image capturing apparatus.

Summary of Invention It is an objective of the invention to provide for a videoconferencing system that can detect the active speaker and transmit the data with variable frame rate, so that the image of the active speaker is transmitted with higher frame rate whereas the other participants are transmitted at a slower frame rate.

It is also an objective of the present invention to provide for a video teleconferencing system, which allows the participants in one location to view the participants in a different location or locations.

It another objective of the present invention to provide for a video teleconferencing system, which allows only the active speaker to be viewed by the participants in another location.

It is also an objective of the present invention to provide for a video teleconferencing system, which allows participants to view other participants in a remote location in the absence of general light source such as room lights, during presentations

It is also an objective of the present invention to provide for a video teleconferencing system, which provides an environment where all the participants are able to have an eye- to eye contact with participants in one or more remote location(s).

Description of Drawings

Figure 1 Overview of video teleconferencing system

Figure 2a An embodiment of invention

Figure 2b An embodiment of invention

Figure 3 Detection of highest amplitude signal

Figure 4 Tilt adjustment of mirrors

Figure 5 Rotational adjustment of camera

Figure 6 Video teleconferencing audiovisual processing device Detailed Description

The present invention will now be detailed with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. The invention may, however be embodied in other different forms and should not be construed as being limited to the embodiments discussed herein, rather these embodiments are provided so that this disclosure will convey the concept of the invention to those skilled in the art. In the following description of the present invention, known methods and functions will be omitted as it would be already known to those skilled in the art.

The present invention involves a video teleconferencing apparatus and a video

teleconferencing system. The video teleconferencing apparatus captures the local panoramic scene and audio signal of the video teleconference and feeds the images and audio to the video teleconferencing system. The system functions to transmit the audio and video signals from one location to a remote video teleconference station; and also to transmit the audio and video signals from a remote location to a video teleconference station. An overview of the video teleconferencing system is as illustrated in Figure 1.

The video teleconferencing device of the present invention is described in brief below and is described in further detail in co-owned and co-pending Malaysian application no. PI (to be furnished once made available). The video teleconferencing apparatus comprises of a camera, a wide-angle lens, a detachable reflective means, a detachable illumination means and a detachable sound detection means. The video teleconferencing apparatus is connected to at least one visual display apparatus for displaying visuals received from one or more video teleconferencing apparatus in one or more remote locations (not shown) during a video teleconferencing session.

The video signal received from one or more teleconferencing apparatus in the system will be sent across the network and therefore the IP video stream can be accessed by more than one. teleconferencing system that is connected to the network. Furthermore, at the receiving end, multiple duplicated panoramic display monitor can be installed in multiple location.

The video teleconferencing apparatus can be operated in three modes i.e. a. Standard mode

b. Bandwidth Saving Mode

c. Active speaker mode a. Standard mode

Standard mode provides full view of the room with all parts of the video encoded at the same rate b. Bandwidth saving mode

Bandwidth saving mode combines the fast frame rate and slow frame rate into a single video. The active speaker window is encoded with higher frame rate and the other part of the scenes are encoded at slower rate c. Active speaker mode

Active speaker mode provides view of the active speaker only.

An overview of the video teleconferencing system is as illustrated in Figure 6.

The audio visual processing device of the video teleconferencing system as claimed in the present invention will now be discussed in details referring to Figure 6. As can be seen from the figure, the device is divided into 6 blocks. Block A

Block A represents the architecture of the audio system of the video teleconferencing system. The audio inputs collected by the microphone array are multiplexed using a microphone multiplexer, Al . The output of Al is a time multiplexed serial output, where each of the microphones will be allocated a slot in time.

The output of Al is fed into the video teleconferencing system. The receiving end of the black box is the Audio Signal Receiver (A2). Audio signals fed to the Audio Signal Receiver (A2) are then fed into the Max Amplitude (A3) and Audio Encoder (A5).

Max Amplitude (A3) detector compares the serial audio signal it receives to detect the audio signal with the highest amplitude. When the highest amplitude has been determined, the direction of the sound source is calculated.

Audio Encoder (A5) encodes the audio signals to be fed to the (a) Audio Compression (Dl) for recording purposes or to (b) the transmitters (D3 and D4) for transmission to the audiovisual display. Block B.

The Video signal received by the camera is fed to the Panoramic Image Transformation (Bl). Images acquired from a wide-angle lens are hemispherical and would be distorted when viewed in a video teleconferencing. Therefore, the Panoramic Image

Transformation section transforms the original video to a panoramic image into a rectangular representation, by means of a pre-determined computer program. The output of the Panoramic Image Transformation is fed into three components i.e. In Bounding Box View (B2), Out Bounding Box View (B3) and to Video Encoder (C3). The video signal received by Video Encoder (C3) is the full panoramic image. In Bounding Box View (B2) receives input from Panoramic Image Transformation (Bl) and also Bounding Box Location Reference (A4). The output of In Bounding Box View (B2) is the video signal detected at the speaker winder. This video signal is then fed to Fast Frame Video Encoder (B4). Fast Frame Video Encoder is responsible for encoding the panoramic frames. The frames are encoded at high speed such as 25 frames per second whereas, for Out Bounding Box View, the frames are encoded at a slower rate such as 5 frames per second since the content of the out bounding image is static or not the focus scene.

Out Bounding Box View (B3) receives input from Panoramic Image Transformation (Bl) and also Bounding Box Information. The output of Out Bounding Box View (B3) is fed to Slow Frame Vide Encoder (B5). The outputs of B4 and B5 are fed to Video Combiner (C2), where the video signals are combined. The combined signal is the Bandwidth Saving Mode signal where the speaker window has higher frame rate and the other scene will have slower rate.

Block C

Block comprises of video encoders.

In CI the signal from In Bounding Box View (B2) encoded. Therefore, the video signal for standard mode is output from CI.

In C2, the output signal from B4 and B5 are combined and the speaker window signal is encoded with higher rate and other scenes are encoded with slower rate. The output of C2 is the video signal for bandwidth saving mode.

In C3 the full panoramic signal from Bl is encoded at standard rate. The output of C3 is the video signal of the speaker mode. Block D

Audio Compression (Dl) receives its audio input from the Audio Encoder (A5). Video Compression (D2) receives its video input from CI, C2 or C3 depending on the mode that has been selected i.e. standard, bandwidth saving or speaker. The video and audio signals are then compressed and sent to the Recording medium

Wireless transmitter (D3) also receives its audio input from Audio Encoder (A5) and its video input from either C 1, C2 or C3 depending the mode that has been selected i.e. standard, bandwidth saving or speaker. The output is transmitted to the local scene via a wireless transmitter.

Wired transmitter (D4) has the same function as D3, however D4 receives its audio input from Audio Encoder (A5) and its video input from either CI, C2 or C3 depending the mode that has been selected i.e. standard, bandwidth saving or speaker, and transmits the signal via a wired transmission.

Block F

The F block consists of interfaces that connect the present video teleconferencing system to the outside world.

Fl is the connector that can retrieve data stored in the storage medium. It can be any of the modern data transfer technology such as USB or fire wire. The data from the output of Fl is stored in the local medium and can be retrieved.

F2 is the transmitter to transit wireless information via the means of RF, WiFi, WiMax or any other modern wireless technology. F3 is the port for wired communication where the data is communicated via LAN or any other similar technology.

The system also includes switches (SW) which are used as connectors and also switches, which allows the user to choose between the three modes i.e. standard, bandwidth saving or speaker mode.

Claims

A video teleconferencing system comprises

at least one video teleconferencing apparatus that captures the audio and video signal from one location;

and a system architecture for receiving audio and video signal from the apparatus and transmitting the signals to at least one audio visual apparatus in a remote location.

A system according to claim 1, wherein the system architecture receives and transmits video and audio signals characterized by the steps of

(a) Audio signals

(i.) audio signals received from the apparatus is fed to a microphone multiplexer,

(ii.) the multiplexed signal is fed into a audio signal

receiver,

(iii.) the audio signal receiver feeds the signals it has

received to the audio encoder and to Max amplitude detector

(iv.) the Max amplitude detector determines the microphone with the highest amplitude

(v.) the information computed by Max amplitude detector is fed into the bounding box determining, where the information received from the Max amplitude detector is used to determine the exact location of the sound source of the highest amplitude

(vi.) the output of Bounding Box Location Determining is fed into the In Bounding View and Out Bounding View

(b) Video signal I

(i.) the video signal received from the apparatus is fed into the panoramic image transformation, wherein the wide angle image is converted to a rectangular representation (ii.) the output signal from the panoramic image

transformation is fed into the In Bounding Box View and Out Bounding View

(iii.) In the In Bounding Box View the output of Bounding Box Location Determining is combined with the output of

Panoramic Image Transformation, and the result is fed into a video encoder and Fast Frame Video Encoder

(iv.) in the Out Bounding Box view the output of Bounding Box Location Determining is combined with the output of Panoramic Image Transformation, and the result if fed to a video encoder and the Slow Frame video Encoder

(c) Video encoder

(i.) The output of step (b)(iii) is fed into a first video encoder

(CI), wherein the data is encoded

(ii.) The output of steps (b)(iii) and (b)(iv) are fed to a video combiner (C2)to be combined

(iii.) The output of steps (b)(i) is fed into a second video

encoder (C3), wherein the data is encoded.

(d) Audio Compression (Dl) receives audio data from Audio Encoder (A5) and the data is compressed

(e) Video Compression

(D2) receives data from outputs of steps (c)(i), (c)(ii) and (c)(iii) and compresses the video data

(f) The audio and video signals from steps (d) and (e) are fed into

Recording Medium (El) for data storage\

(g) The audio signals from Audio Encoder (A5) and video signals from steps (c)(i), (c)(ii) and (c)(iii) are fed into Wireless transmitter (D3) for wireless transmission

(h) The audio signals from Audio Encoder (A5) and video signals from steps (c)(i), (c)(ii) and (c)(iii) are fed into Wired transmitter (D4) for wired transmission

3. A system according to claim 1, wherein the audio and video data transmitted to a audio visual display apparatus is selected from standard mode, bandwidth saving mode or speaker mode

4. A system according to claim 1, wherein the video data for standard mode is the output from the first video encoder (CI )

5. A system according to claim 1, wherein the video data for the bandwidth saving mode is the output from the video combiner (C2)

6. A system according to claim 1, wherein the video data for the speaker mode is the output from the second video encoder (C3)

7. A system according to claim 1, wherein the audio data for the standard mode, bandwidth saving mode or speaker mode is the output from audio encoder (A5)

8. A system according to claim 1, wherein the audio signal and video signal are fed into Wireless Transmitter