WO2010130084A1 - 远程呈现系统、方法及视频采集设备 - Google Patents

远程呈现系统、方法及视频采集设备 Download PDF

Info

Publication number
WO2010130084A1
WO2010130084A1 PCT/CN2009/071745 CN2009071745W WO2010130084A1 WO 2010130084 A1 WO2010130084 A1 WO 2010130084A1 CN 2009071745 W CN2009071745 W CN 2009071745W WO 2010130084 A1 WO2010130084 A1 WO 2010130084A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
projection screen
camera
video
panoramic
Prior art date
Application number
PCT/CN2009/071745
Other languages
English (en)
French (fr)
Inventor
刘源
王静
赵光耀
Original Assignee
华为终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为终端有限公司 filed Critical 华为终端有限公司
Priority to EP09842025A priority Critical patent/EP2290969A4/en
Priority to PCT/CN2009/071745 priority patent/WO2010130084A1/zh
Priority to RU2011130860/07A priority patent/RU2518218C2/ru
Priority to BRPI0924076-4A priority patent/BRPI0924076B1/pt
Priority to CN2009801168492A priority patent/CN102084650B/zh
Priority to US12/888,769 priority patent/US8692861B2/en
Publication of WO2010130084A1 publication Critical patent/WO2010130084A1/zh
Priority to US13/716,508 priority patent/US8836750B2/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • H04N7/144—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Definitions

  • Telepresence system method and video capture device
  • the present invention relates to the field of video conferencing technologies, and in particular, to a telepresence system, method, and video capture device.
  • Telepresence is a video conferencing system that implements a virtual meeting environment.
  • the goal of the virtual meeting environment is to fully reflect the human factors of the participants and to replicate the real experience of the participants as much as possible, which can greatly improve the acceptance of the end users, thereby improving equipment utilization, return on investment and customer satisfaction.
  • the ideal Telepresence system has many advantages over traditional video conferencing systems, including: life-size images; smooth motion; precise limb behavior; high-definition, studio-level video, lighting and audio; The user group's approximate eye-catching communication; the immersive meeting environment makes the participants feel at the same meeting place; the consistency of different meeting places; hiding the camera, reducing the impact on the user.
  • the inventor of the present application found in the research process that the current Telepresence system has a problem that the panoramic rendering effect is not good.
  • Most of the current Telepresence systems use multiple cameras and multiple large-screen displays to capture/display. Each camera/display captures/displays one or more local/remote, such as Telepresence from companies such as Cisco and HP. system. Since the current Telepresence system uses multiple cameras for shooting, the images of the multiple cameras have parallax, so it is impossible to splicing multiple images into a panorama through the position of the camera.
  • the inventor of the present application further finds in the research process that the current Telepresence system needs to be improved at least in the following aspects:
  • Seeing God Communication is a very important non-verbal communication method. Eye communication physiologically causes changes in heartbeat, blood pressure, and increases brain activity. Gaze perception provides a basis for many communication, including feedback, dialogue, and emotional expression, and is a key way to perceive each other's thoughts.
  • the traditional video conferencing system and the current Telepresence system can't make users have a good eye contact.
  • the main reason is due to the parallax problem: intuitively, the user will stare at the other side of the screen instead of staring at the camera, and the camera often They are not placed in the center of the screen, which causes a parallax between the camera and the user's front view, which makes it impossible to achieve good eye contact.
  • the embodiment of the present invention provides an improved Telepresence system, method, and video capture device, so as to solve the problem that the panoramic display effect of the current Telepresence system is not good. Further, the embodiment of the present invention provides depth presentation, seamless display, and eye contact. Improvements have been made.
  • the solution provided by the present invention includes:
  • a telepresence system includes a video capture device, a video display device, an audio capture device, an audio playback device, and an audio and video communication device, and the audio and video communication device passes the image collected by the local video capture device and the audio collected by the audio capture device.
  • the network transmits to the remote end, and the video display device and the audio playback device respectively display and play the image and audio, and the video capture device is a panoramic camera; the system further includes a splicing unit, and the splicing unit is used for The low-resolution images of different viewing angles captured by the panoramic camera are spliced into a high-resolution panoramic image.
  • a remote presentation method comprising: acquiring a local panoramic image and audio, wherein: capturing images of different viewing angles by the panoramic camera, and splicing the low-resolution images of different viewing angles captured by the panoramic camera by using a splicing unit High-resolution panoramic image; local panoramic image and audio are transmitted to the remote end for display and playback through the network.
  • a video capture device in a remote presentation system the remote presentation system further comprising a video display device, an audio capture device, an audio playback device, and an audio and video communication device, the audio and video communication device
  • the image captured by the local video capture device and the audio collected by the audio capture device are transmitted to the remote end through the network, and the video display device and the audio playback device respectively display and play the image and audio, and the video capture device is
  • the panoramic camera, the low-resolution images of different viewing angles captured by the panoramic camera are spliced into a high-resolution panoramic image by a splicing unit.
  • the embodiment of the present invention is upgraded on the existing Telepresence system, and only the ordinary camera is replaced by the panoramic camera, and the panoramic view of the local conference room can be captured, and the conference panorama is provided for the opposite participants, thereby solving the current Telepresence system.
  • the ordinary projection screen or the holographic transparent projection screen is used for seamlessly presenting the image, thereby overcoming the defects of the combination of the plurality of flat-panel televisions and achieving seamless presentation.
  • the sensation of depth presentation can be provided to the participants.
  • the panoramic camera is not affected by the projected image of the projector when the local image is captured, thereby avoiding the parallax caused by the camera being unable to be placed on the line of sight of the user, thereby enabling the opposite end parameter
  • eye contact can be achieved using a half mirror or a light conducting element or a linear polarizer.
  • a dark background is arranged behind the user to separate the user image and the background image, thereby realizing a rendering effect with a sense of depth.
  • Embodiment 1 is a top view of a conference room layout of Embodiment 1 of the Telepresence system of the present invention
  • FIG. 2 is a schematic diagram 1 of Embodiment 1 of the Telepresence system of the present invention.
  • FIG. 3 is a schematic diagram of a holographic projection of the embodiment 1 of the Telepresence system of the present invention.
  • FIG. 4 is a schematic diagram of a panoramic camera of the embodiment 1 of the Telepresence system of the present invention.
  • FIG. 5 is a schematic diagram of a multi-reflection panoramic camera of the embodiment of the Telepresence system of the present invention
  • FIG. 6 is a schematic diagram of Embodiment 1 of the Telepresence system of the present invention
  • Figure 7 is a top view of a conference room layout of the embodiment 1 of the Telepresence system of the present invention.
  • Figure 8 is a schematic diagram 2 of Embodiment 1 of the Telepresence system of the present invention.
  • Figure 9 is a top view of the layout of the conference room of the embodiment 1 of the Telepresence system of the present invention.
  • Figure 10 is a schematic diagram 3 of Embodiment 1 of the Telepresence system of the present invention.
  • FIG. 11 is a plan view of a conference room layout of Embodiment 2 of the Telepresence system of the present invention
  • FIG. 12 is a schematic diagram 1 of Embodiment 2 of the Telepresence system of the present invention
  • Figure 13 is a schematic diagram 2 of the embodiment 2 of the Telepresence system of the present invention.
  • FIG. 14 is a top view of a conference room layout of the embodiment of the Telepresence system of the present invention
  • FIG. 15 is a schematic diagram of Embodiment 3 of the Telepresence system of the present invention
  • FIG. 16 is a schematic diagram of Embodiment 4 of a Telepresence system according to the present invention.
  • FIG. 17 is a schematic diagram of Embodiment 5 of a Telepresence system according to the present invention.
  • FIG. 18 is a schematic diagram of Embodiment 6 of a Telepresence system according to the present invention.
  • Figure 19 is a schematic diagram 1 of Embodiment 7 of the Telepresence system of the present invention.
  • Figure 20 is a schematic diagram 2 of Embodiment 7 of the Telepresence system of the present invention.
  • Figure 21 is a schematic diagram 3 of Embodiment 7 of the Telepresence system of the present invention.
  • Figure 22 is a schematic diagram of Embodiment 7 of the Telepresence system of the present invention.
  • Figure 23 is a flow chart of the Telepresence method of the present invention.
  • FIG. 1 a top view of a conference room layout of a preferred embodiment 1 of the Telepresence system of the present invention is shown.
  • the front and rear walls 13, 14 of the venue are designed with a curved surface or a plane, and a conference table 2 is arranged in the middle.
  • a microphone array 3 is installed on the conference table 2.
  • the microphone array 3 can be placed in the middle of the conference table 2, and a plurality of participants are placed on one side of the conference table 2.
  • the participant seat faces a projection screen 4, which is a curved projection screen or a flat projection screen (in the figure, an arc-shaped projection screen is drawn), which constitutes a front surface of a black box 5.
  • a panoramic camera 7 is installed in the black box 5 (the panoramic camera 7 here includes three cameras, see FIG. 4 and its description), the audio and video communication device 8, the plurality of projectors 9A, 9B, 9C and a plurality of speakers.
  • a speaker array composed of five speakers 11 A, 11 B , 11 C , 11 D , 11 E shown in the drawing.
  • the inner wall 6 of the dark box 5 opposite to the projection screen 4 is a specially decorated backdrop, and the so-called special decoration refers to a decoration having a depth suggesting effect and capable of hiding the camera.
  • the back of the participant's seat is also a specially decorated dark background wall curtain 13.
  • An auxiliary display 12 is placed on one side of the conference room.
  • One or more object cameras 10A, 10B are placed above the black box 5 to form one or more stereo camera pairs, wherein one stereo camera pair is composed of two common object cameras, simulating the human visual principle, from left to right
  • the scene is synchronized to shoot the scene, and the left and right images are obtained.
  • FIG. 2 there is shown a schematic view 1 of Embodiment 1 of the present invention.
  • the local and remote Telepresence sites have the same configuration and are interconnected through the network.
  • the local projection screen A4 and the far-end projection screen B4 employ a Holographic transparent projection screen.
  • the holographic transparent projection screen is a rear projection screen using holographic technology. It has the characteristics of a holographic image, and only displays images from a certain angle of the back projection, while ignoring other angles of light.
  • the holographic screen has a very bright, clear display, even in areas where the ambient light is bright, and also has a transparent feature that allows the viewer to see through the objects behind the screen.
  • holographic transparent projection screens such as Woehburk, HoloPro,
  • Figure 3 shows the basic principle of holographic projection.
  • the incident light incident at an angle is scattered by a holographic transparent projection screen composed of a holographic material and a transparent panel, so that the user can see the projected content without seeing the object behind the projected content area, and the projection screen other than the projected content.
  • the horizontal light emitted by some of the back objects is transmitted by the projection screen, so that the user can see the objects behind the projection screen through the portion of the projection screen.
  • the projection angle and the horizontal line of the projector A9 in the local black box A5 have a mounting angle, and the projector A9 projects the image of the far-end participant B15 onto the local holographic transparent screen A4. Since the resolution of the remote scene B is presented, the resolution of one projector may not meet the requirements, so the audio and video communication device A8 divides the image of the far end into multiple copies and presents it through multiple projectors A9 (due to In Fig. 2, the right side view is taken as a schematic view thereof, so that multiple projectors overlap, but not all. In order to hide the projector, the color of the projector A9 is best integrated with the back black box background A6.
  • a panoramic camera A7 is installed in the black box A5. See picture
  • FIG. 4 (A) and Figure 4 (B) show the specific principles of an implementation of panoramic camera imaging.
  • the panoramic camera is realized by the principle that the plane mirror reflects the virtual common optical center (virtual common mode panoramic camera).
  • the prism structure 1 001 has three reflecting surfaces 1002, 1003, 1004. These surfaces are flat mirrors, and three cameras C01, C02 and C03 are placed under the mirror surface. Take one of the videos below
  • the machine C02 is taken as an example to illustrate the principle of virtual common light. Referring to FIG. 4(B), L02 is incident light, R02 is reflected light, the normal line perpendicular to the reflective surface 1003 is 1006, and the angle between the normal line 1006 and the horizontal line 1010 is ⁇ the reflection point to the actual optical center 02 of the camera C02.
  • the vertical distance is d.
  • the camera will capture a virtual image with a virtual optical heart ⁇ 0.
  • the virtual optical centers of cameras C01, C02 and C03 can be located exactly at the same point, thus obtaining three images of the common optical center. By stitching these three images, images that are seamlessly stitched at any depth can be obtained.
  • the lower the optical center position of the camera the better.
  • the geometric height of the camera is constant, this can be achieved by reducing the horizontal distance between the camera and the mirror surface, but this distance is limited by the size of the camera lens and mirror, which will reduce the viewing angle, as shown in Figure 4 (C). .
  • Another implementation of the panoramic camera is to use a converged multi-camera model to obtain multiple images by capturing multiple images and then digital image stitching to obtain a panoramic image (aggregation mode panoramic camera). As shown in Figure 4 (D). Because the optical center is inside the camera, this kind of camera model can't achieve common optical center only by positional placement. There is parallax in the overlapping area of the output image, and it must rely on image processing technology to achieve better stitching effect.
  • FIG. 4(E) Another way to make a panoramic camera is to use a multi-camera array (multi-camera array mode panoramic camera), as shown in Figure 4(E).
  • Camera arrays can take different forms depending on different scenarios, such as linear arrays, ring arrays or matrix arrays. Each camera in the camera array uses a lower resolution camera with a smaller separation between adjacent cameras and a larger overlap of shots.
  • Image stitching processing technology can be used to stitch multiple lower resolution camera images into a high resolution panoramic image.
  • the basic principle of the image stitching algorithm is to estimate the internal parameters of multiple cameras (such as focal length, principal point and distortion, etc.) and the positional parameters between the cameras (such as the rotation matrix, translation vector), and the algorithm will be used by using the estimated parameters. Camera image alignment, elimination of overlapping areas, edge blending, parallax removal, etc., to obtain a high-resolution panoramic image.
  • the basic principle of the image stitching algorithm is to estimate the internal parameters of multiple cameras and the positional parameters between them.
  • the estimated parameters are used to align multiple camera images by algorithm, eliminate overlapping regions, and perform edge blending and parallax removal.
  • a high resolution panoramic image According to the principle of projective geometry, the transformation relationship of 3D point projections in space onto the imaging plane of the camera is:
  • X is the homogeneous representation of the world coordinate system; and is the equivalent focal length in the horizontal and vertical directions;
  • s is the distortion coefficient of the image;
  • w . , v . Is the image's primary point coordinates.
  • R is the rotation matrix of the camera, which is the camera translation vector.
  • the nickname is the internal reference of the camera, R and the external reference called the camera.
  • H is a 3 x 3 matrix with a degree of freedom of 8, which represents the transformation relationship between the two imaging planes, called the homography matrix.
  • H 01 K 1 R 1 R 0 1 K 0 1 (4)
  • feature point extraction can be performed in an overlapping region by a feature point extraction algorithm such as a SIFT (Scale-invariant feature transform) algorithm.
  • a feature point extraction algorithm such as a SIFT (Scale-invariant feature transform) algorithm.
  • Find multiple feature points establish the matching relationship between feature points, use (3) to establish multiple equations, and find the homography matrix between the two images by iterative optimization algorithm.
  • the two images can be combined together by a coordinate transformation to align the pixels of the overlapping area.
  • x' s tan 1 —
  • x f tan— (5)
  • a major influencing factor is parallax.
  • the algorithms discussed at present can only deal with the stitching of a certain depth level in the image, that is, the stitching on a plane. In theory, it is impossible to seamlessly stitch objects of other depth levels through one transform, except for the depth level on the image. The object will have a ghost phenomenon. It’s hard to be heavy by image processing algorithms. The shadow is completely removed, and a better method is to use the common-optical camera model to minimize parallax.
  • Another major factor is the difference in brightness/chrominance of the image caused by camera exposure/color difference, especially at the seams of the two images.
  • the solution to the cartridge is to perform Alpha blending at the overlap of the seams.
  • a better method is to perform Laplacian pyramid blending or Gradient domain fusion on the entire image. Blending). After the relevant processing, a better panoramic seamless image can be obtained.
  • the panoramic camera A7 is preferably placed at the approximate height of the participant's line of sight A100 (see Figure 2).
  • the panoramic camera A7 is composed of a plurality of ordinary color cameras, and in order to be able to capture fast moving objects in the scene, multiple color cameras need to be synchronized. Since the images obtained by multiple cameras may not be directly spliced, it is also necessary to use an image stitching algorithm to perform image stitching processing on the three video images in order to obtain a seamless panoramic image.
  • the multi-channel video stream output by the panoramic camera A7 can be directly connected to the audio-visual communication device A8, and the image splicing work can be completed by the audio-video communication device A8; or the third-party device (not shown) can be directly connected by the panoramic camera A7.
  • the panoramic image after splicing is input into the audio and video communication device A8, or spliced by the panoramic camera A7 itself, and the spliced image is input to the audio and video communication device A8 by way of one or more video streams.
  • the device with the above image stitching function is called a splicing unit (the principle is described in the previous paragraphs, and the connection relationship with other units is described later in the introduction to FIG. 6).
  • a single projector cannot display a panoramic image well at the time of display, and therefore, it is preferable that the panoramic image can be divided into a plurality of projectors, each of which displays a part of the image. Due to the difference in position, brightness and chromaticity of the projector, it is necessary to process the segmented panoramic image, geometrically correct the image, and eliminate the seam between adjacent images by using luminance/chroma fusion.
  • This function can be completed by a separate third-party device (not shown), or it can be integrated into the audio and video communication device A8.
  • This article refers to the device with image correction/fusion function as the correction/fusion unit. (See the introduction to Figure 6 below).
  • the color of the camera is preferably integrated with the background of the rear black box A6, so that the user is not easily in the illustration of Embodiment 1, the panoramic camera 7 is vertically placed, and the incident light directly passes through the mirror surface. Reflected into the camera.
  • the method of changing the optical path of the incident light by using multiple reflection methods can make the position of the panoramic camera arbitrarily placed as needed.
  • Figure 5 shows the panoramic camera The machine is placed horizontally. By adding a mirror 2 to the mirror 1 of the panoramic camera itself, it is possible to change the horizontally incident light into a vertically incident light, thereby changing the position at which the camera is placed. Since there are multiple cameras, it is necessary to design a corresponding suitable reflecting surface on the top of each camera.
  • the first embodiment of the present invention uses a time division method to coordinate the acquisition of the camera A7 and the projection of the projector A9.
  • the time division method divides the working mode of the system into two modes, a display mode and an acquisition mode.
  • the display mode the projector A9 projects the image of the far end B onto the transparent projection screen A4, and the panoramic camera A7 is in the inactive mode, and no acquisition is performed; in the acquisition mode, the projector A9 is in the inactive mode.
  • the projection is not performed, and the panoramic camera A7 shoots the scene through the transparent projection screen A4.
  • a special sync unit A16 is required to output the sync signal to panoramic camera A7 and projector A9 to control the operating mode of both devices.
  • the synchronization unit A1 6 controls the panoramic camera A7 to acquire during the vertical retrace interval between the two frames/field image of the projector A9, but at this time, the exposure time of the panoramic camera A7 becomes shorter, and the brightness of the image is reduced. .
  • the scene of the remote user B15 is a panoramic image obtained by the panoramic camera B7 and presented locally through the projection screen A4, the local user A15 feels surrounded by the far-end scene, and feels a panoramic view.
  • the seamlessly displayed image does not have a clear sense of speaking to the screen, and the user's immersive experience is enhanced.
  • the image of the remote user B15 is presented on the local transparent projection screen A4, while the periphery of the remote user B15 is a dark background. These portions are not imaged on the transparent projection screen A4, so the local user A15 can see through these portions. Go to the background A6 of the black box A5.
  • the panoramic camera A7 is not affected by the projection image of the projector A9 when the local A image is captured (same reason, the panoramic camera B7 is not subject to the projection image of the projector B9 when capturing the far-end B image.
  • the effect is that the camera can be placed behind the center of the projection screen in the direction of the participant's line of sight, thereby avoiding the generation of vertical parallax and enabling the peer participants to enjoy good eye contact.
  • the synchronization unit A16/B16 has one in the Telepresence system. Some remote collaboration tasks, such as two design teams need to watch design prototypes.
  • the current Telepresence system only supports 2D video, and users cannot see an object with a sense of depth.
  • the recommended solution of the embodiment of the present invention can implement the 3D video when the object is presented by using the stereo camera as the target camera. Referring to FIG. 2, the stereo camera B1 0 of the far end B collects 3D image information of the object to be presented, such as "left eye image + right eye image", or "left eye image + depth map", input to have 3D video coding.
  • the capable audio-video communication device B8, the audio-video communication device B8 processes the 3D image and encodes the audio-video communication device A8 that is transmitted to the local A.
  • the audio and video communication device A8 of the local A decodes and presents the 3D video code stream. If there is a 3D video rendering device locally, it is presented in the form of 3D video, otherwise it is presented in 2D video. For example, if the auxiliary display A12 of the local A is a 3D display, the audio-visual communication device A8 outputs the video of the 3D format to the A12 for display. If the local auxiliary display A12 is a normal 2D display, the audio-visual communication device A8 outputs the 2D format. The video is displayed to the A12.
  • 3D video rendering devices include 3D glasses, autostereoscopic displays, multi-view 3D displays, and the like.
  • audio presentation is preferably performed in the manner of microphone array A3 and speaker array A11.
  • the following tube will introduce the background of the microphone array and speaker array.
  • the reproduction of audio includes the reproduction of sound quality and spatial sense.
  • the former can be achieved with an efficient wide-band compression algorithm.
  • the sense of space not only brings the impression of position and direction, but also enhances the feeling in the same room.
  • the intelligibility of speech and the quick recognition of the speaker are added.
  • the first embodiment of the present invention uses a plurality of microphones or microphone arrays for audio collection, and a plurality of speakers or speaker arrays for audio presentation, which effectively improves the reproduction effect of the sound space.
  • a microphone array is a system consisting of a set of unidirectional mics placed in a certain geometry. Unlike traditional directional microphone acquisition, a directional single microphone can only collect one signal, while a microphone array system can acquire multiple signals. Due to the difference in the position of each microphone, there is some difference in the time or space of the collected data, so that the signal required by the multi-channel signal can extract the required information.
  • the microphone array system also has a spatial selection characteristic that forms a beam-aligned sound source that suppresses the sound of other speakers and ambient noise, thereby obtaining a high-quality sound source signal.
  • sound source localization is a main application scenario of a microphone array.
  • the positioning problem based on the microphone array can be considered as a spatial position of the sound source by using a set of microphones placed at a certain geometric position.
  • Sound source localization algorithms based on microphone arrays can be divided into three categories: controllable beamforming techniques based on maximum output power, orientation techniques based on high resolution spectral estimation, and time-of-arrival (TDOA) based techniques.
  • the first method for the microphone array The received speech signals are filtered, weighted and summed, and then the microphone is directly controlled to direct the beam to the direction of maximum output power.
  • the second method uses the correlation matrix between the mic signals to determine the direction angle to determine the sound source position.
  • the third method first finds the time difference between the sounds reaching the microphones at different positions, and then uses these time differences to find the distance difference between the sounds reaching the microphones at different positions, and finally uses the search or geometric knowledge to determine the sound source position.
  • the speaker array can perform sound field reconstruction and reproduction based on the input audio signal and position information.
  • the speaker array can combine a plurality of sound field units in some manner for sound reinforcement. Compared to direct radiation from a single speaker, the speaker array can boost sound power, increase sound radiation efficiency at low frequencies, improve directivity to improve sound field non-uniformity and improve speech intelligibility in reverberant environments. Wavefront synthesis techniques can be used for speaker arrays.
  • the microphone array A3 can be a linear microphone array or a circular microphone array, placed on a table, or placed in a ceiling.
  • Speaker Array The A11 contains multiple speakers that are integrated with the black box A5 and are distributed in the same direction as the far-end participants shown on projection screen A5.
  • the audio-visual communication device B8 can detect the position information of the participant B15 who is talking, and transmit the audio signal of the far-end B and the position information of the participant B15.
  • the speaker array A11 can perform sound field reconstruction and reproduction based on the input audio signal and position information.
  • the local user A15 can feel that the sound of the far-end user B15 is exactly emitted from the position of the screen B15, so that an experience similar to a face-to-face conversation can be obtained.
  • Embodiment 1 a system operation flow diagram of Embodiment 1 is shown, showing the workflow of the remote B transmitting audio and video information to the local A.
  • the panoramic camera B7 (consisting of a plurality of cameras) synchronously acquires images of scenes of different viewing angles under the control of the synchronizing unit B16.
  • the collected multiple images are sent to the splicing unit for image splicing to obtain a panoramic image of the remote scene B.
  • the panoramic image is output from the splicing unit, subjected to image processing, output to the video encoding unit 1 for encoding, and transmitted in a packet stream in the form of a packet stream.
  • the resolution of the stitched image may be large, and one video encoding unit 1 cannot perform real-time encoding. Therefore, the stitched image may be divided into multiple copies and output to multiple video encoders for encoding.
  • one or more code streams may be formed to be transmitted through the packet network. Due to the delay coding and jitter factors in distributed coding and network transmission, multiple code streams may become out of sync, so it may be necessary to make some marks (such as timestamps) in the code stream, based on the decoding end. These tags synchronize the multistream.
  • the microphone array of the remote B collects the audio signal of the scene, and is encoded by the audio coding unit to form an audio coded stream, which is transmitted through the network in a data packet manner.
  • the audio data and the video data are preferably lip-synchronized, and the lip-synchronization is a prior art in the field of audio and video, and will not be described here.
  • the remote B uses the microphone array to collect audio signals
  • the audio positioning algorithm can also calculate the position information of the speaking participant, and the location information can also be transmitted to the local network.
  • one or more object cameras B10 capture objects that need to be presented in the scene. If it is a plurality of target cameras B10, a stereo camera can be constructed to obtain a 3D image of the scene.
  • the synchronization unit B16 is also required to perform synchronous acquisition between the plurality of cameras.
  • One or more video streams of the object camera B10 are input to the video encoding unit 2 for encoding.
  • the video encoding unit 2 supports a 2D/3D video encoding format, and the encoded code stream data is also transmitted through a packet network.
  • video decoding unit 1 receives the panoramic video stream packet of remote B for decoding. Since the resolution of the panoramic image may be large, one video decoding unit 1 cannot complete the decoding, so multiple video decoding units may be required to work in parallel. In decoding, it is necessary to determine the order in which the video image frames are played according to the synchronization flags in the code stream.
  • the decoded image may be a complete panoramic image or multiple images after segmentation. If it is a complete panoramic image, it is also necessary to divide the image and output it to multiple projectors A9 synchronously. Multiple projectors A9 present images on projection screen A4 in a seamless manner.
  • the audio data stream is decoded by the audio decoding unit to form an audio data signal, which is output to the speaker array.
  • the speaker array can select the one or more speakers closest to the remote participant location displayed on projection screen A4 to present the voice of the remote participant.
  • the video stream of the target camera B10 of the opposite end B is decoded by the video decoding unit 2, and then presented by the auxiliary display A12.
  • the complete audio and video communication device A8 or B8 includes a splicing unit, a video encoding unit 1, 2, an audio encoding unit, a video decoding unit 1, 2, and an audio decoding unit.
  • FIG. 7 and 8 a top view of the conference room layout of the embodiment 1 of the Telepresence system of the present invention is shown in Figure 2 and Figure 2, which uses a front projection technique.
  • Projector A9 can be placed in front of projection screen A4. Hang it on top of the conference table A2 (shown in the picture) or place it under the conference table for projection.
  • the advantage of this scheme is that the light emitted by projector A9 does not cause interference to the user.
  • a conference room layout view 3 and a schematic view 3 of a single embodiment of the present invention are shown.
  • the panoramic camera A7 is placed above the projection screen A4 to obtain a panoramic image of the scene.
  • the synchronization unit A16 is not required for the acquisition and projection synchronization control of the panoramic camera A7 and the projector A9, so that the design of the panoramic camera A7 and the projector A9 can be reduced and the cost can be reduced.
  • the panoramic camera 7 is not placed at the line of sight of the user A15, the effect of the vertical eye on the eye is deteriorated. In general, if the vertical viewing angle difference is ⁇ 7°, the vertical eye view is acceptable for the eye.
  • the mirror of panoramic camera A7 can be placed below and the color camera placed above.
  • the projection screen A4 can adopt a holographic transparent projection screen or a common projection screen. If a holographic transparent projection screen is used, the user can feel the depth level of the scene; If you cast, you can't show the depth of the scene.
  • the backdrop A13 behind the user and the backdrop A6 of the black box A5 can be left without special decoration.
  • a panoramic camera (A7 or B7) is adopted, thereby capturing a panoramic view of the local conference room and providing a panoramic view of the conference for the opposite participants, thereby solving the problem.
  • the panoramic view of the Telepresence system is not good.
  • the projection screen A4 (B4) adopts an ordinary projection screen or adopts a holographic transparent projection screen, the image is presented in an integrated manner, thereby overcoming the defects caused by the television frame when a plurality of flat panel televisions are combined, and achieving seamless Presented.
  • the projection screen A4 (B4) can provide a feeling of depth presentation to the participants by employing a holographic transparent projection screen.
  • the panoramic camera A7 (B7) can capture the local A (distal B) image without being affected by the projected image of the projector A9 (B9), thereby Avoid the generation of vertical viewing angles, so that the opposite participants can enjoy good communication.
  • a panoramic camera A7 (B7) can also be placed over the projection screen A4 (B4) to acquire a panoramic image of the scene.
  • the sync unit A16 (B16) can be omitted, thereby reducing the design and reducing the cost.
  • the mirror of the panoramic camera A7 (B7) can be placed below, the color camera. Place it on top.
  • the projector A9 (B9) is placed in the black box A5 (B5) to protect the projector A9 from ambient light.
  • the projector A9 (B9) can be placed in front of the projection screen A4 (B4), suspended above the conference table or placed underneath for projection using front projection technology. The advantage of this scheme is the use of holographic transparent projection. The light from projector A9 (B9) does not cause interference to the user.
  • Embodiment 2 of the present invention will be described below.
  • FIG. 11 a top view of a conference room layout of Embodiment 2 of the Telepresence system of the present invention is shown.
  • FIG. 12 it is a schematic diagram 1 of Embodiment 2 of the Telepresence system of the present invention corresponding to FIG.
  • the black box design was removed, and the projection screen 4 was placed directly on the opposite side of the seat 1 A, 1 B, 1 C.
  • the projection screen can be designed as a liftable structure, and the projection screen 4 can be raised up when not in use, or hidden in the table 2; when used, the projection screen 4 can be lowered, or from the table. 2 inside rose.
  • the advantage of this is that you can use the table 2 to hold a regular meeting without a video conference.
  • the speaker can be placed under the projection screen 4 using a very thin flat panel speaker; or the vibration module can be directly attached to the screen to make the screen a speaker.
  • the panoramic camera 7 can be hidden into the background wall 14 behind the projection screen 4, and the panoramic camera 7 and the background wall 14 are integrated by special decoration.
  • the projector 9 and the panoramic camera 7 need to be synchronized, and the synchronization mode is the same as that described in the first embodiment, and the synchronization unit A16 is still used for synchronization control.
  • the projector 9 in this embodiment can be in a front projection mode (shown in Fig. 11) or a rear projection mode.
  • FIG. 13 is a schematic diagram 2 of the embodiment 2 of the Telepresence system of the present invention.
  • a frame with a certain thickness for support can be designed, the projection screen A4 is embedded in the frame, and then the panoramic camera A7 is placed in front of the frame or embedded in the frame.
  • the sync unit is not required for the acquisition and projection synchronization control of the panoramic camera A7 and projector A9.
  • the panoramic camera 7 is not placed at the line of sight of the user A15, the effect of the vertical eye on the eye is deteriorated. In general, if the vertical viewing angle difference is ⁇ 7°, the effect of the vertical eye on the eye is acceptable.
  • the mirror of the panoramic camera 7 can be placed below and the color camera placed above.
  • Embodiment 2 referring to FIG. 11, since the projection screen 4 is directly placed opposite the seat, The projection screen can be retracted when not in use, so the solution is compatible with traditional conferences, that is, when the Telepresence conference is not required, the related equipment is hidden, and the conference room is used for ordinary conferences.
  • Embodiment 3 of the present invention will be described below.
  • FIG. 15 is a schematic diagram of Embodiment 3 of a Telepresence system according to the present invention.
  • the third embodiment further simplifies the scheme, and uses a plurality of large-sized televisions 30A, 30B and 30C to form a display system without using a projection screen.
  • the television used can be an LCD TV, a PDP TV or a DLP rear projection television.
  • the panoramic camera A7 is placed above the display to capture the local scene. Because of the TV presentation, the depth effect cannot be obtained, so the backdrop A13 behind the user and the backdrop A6 of the black box can be left without special decoration.
  • Embodiment 3 is upgraded on the existing Telepresence system, and only the replacement of the ordinary camera as a panoramic camera can solve the problem that the panoramic rendering effect is not good.
  • Embodiment 4 of the present invention will be described below.
  • Embodiment 4 of the Telepresence system of the present invention is used.
  • a depth representation is implemented using a half-reflex lens.
  • the holographic transparent projection screen was cancelled and replaced by a half-reflex lens A21.
  • the semi-reverse half lens is attached to the front of the dark box, and the projection screen A22 is located obliquely above the semi-reverse half lens, and is placed at an angle with the half-reflex lens.
  • Projection screen A22 is imaged by rear projection.
  • the image projected by the projector A9 is first imaged by the mirror A23 and projected on the projection screen A22, and the image at A22 is formed by the half mirror A21 to form a virtual image A101, so that the local user A15 feels that the image is seen at a certain depth.
  • the panoramic camera A7 captures the user's image through the half-reverse lens A21 and blends in with the background A6 of the black box A5.
  • the remote user is followed by a dark background so that the locally decorated background A6 can be seen by the local user A15 through dark areas other than the bright areas of the remote user's body.
  • the remote user image seen by the local user appears to be in front of the background.
  • the mirror A23 is optional, and when the mirror A23 is not used, the projector A9 can adopt the front projection scheme.
  • the semi-reverse half lens A21 can solve the problem of deep presentation and witnessing the communication.
  • Embodiment 5 of the present invention will be described below.
  • Embodiment 5 of the Telepresence system of the present invention employs a transparent light-conducting element A25 having a light redirecting area A25 for inputting an image.
  • the A24 acts as a waveguide device that internally transmits the image of the participant A15 to the panoramic camera A7 at the bottom of the black box A5 by means of internal reflection.
  • the incident light A102 is internally reflected multiple times between the two inner surfaces A26 and A27 of the light guiding element, and finally emitted as the outgoing light A1 03 at the bottom of the dark box A5, and is captured by the panoramic camera A7.
  • the light conducting element is placed in front of the projection screen A4, and the input area A25 covers the surface of the projection screen A4.
  • A25 must be sufficiently transparent to not interfere with user A15.
  • the inner surfaces of the input area A26 and A27 can be realized by a holographically derived grating, and the components are made of transparent panels made of glass or plastic.
  • Embodiment 5 realizes eye contact through the light conducting element A25 on the basis of realizing panoramic rendering by the panoramic camera A7 and seamless display by the projection screen A4, and preferably, the projection screen A4 adopts a holographic transparent projection screen, The effect of deep rendering can be achieved.
  • Embodiment 6 of the present invention will be described below. This embodiment uses a polarizer to implement a panoramic Tslsprsssncs system that supports vertical eye-to-eye effects.
  • the first principle is to introduce the well-known principle of polarized light.
  • the light wave is a transverse wave, that is, the vibration direction of the light wave vector is perpendicular to the direction of light propagation.
  • the vibration of the light wave vector is randomly oriented perpendicular to the direction of propagation of the light, but statistically, in all possible directions of space, the distribution of the light wave vector can be regarded as equal opportunity.
  • the sum of them is symmetrical with the direction of light propagation, that is, the light vector has axial symmetry, uniform hook distribution, and the same amplitude of vibration in all directions.
  • This kind of light is called natural light.
  • Polarized light means that the direction of vibration of the light vector is constant, or has a light wave that changes regularly.
  • polarized light can be divided into plane polarized light (linearly polarized light), circularly polarized light and elliptically polarized light, and partially polarized light. If the vibration direction of the light wave electric vector is limited to a certain plane, the polarized light is called plane polarized light, and if the track is a straight line during propagation, it is also called linearly polarized light. If the light wave electric vector changes regularly with time, that is, the electric vector end track is a straight line in the vertical trajectory during the propagation process, it is also called linearly polarized light.
  • the plane is circular or elliptical, which is called circularly polarized light or elliptically polarized light. If the vibration of the light wave electric vector only has a comparative advantage in a certain direction during the propagation, the polarized light is called partially polarized light.
  • the polarizing plate is a film made by an artificial method, and is made by a special method for regularly arranging fine crystals with strong selective absorption in a transparent rubber layer, which allows light to pass through a direction of vibration of an electric vector. (This direction is called the polarization direction), and the light that absorbs its vertical vibration has dichroism.
  • FIG. 18 it is a schematic diagram of Embodiment 6 of the Telepresence system of the present invention.
  • a linear polarizer is placed in front of the lens of the projector A9 and the panoramic camera A7, and there is a difference between the polarization angle of the linear polarizer of the projector A9 and the polarization angle of the linear polarizer of the panoramic camera A7 (because The polarization direction of the polarizer in front of the panoramic camera is different from the polarization direction of the light emitted by the projector, and has a difference.
  • the principle of polarized light the light emitted by the projector cannot enter through the polarizer in front of the panoramic camera.
  • the difference is 90 degrees, that is, the polarization direction of the projector A9 and the polarization direction of the panoramic camera A7 are vertical.
  • the polarizing direction of projector A9 is vertical
  • the polarizing direction of panoramic camera A7 is horizontal.
  • a projection screen A4 made of a special material may be used, which is a translucent screen formed by the polarizing material A41 of the panoramic camera A7 and other materials A42.
  • the circularly polarized light input to the scene becomes horizontally polarized after passing through the projection screen A4, and can be captured by the panoramic camera A7; and the light emitted by the projector A9 is vertically linearly polarized, which cannot pass through the camera A7.
  • the horizontal linear polarizer is thus not captured by camera A7. In this way, the shooting of the panoramic camera A7 and the projection of the projector A9 will not cause interference.
  • Embodiment 6 on the basis of panoramic display using the panoramic camera A7 and seamless display by the projection screen A4, eye contact is realized by adding a polarizer in front of the camera and the projector.
  • Embodiment 7 of the present invention will be described below.
  • Embodiment 7 aims to solve the problem of the arrangement of the dark background behind the user in the previous scheme.
  • the background behind the user needs to be arranged as a fixed black background, such as using a black curtain, or painting the wall black.
  • This background arrangement may not be acceptable to users in some conference rooms. For example, users feel that placing a black background behind them and meeting room decoration schemes are not coordinated.
  • FIG. 19 it is a schematic diagram 1 of Embodiment 7 of the Telepresence system of the present invention. Using a background projector A50, the background of the user to be displayed is projected onto a pure black projection screen A13.
  • the background projector A50 is connected to a sync unit A16 which coordinates the acquisition of the panoramic camera A7 and the projection of the background projector A50.
  • the time division mode of the system is divided into two modes, a background projection mode and a camera acquisition mode.
  • the background projector A50 projects the background onto the black screen A13, at which time the panoramic camera A7 is in the inactive mode and does not acquire; in the camera acquisition mode, the background projector A50 is in the inactive mode, not Projection, while the panoramic camera A7 shoots the scene.
  • the background of the user A15 photographed by the panoramic camera A7 is a black background.
  • the local background seen by the user is not black, but the image projected by the background projector A50.
  • the image is replaceable and can therefore be coordinated with the conference room renovation program.
  • FIG. 20 it is a schematic diagram 2 of Embodiment 7 of the Telepresence system of the present invention.
  • a linear polarizing plate is attached in front of the background projector A50 for projecting the background, so that the light projected onto the background wall is linearly polarized light.
  • a linear polarizer is also placed in front of the panoramic camera A7 with a polarization angle perpendicular to the polarization angle of the polarizer in front of the background projector A50.
  • the background light emitted by the background projector A50 cannot be captured by the panoramic camera A7, and the room lighting reflected by the foreground person is circularly polarized and can be photographed by the camera A7.
  • the background behind the person is black on the captured image, thus solving the dark background problem.
  • FIG. 21 it is a schematic diagram 3 of Embodiment 7 of the Telepresence system of the present invention.
  • the background image is displayed on the back of the user using a large-sized background display A51.
  • the background display A51 is connected to the sync unit A1 6 which coordinates the acquisition of the panoramic camera A7 and the display of the background display A51.
  • the time division mode of the system is divided into two modes, the background display mode and the camera acquisition mode.
  • the background display mode the background display A51 displays a normal background image
  • the panoramic camera A7 is in the inactive mode, and no acquisition is performed
  • the background display A51 displays a pure black background image
  • the panoramic camera A7 Take a picture.
  • the background of the user A15 captured by the panoramic camera A7 is a black background.
  • the image displayed by the A51 that the user sees is not black.
  • FIG. 22 it is a schematic diagram 4 of Embodiment 7 of the Telepresence system of the present invention.
  • a large background display A51 is placed behind the person and in front of the background display A51 and panoramic view
  • the front side of the camera A7 is respectively provided with a linear polarizing plate.
  • the background display A51 is an LCD display, since the background light emitted by the LCD itself is polarized light, a linear polarizing plate can be added only in front of the panoramic camera A7, and its polarization is The angle of polarization of the background polarized light emitted by the background display A51 is perpendicular.
  • the background light of the background display A51 cannot be captured by the panoramic camera A7, and the foreground person reflects that the conference room light is circularly polarized and can be photographed by the panoramic camera A7.
  • the background behind the person is black on the captured image, thus solving the dark background problem.
  • Embodiment 7 projects the background of the user to be displayed on a pure black projection screen A13 through the background projector A50 or the background display A51, thereby solving the problem of the layout of the dark background behind the user.
  • This embodiment 7 can be applied in combination with the embodiments 1 to 6.
  • the embodiment of the present invention is upgraded on the existing Telepresence system, and only needs to replace the ordinary camera as a panoramic camera to capture the panoramic view of the local conference room, and provide a panoramic view of the conference for the opposite participants, thereby solving the problem.
  • the Telepresence system has a problem of poor panoramic rendering and is easily compatible with existing Telepresence systems.
  • the ordinary projection screen or the holographic transparent projection screen is used for the unitary presentation of the panoramic image, thereby overcoming the defects of the combination of the plurality of flat panel televisions, and achieving seamless presentation.
  • the sensation of depth presentation can be provided to the participants.
  • the panoramic camera is not affected by the projected image of the projector when the local image is captured, thereby avoiding the parallax caused by the camera being unable to be placed on the line of sight of the user, thereby enabling the opposite end parameter
  • eye contact can be achieved using a half mirror or a light conducting element or a linear polarizer.
  • a dark background is arranged behind the user to separate the user image and the background image, thereby realizing a rendering effect with a sense of depth.
  • the embodiment of the present invention further provides a remote presentation method.
  • the method embodiment includes:
  • S2301 Acquire a local panoramic image and audio, wherein images of different viewing angles are captured by the panoramic camera, and low-resolution images of different viewing angles captured by the panoramic camera are stitched into a high-resolution panoramic image by a splicing unit ;
  • S2302 The local panoramic image and the audio are transmitted to the remote end through the network for display and playback, respectively.
  • the panoramic camera adopts the plane mirror virtual common optical mode, the concentrated multi-camera mode or the dense camera array mode to shoot.
  • the images and audio are alternately acquired in chronological order.
  • the method further comprises: collecting the local three-dimensional video through the stereo camera, and transmitting to the remote end through the network for display through the auxiliary display device.
  • the method before displaying the panoramic image, the method further comprises: performing geometric correction and edge fusion on the panoramic image.
  • the method may further comprise the steps of: receiving remote participant location information; performing reconstruction and reproduction of the sound field on the received audio based on the location information.
  • the method may further comprise the step of: lip-synchronizing the locally acquired audio data and video data.
  • an embodiment of the present invention further provides a video collection device in a remote presentation system.
  • the video capture device works in conjunction with the video display device, the audio capture device, the audio playback device, and the audio and video communication device in the remote presentation system, wherein the audio and video communication device collects images and audio collected by the local video capture device.
  • the audio collected by the device is transmitted to the remote end through the network, and the video display device and the audio playback device respectively display and play the image and audio.
  • the video capture device in the embodiment of the present invention is a panoramic view.
  • the camera, the low-resolution images of different viewing angles captured by the panoramic camera are spliced into a high-resolution panoramic image by a splicing unit.
  • the splicing unit is an independent device, either as part of a panoramic camera or as part of the audio and video communication device.
  • the panoramic camera adopts a planar mirror virtual common optical mode, a converged multi-camera mode or a dense camera array mode for shooting.

Description

远程呈现系统、 方法及视频采集设备
技术领域
本发明涉及视频会议技术领域， 特别涉及一种远程呈现（Telepresence ) 系统、 方法及视频采集设备。
背景技术
Telepresence 是一种实现虚拟会议环境的视频会议系统。 该虚拟会议环 境的目标是， 充分体现参会者的人性化因素， 并尽可能地复制参会者的真实体 验， 能够极大地提高最终用户的接受度， 从而提高设备使用率、投资回报率和 用户满意度。理想的 Telepresence系统相比传统视讯会议系统具有很多优点， 包括： 真人大小的图像； 流畅的运动； 精确的肢体行为； 高清的、 演播室级的 视频、 光照和音频； 目艮神交流， 以及大型用户群的近似目艮神交流； 沉浸式会议 环境， 使参会者感觉处于相同的会议地点； 不同会议地点的一致性； 隐藏摄像 机， 减少对用户的影响等。
本申请发明人在研究过程中发现， 目前 Telepresence系统存在全景呈现 效果不好的问题。 目前的 Telepresence系统大部分采用多个摄像机以及多个 大屏幕显示器的方式进行采集 /显示，每个摄像机 /显示器采集 /显示本地 /远端的 1人或多人，如 Cisco、HP等公司的 Telepresence系统。由于目前 Telepresence 系统都采用多个摄像机进行拍摄， 由此， 这多个摄像机的图像存在视差， 因此 无法通过摄像机位置摆放将多个图像拼接为全景，需要使用显示器边框掩盖摄 像机视角交接处图像的瑕疵， 由此， 目前 Telepresence系统无法给参会者一 个良好的全景体验， 参会者在显示器边框附近区域运动时图像效果不理想， 甚 至让人难以接受。
另外， 由于目前 Telepresence产品还处于初级阶段， 因此除了上述存在 的全景呈现效果不好的问题之外， 本申请发明人在研究过程中进一步发现， 目 前 Telepresence系统至少在以下方面还需要改进：
1、 深度呈现
大部分 Telepresence系统呈现的图像都还是 2D图像，即用户看到的是一 个平面图像， 无法感受到对方会议的场景深度信息。
2、 无缝显示 目前的 Telepresence系统大多采用多个大尺寸平板电视（液晶或等离子） 组合进行呈现，这种方式呈现的场景图像在两个平板电视相邻的显示区域会被 电视边框遮挡住一部分内容， 无法给参会者提供一个全景无缝的体验。
3、 目艮神交流 /凝视感知
目艮神交流（眼对眼， Eye-to-Eye )是一种非常重要的非语言交流方式。 眼 神交流在生理上会引起心跳、 血压的变化， 并提高大脑的活跃度。 凝视感知提 供了许多交流的基础， 包括反馈、 对话方式以及情感表达等， 是感知对方思想 的一种关键方式。 传统的视频会议系统以及目前的 Telepresence系统无法使 用户进行很好的眼神交流， 其主要原因是由于视差问题： 在直觉上用户会盯着 屏幕上的对方看而不是盯着摄像机看， 而摄像机往往都不是放置在屏幕中心 的， 这导致摄像机拍摄的画面和用户的正视画面存在一个视差，从而无法实现 良好的眼神交流。
发明内容
本发明实施例提供了一种改进的 Telepresence系统、 方法及视频采集设 备， 以解决目前 Telepresence系统存在的全景呈现效果不好的问题， 进一步， 本发明实施例在深度呈现、 无缝显示以及眼神交流方面作了改进。
本发明提供的方案包括：
一种远程呈现系统， 包括视频采集设备、 视频显示设备、 音频采集设备、 音频播放设备以及音视频通信设备，所述音视频通信设备将本地视频采集设备 采集的图像以及音频采集设备采集的音频通过网络传输给远端，由远端的视频 显示设备和音频播放设备分别对图像和音频进行显示和播放，所述视频采集设 备为全景摄像机； 所述系统还包括拼接单元， 所述拼接单元用于将所述全景摄 像机拍摄到的不同视角的低分辨率的图像拼接为高分辨率的全景图像。
一种远程呈现方法， 包括： 获取本地的全景图像以及音频， 其中， 通过全 景摄像机拍摄不同视角的图像，并通过一拼接单元将所述全景摄像机拍摄到的 不同视角的低分辨率的图像拼接为高分辨率的全景图像；通过网络将本地的全 景图像以及音频传输给远端分别进行显示和播放。
一种远程呈现系统中的视频采集设备，所述远程呈现系统还包括视频显示 设备、 音频采集设备、 音频播放设备以及音视频通信设备， 所述音视频通信设 备将本地视频采集设备采集的图像以及音频采集设备采集的音频通过网络传 输给远端，由远端的视频显示设备和音频播放设备分别对图像和音频进行显示 和播放， 所述视频采集设备为全景摄像机， 所述全景摄像机拍摄到的不同视角 的低分辨率的图像通过一拼接单元拼接为高分辨率的全景图像。
可见，本发明实施例在现有 Telepresence系统上升级， 只需更换普通摄像 机为全景摄像机，即可拍摄到本地会议室的全景，为对端参会者提供会议全景， 由此解决了目前 Telepresence系统存在的无法获得良好的全景呈现效果的问 题， 并与现有 Telepresence系统容易实现兼容。
优选地， 采用普通投影屏幕或全息透明投影屏幕， 都是对图像进行无缝一 体式呈现， 由此克服了多个平板电视组合呈现的缺陷， 实现了无缝呈现。
优选地，通过采用全息透明投影屏幕和半反半透镜， 可为参会者提供深度 呈现的感觉。
优选地，通过同步单元的控制，使全景摄像机拍摄本地图像时不受投影仪 投影图像的影响，由此可避免因摄像机无法摆放在用户视线位置上而导致的视 差， 从而可使对端参会者获得良好的眼神交流享受。 另外， 也可采用半反半透 镜或者光传导元件或者线性偏振片的方案实现眼神交流。
优选地，通过布置特殊的暗背景，使用背景投影仪或者背景显示器的方式， 在用户后面布置一个暗的背景，使用户图像和背景图像分离显示，从而实现具 有深度感的呈现效果。
附图说明
图 1 为本发明 Telepresence系统实施例 1会议室布局俯视图 1 ;
图 2 为本发明 Telepresence系统实施例 1示意图 1 ;
图 3为本发明 Telepresence系统实施例 1全息投影原理图；
图 4为本发明 Telepresence系统实施例 1全景摄像机原理图；
图 5为本发明 Telepresence系统实施例 1多次反射全景摄像机原理图； 图 6为本发明 Telepresence系统实施例 1原理图；
图 7为本发明 Telepresence系统实施例 1会议室布局俯视图 2;
图 8为本发明 Telepresence系统实施例 1示意图 2;
图 9为本发明 Telepresence系统实施例 1会议室布局俯视图 3; 图 10为本发明 Telepresence系统实施例 1示意图 3;
图 11为本发明 Telepresence系统实施例 2会议室布局俯视图； 图 12 为本发明 Telepresence系统实施例 2示意图 1 ;
图 13 为本发明 Telepresence系统实施例 2示意图 2;
图 14为本发明 Telepresence系统实施例 3会议室布局俯视图； 图 15为本发明 Telepresence系统实施例 3示意图；
图 16为本发明 Telepresence系统实施例 4示意图；
图 17为本发明 Telepresence系统实施例 5示意图；
图 18为本发明 Telepresence系统实施例 6示意图；
图 19为本发明 Telepresence系统实施例 7示意图 1；
图 20为本发明 Telepresence系统实施例 7示意图 2;
图 21为本发明 Telepresence系统实施例 7示意图 3;
图 22为本发明 Telepresence系统实施例 7示意图 4;
图 23为本发明 Telepresence方法流程图。
具体实施方式
为了使本发明的目的、技术方案和优点更加清楚， 下面结合附图和具体实 施例对本发明进行详细描述。
首先介绍本发明实施例 1。
参见图 1 , 为本发明 Telepresence系统优选实施例 1会议室布局俯视图 1。 图 1中， 会场的前后墙壁 13、 14采用弧面或平面设计， 中间设置一个会议桌 2。 会议桌 2上安装了一个麦克风阵列 3 , 为了达到更好的会议音频数据的采集效 果， 麦克风阵列 3可放在会议桌 2的中间位置， 会议桌 2的一侧放置了多个参会 者座椅： 1 A, 1 B , 1 C。 参会者座椅面对一个投影屏幕 4, 该投影屏幕 4为弧形 投影屏幕或平面投影屏幕（在图中， 画出了弧形的投影屏幕）， 构成了一个暗 箱 5的一个前表面， 暗箱 5中安装了一个全景摄像机 7 (此处的全景摄像机 7包 括三个摄像机， 具体参见图 4及其说明）、 音视频通信设备 8、 多台投影仪 9A, 9B , 9C以及包含多个扬声器， 例如图中示出的 5个扬声器 11 A, 11 B , 11 C , 11 D , 11 E构成的扬声器阵列。 和投影屏幕 4相对的暗箱 5内壁 6为经过特殊装 饰的背景幕，所谓特殊装饰，是指具有深度提示效果并能够隐藏摄像机的装饰。 参会者座椅的背面也是经过特殊装饰的暗背景墙幕 13。会议室的一侧放置了一 台辅助显示器 12。 暗箱 5的上方放置了一个或多个对象摄像机 10A, 10B , 构 成了一个或多个立体摄像机对， 其中， 一个立体摄像机对由两个普通对象摄像 机组成， 模拟人的视觉原理， 从左右两个视角同步拍摄场景， 得到左右图像。
参见图 2, 显示了本发明实施例 1的示意图 1。 从图 2可以看出， 本地和远 端的 Telepresence会场具有相同的配置， 并通过网络进行互联。
优选地，在实施例 1中， 本地的投影屏幕 A4和远端的投影屏幕 B4采用了全 息 ( Holographic )透明投影屏幕。 全息透明投影屏幕是采用全息技术的一种 背投影屏幕， 具有全息图像的特点， 只显示来自背投射某一特定角度的图像， 而忽略其他角度的光线。 全息屏幕具有非常明亮、 清晰的显示效果， 即使是在 环境光线很亮的地方， 同时还具有透明的特点，使观者可以透视到屏幕后面的 物体。 目前已经有一些厂商生产全息透明投影屏幕， 如 Woehburk, HoloPro,
Sax3D等厂商。 图 3显示了全息投影的基本原理。 如图， 成 角入射的投影光 线被全息材料和透明面板构成的全息透明投影幕散射，这样用户就能够看到投 影内容而无法看到投影内容区域后面的物体，而投影内容之外的投影幕部分后 面物体发出的水平光线被投影幕透射，这样用户就能透过这部分投影幕区域看 到投影幕后面的物体。
在实施例 1中， 本地的暗箱 A5中的投影仪 A9的投影角度和水平线具有一个 安装夹角 ,投影仪 A9将远端参会者 B15的图像投影到本地的全息透明屏幕 A4 上。由于呈现的是远端场景 B的全景，一个投影仪的分辨率可能无法满足要求， 因此音视频通信设备 A8将远端的图像分割为多份， 并通过多台投影仪 A9进行 呈现（由于在图 2中采用右视图作为其示意图， 因此， 多台投影仪重叠， 而不 能全部呈现）。 为了隐藏投影仪， 投影仪 A9的颜色最好和后面的暗箱背景 A6 融为一体。
为了采集本地 A的全景图像， 暗箱 A5中安装了 1台全景摄像机 A7。 参见图
4 ( A )和图 4 ( B ), 显示了全景摄像机成像的一种实现方案的具体原理。 该全 景摄像机采用平面镜反射虚拟共光心的原理实现（虚拟共光心模式全景摄像 机）。 其中棱台结构 1 001具有 3个反射表面 1002, 1003 , 1004, 这些表面为平 面镜面， 镜面的下方放置 3个摄像机 C01 , C02和 C03。 下面以其中的一个摄像 机 C02为例说明虚拟共光心原理。 参见图 4 ( B ), L02为入射光线， R02为反射 光线，垂直于反射面 1003的法线为 1006,法线 1006和水平线 1010的夹角为^ 反射点到摄像机 C02的实际光心 02的垂直距离为 d。 根据光线反射原理， 摄像 机会拍摄到一个虚像， 该虚像有一个虚拟光心〇0。 只要通过合理的设计 和 d 的值， 就可以使摄像机 C01 , C02和 C03的虚拟光心恰好位于同一点， 从而得 到共光心的 3个图像。对这 3个图像进行拼接，可以得到在任意深度上都是无缝 拼接的图像。 在设计全景摄像机时， 为了获得更好的垂直眼对眼效果， 摄像机 的光心位置越低越好。如果摄像机的几何高度不变， 可以通过减小摄像机和反 射镜面的水平距离达到这个目的，但是这个距离受摄像机镜头和反光镜尺寸的 限制， 会减小拍摄视角， 如图 4 ( C )所示。 全景摄像机的另一种实现方案是 采用汇聚多摄像机模型，通过拍摄得到多个图像然后进行数字图像拼接得到全 景图像（汇聚模式全景摄像机）。 如图 4 ( D )所示。 因为光心在摄像机内部， 因此这种摄像机模型仅依靠位置摆放是无法做到共光心的，输出图像的重叠区 域存在视差， 必须依赖图像处理技术才能取得较好的拼接效果。全景摄像机的 另一种方式是使用多摄像机阵列（多摄像机阵列模式全景摄像机 ),如图 4( E )。 摄像机阵列可以才艮据不同的场景可以采用不同的形式，如线性阵列， 环形阵列 或矩阵阵列等。摄像机阵列中的每个摄像机都采用分辨率较低的摄像机，相邻 两个摄像机之间的间隔较小， 具有较大的拍摄重叠区域。 利用图像拼接处理技 术能够将多个较低分辨率的摄像机图像拼接为一个高分辨率的全景图像。图像 拼接算法的基本原理是估算出多个摄像机的内部参数（如焦距，主点和畸变等） 和摄像机之间的位置参数（如旋转矩阵， 平移向量）， 利用估算出的参数通过 算法将多摄像机图像对齐，消除重叠区域，并进行边缘融合、视差去除等处理， 获得一个高分辨率的全景图像。
为了理解顺畅， 下面介绍公知的图像拼接处理技术：
图像拼接算法的基本原理是估算出多个摄像机的内部参数和之间的位置 参数， 利用估算出的参数通过算法将多摄像机图像对齐， 消除重叠区域， 并进 行边缘融合、 视差去除等处理， 获得一个高分辨率的全景图像。 根据射影几何 原理， 空间中的三维点投影到摄像机成像平面上的变换关系为：
X = K [R I t]x ( 1 )
Figure imgf000009_0001
其中 为平面坐标的齐次表示； X为世界坐标系的齐次表示； 和 为水 平和垂直方向上的等效焦距； s为图像的畸变系数； w。，v。为图像主点坐标。 R为 摄像机的旋转矩阵， 为摄像机平移向量。 其中 Κ称为摄像机的内参， R和 称 为摄像机的外参。对于两个摄像机拍摄的或一个摄像机在不同位置拍摄的具有 重叠区域的多个图像， 空间中的一个点两个图像上的成像关系为：
其中 H为一个 3 x 3的矩阵， 自由度为 8, 其代表了两个成像平面之间的变 换关系， 称之为单应性矩阵。对于只有旋转运动的纯旋转摄像机系统或共光心 摄像机系统， 可以表示为：
H 01 = K 1R 1R 0 1K 0 1 (4) 由此， 可以通过特征点提取算法， 例如 SIFT(Scale-invariant feature transform , 尺度不变性变换)算法， 在重叠区域进行特征点提取， 找到多个特 征点， 建立特征点之间的匹配关系， 利用 （3 )建立多个方程组， 通过迭代优 化算法求出两个图像之间的单应性矩阵 。 求出 H之后就可以通过一个坐标变 换将两个图像合成到一起，将重叠区域的像素对齐。对于只有水平方向旋转的 摄像机模型，我们还可以利用柱面坐标变换将平面坐标转换为柱面坐标，在柱 面坐标下通过对图像的平移来进行对齐。 柱面坐标的变换和反变换为： x' = s tan 1— x = f tan— (5)
f s
Figure imgf000009_0002
通过上述的方法进行了图像变换后，一般还是无法得到比较理想的无缝全 景图像， 还必须考虑一些其它因素的影响。 一个主要的影响因素是视差。 目前 讨论的算法都只能处理图像中某个深度层次的拼接，即近似在一个平面上的拼 接，理论上无法通过一个变换使其它深度层次的物体都无缝拼接， 图像上除了 该深度层次上的对象都会出现重影的现象。通过图像处理算法目前都很难将重 影完全去除， 比较好的方法是使用共光心摄像机模型尽量减少视差。 另一个主 要因素是摄像机曝光 /颜色差异导致的图像在亮度 /色度上的差异， 在两个图像 的接缝处尤其明显。 筒单的解决方法是在接缝处的重叠区域进行 Alpha融合 ( Alpha blending ), 比较好的方法可以在整个图像上进行拉普拉斯金字塔融 合 ( Laplacian pyramid blending ) 或是梯度阈融合 (Gradient domain blending)。 在进行了相关的处理后， 可以得到一个比较好的全景无缝图像。
为了得到良好的垂直眼对眼效果， 全景摄像机 A7最好安放在参会者视线 A100的近似高度上（请参见图 2 )。 该全景摄像机 A7由多台普通彩色摄像机构 成， 为了能够拍摄场景中快速运动的物体， 多台彩色摄像机需要进行同步。 由 于多台摄像机得到的图像可能无法直接进行拼接，因此还需要采用图像拼接算 法对 3路视频图像进行图像拼接处理， 以便得到无缝的全景图像。 全景摄像机 A7输出的多路视频流可以直接连接到音视频通信设备 A8 , 由音视频通信设备 A8完成图像拼接工作； 也可以由全景摄像机 A7直接连接一个第三方设备（图 中未标出）进行拼接， 拼接完成后的全景图像再输入到音视频通信设备 A8中， 或是由全景摄像机 A7自身进行拼接， 将拼接好的图像以 1路或多路视频流的方 式输入到音视频通信设备 A8中。 本文将具有上述图像拼接功能的设备称为拼 接单元（其原理参见前几段介绍， 与其他单元的连接关系详见后面对图 6的介 绍）。 另外， 在显示时， 单台投影仪无法很好地显示全景图像， 因此优选地， 可将全景图像进行按照多个投影仪的数量进行分割，每个投影仪显示图像的一 部分。 由于投影仪之存在位置、 亮度和色度等的差异， 因此需要对分割后的全 景图像进行处理， 对图像进行几何校正， 并采用亮度 /色度融合的方法消除相 邻图像间的接缝。 该功能可以由一个独立的第三方设备（图中未标出） 完成， 也可以将此功能集成到音视频通信设备 A8中完成， 本文将具有图像校正 /融合 功能的设备称为校正 /融合单元（详见后面对图 6的介绍）。 此外为了隐藏全景 摄像机 A7, 该摄像机的颜色最好和后面的暗箱 A6背景融为一体， 使用户不易 在实施例 1的图示中，全景摄像机 7是垂直摆放的，入射光线直接通过反射 镜面反射到摄像机中。但在实际设计时， 采用多次反射的方法改变入射光的光 路的方法， 可以使全景摄像机的位置根据需要任意摆放。 图 5显示了全景摄像 机水平放置的方案。 通过在全景摄像机本身的反光镜 1的上方再加上一个反光 镜 2, 可以将水平入射的光线改变成垂直入射的光线， 从而可以改变摄像机放 置的位置。 由于有多个摄像机， 因此每个摄像机对应的上方都需要设计相应的 合适反射面。
为了使全景摄像机 A7拍摄本地 A图像时不受投影仪 A9投影图像的影响， 优 选地， 本发明实施例 1采用了时分的方法来协调摄像机 A7的采集和投影仪 A9 的投影。该时分的方法将系统的工作方式分为两种模式，显示模式和采集模式。 在显示模式下，投影仪 A9将远端 B的图像投影到透明的投影屏幕 A4上，此时全 景摄像机 A7处于未激活模式， 不进行采集； 在采集模式下， 投影仪 A9处于未 激活模式， 不进行投影， 而全景摄像机 A7透过透明的投影屏幕 A4拍摄场景。 为了使摄像机 A7和投影仪 A9能够协调工作， 需要一个特殊的同步单元 A16输 出同步信号到全景摄像机 A7和投影仪 A9 , 控制两个设备的工作模式。 例如， 同步单元 A1 6控制全景摄像机 A7在投影仪 A9两帧 /场图像之间的垂直回扫时间 间隔内进行采集， 但此时全景摄像机 A7的曝光时间变短， 图像的亮度会有所 降低。为了解决该问题，可以采用曝光时间短的摄像机或降低投影仪的刷新率。
通过上面的介绍可以看出， 由于远端用户 B15的场景通过全景摄像机 B7 获得的是一个全景图像， 在本地通过投影屏幕 A4呈现， 因此本地用户 A15感觉 到被远端场景包围，感觉是一个全景且无缝显示的图像， 不会有明显的对着屏 幕说话的感觉， 用户的沉浸感体验得到了增强。 而且， 远端用户 B15的图像在 本地透明投影屏幕 A4上呈现， 而远端用户 B15的周围是暗背景， 这些部分不会 在透明投影屏幕 A4上成像， 因此本地用户 A15可以透过这些部分看到暗箱 A5 的背景 A6。 由于透明投影屏幕 A4和暗箱 A5的背景 A6之间有一个物理距离， 而 且暗箱 A5的背景 A6被特殊装饰过，能够给人带来深度错觉，因此本地用户 A15 看到的远端用户 B15图像就具有了深度层次。另夕卜，通过同步单元 A1 6的控制， 使全景摄像机 A7拍摄本地 A图像时不受投影仪 A9投影图像的影响（同理，全景 摄像机 B7拍摄远端 B图像时不受投影仪 B9投影图像的影响）， 由此可以将摄像 机放在投影屏幕中心的后面，位于参会者视线的方向上，从而可以避免垂直视 差的产生， 使对端参会者获得良好的眼神交流享受。
同步单元 A16/B16除了面对面的视频交流外， Telepresence系统中还有一 些远程协作任务， 如两个设计团队需要观看设计原型等。 目前的 Telepresence 系统都只支持 2D视频， 用户无法看到一个具有深度感的物体。 本发明实施例 的推荐方案可采用立体摄像机作为对象摄像机的方式实现对象呈现时的 3D视 频。参见图 2,远端 B的立体摄像机 B1 0采集需要呈现物体的 3D图像信息，如"左 眼图 +右眼图"， 或 "左眼图 +深度图" 的形式， 输入到具有 3D视频编码能力的 音视频通信设备 B8 , 音视频通信设备 B8对 3D图像进行处理并编码发送到本地 A的音视频通信设备 A8。本地 A的音视频通信设备 A8对 3 D视频码流进行解码呈 现。 如果本地有 3D视频呈现设备， 则以 3D视频的方式呈现， 否则以 2D视频的 方式呈现。 例如如果本地 A的辅助显示器 A12是一个 3D显示器， 则音视频通信 设备 A8输出 3D格式的视频给 A12进行显示，如果本地辅助显示器 A12为普通的 2D显示器， 则音视频通信设备 A8输出 2D格式的视频给 A12进行显示。 3D视频 呈现设备包括 3D眼镜、 自动立体显示器和多视角 3D显示器等。
除视频之外， 为了获得良好的沉浸式音频体验， 优选采用麦克风阵列 A3 和扬声器阵列 A11的方式进行音频呈现。 下面筒要介绍一下麦克风阵列和扬声 器阵列的背景。在 Telepresence系统中，音频的再现包括音质和空间感的再现， 前者可以用高效的宽频压缩算法做到， 空间感除了带来位置和方向的印象之 夕卜，也增强了在同一个房间的感觉， 此外增加了语音的可理解度以及说话人的 快速辨别。 本发明实施例 1的采用多个麦克风或麦克风阵列进行音频采集， 以 及多个扬声器或扬声器阵列来进行音频呈现，有效地改进了声音空间感的再现 效果。 麦克风阵列是由一组按一定几何结构摆放的单向麦克组成的系统。 与传 统的指向性麦克风采集不同，指向性单麦克风一般只能采集一路信号， 而麦克 风阵列系统可以采集多路信号。 由于各个麦克位置的不同， 采集的数据在时间 或者空间上存在某些差异，从而通过多路信号的信号处理技术可以提取出所需 要的信息。 麦克风阵列系统还具有空间选择特性， 其形成的波束对准声源， 抑 制了其他说话人的声音和环境噪声，从而获得高品质的声源信号。 目前音源定 位是麦克风阵列的一个主要应用场景。基于麦克阵列的定位问题可以认为是利 用一组按一定几何位置摆放的麦克定出声源的空间位置。基于麦克风阵列的声 源定位算法可分为三类：基于最大输出功率的可控波束形成技术、 基于高分辨 率谱估计的定向技术和基于到达时间差 (TDOA)技术。 第一种方法对麦克风阵 列接收到的语音信号进行滤波、加权求和， 然后直接控制麦克风指向使波束有 最大输出功率的方向。第二种方法利用求解麦克信号间的相关矩阵来定出方向 角，从而确定声源位置。第三种方法首先求出声音到达不同位置麦克的时间差， 再利用这些时间差求得声音到达不同位置麦克的距离差，最后用搜索或几何知 识确定声源位置。扬声器阵列可以根据输入的音频信号和位置信息进行声场重 建和再现。扬声器阵列可以将多个声场单元以某些方式相组合而进行扩声。和 单个扬声器直接辐射相比，扬声器阵列可以提升声功率， 增加低频时声辐射效 率，提高指向性以改善声场不均勾度并且提高混响环境中的语音清晰度。扬声 器阵列可以使用波前合成技术。
麦克风阵列 A3可以是线性麦克风阵列或者圆形麦克风阵列， 放置在桌子 上， 或者采用吊顶方式放置。 扬声器阵列 A11包含多个扬声器， 和暗箱 A5集成 在一起， 其分布方向和投影屏幕 A5上显示的远端参会者的分布方向一致。 在 会议过程中，通过远端 B的麦克风阵列 B3,音视频通信设备 B8能够检测出正在 说话的参会者 B15的位置信息， 并将远端 B的音频信号和参会者 B15的位置信 息传输到本地 A的音视频通信设备 A8。 扬声器阵列 A11可以根据输入的音频信 号和位置信息进行声场重建和再现。 这样， 本地用户 A15就可以感觉远端用户 B15的声音恰好是从屏幕上 B15的位置上发出的， 因而能够获得类似于面对面 交谈的体验。
参见图 6, 显示了实施例 1的系统工作流程图， 示出了远端 B向本地 A传输 音视频信息的工作流程。
在远端 B, 全景摄像机 B7 (由多个摄像机组成）在同步单元 B1 6的控制下 同步地采集不同视角的场景的图像。采集的多个图像发送到拼接单元中进行图 像拼接， 得到一个远端场景 B的全景图像。 该全景图像从拼接单元中输出， 经 过图像处理后输出到视频编码单元 1进行编码， 以分组码流的形式通过分组网 络进行传输。 需要注意的是， 拼接后图像的分辨率可能很大， 一个视频编码单 元 1无法进行实时编码， 因此可能需要将拼接后的图像分割成多份， 同步地输 出到多个视频编码器进行编码。编码后可能形成一个或多个码流通过分组网络 传输。 由于分布式编码以及网络传输时的延时和抖动因素， 多个码流有可能变 得不同步， 因此可能需要在码流中做一些标记（例如时间戳）， 在解码端根据 这些标记对多码流进行同步。 类似地， 远端 B的麦克风阵列采集场景的音频信 号， 通过音频编码单元进行编码， 形成音频编码码流， 以数据分组方式通过网 络进行传输。 为了避免音频和视频出现不同步的现象， 优选地， 音频数据和视 频数据可进行唇音同步，唇音同步为音视频领域的现有技术，在此不再进行赞 述。 由于远端 B采用了麦克风阵列进行音频信号的采集， 通过音频定位算法还 可以计算出正在说话的参会者的位置信息，该位置信息也可以通过网络传输到 本地。 除全景摄像机 B7外，还有一个或多个对象摄像机 B10拍摄场景中需要呈 现的对象。 如果是多个对象摄像机 B10 , 则可以构成立体摄像机， 获得场景的 3D图像，此时多个摄像机之间也需要使用同步单元 B16进行同步采集。对象摄 像机 B10的一个或多个视频流输入到视频编码单元 2进行编码， 该视频编码单 元 2支持 2D/3D视频编码格式， 编码后的码流数据也通过分组网络进行传输。
在本地 A, 视频解码单元 1接收到远端 B的全景视频码流分组进行解码。 由 于全景图像的分辨率可能很大， 一个视频解码单元 1无法完成解码， 因此可能 需要多个视频解码单元并行工作。在解码时需要根据码流中的同步标记确定视 频图像帧播放的顺序。解码后图像可能是一个完整的全景图像， 也可能是分割 后的多个图像。 如果是完整的全景图像， 还需要将该图像进行分割， 同步输出 到多个投影仪 A9。 多个投影仪 A9将图像以无缝的方式在投影屏幕 A4上呈现。 在投影仪进行呈现之前， 由于投影仪之存在位置、 亮度和色度等的差异， 因此 优选地， 可采用校正 /融合单元对图像进行几何校正， 并采用亮度 /色度融合的 方法消除相邻图像间的接缝。音频数据码流被音频解码单元解码后形成音频数 据信号， 输出到扬声器阵列。 根据远端 B参会者的位置信息， 扬声器阵列可以 选择距离投影屏幕 A4上显示的远端参会者位置最近的一个或多个扬声器呈现 远端参会者的声音。对端 B的对象摄像机 B10的视频码流被视频解码单元 2解码 后， 通过辅助显示器 A12进行呈现。如果辅助显示器 A12支持 3D视频， 则以 3D 方式进行呈现， 如果辅助显示器 A12只支持 2D方式， 则以 2D方式进行呈现。 可见， 完整的音视频通信设备 A8或 B8包括了拼接单元、 视频编码单元 1、 2、 音频编码单元、 视频解码单元 1、 2以及音频解码等单元。
参见图 7、 8, 为本发明 Telepresence系统实施例 1会议室布局俯视图 2和 示意图 2,该方案是使用前投技术。可以将投影仪 A9放置在投影屏幕 A4的前方， 吊在会议桌 A2的上方 （图中所示）或放置在会议桌的下方进行投影。 这种方 案的好处是投影仪 A9发出的光线不会给用户造成干扰。
参见图 9、 图 10 , 显示了本发明实施例 1的一个筒化方案的会议室布局俯 视图 3和示意图 3。 在该筒化方案中， 全景摄像机 A7被放置到投影屏幕 A4上方 获取场景的全景图像。 在这种设置下， 不需要同步单元 A16来进行全景摄像机 A7和投影仪 A9的采集和投影同步控制， 这样可以筒化全景摄像机 A7和投影仪 A9的设计和降低成本。 不过由于全景摄像机 7没有放置在用户 A15的视线位置 上， 因此垂直眼对眼的效果会变差。 一般来说， 如果垂直视角差< 7° , 则垂直 目艮对眼的效果可以接受。 为了减少垂直视角， 可以将全景摄像机 A7的反光镜 放置在下方， 彩色摄像机放置在上方。
此外， 在图 9、 10所示的筒化方案中， 投影屏幕 A4可以采用全息透明投影 屏幕或普通的投影屏幕， 如果采用全息透明投影屏幕， 用户可以感觉到场景的 深度层次； 如果采用普通背投， 则无法呈现出场景的深度感， 用户背后的背景 幕 A13和暗箱 A5的背景幕 A6可以不用进行特殊装饰。
可见， 在实施例 1介绍的 3种方案中， 均采用了全景摄像机（A7或 B7 ), 由 此， 可拍摄到本地会议室的全景， 为对端参会者提供会议全景， 由此解决了目 前 Telepresence系统存在的全景呈现效果不好的问题。
而且， 不论投影屏幕 A4 ( B4 )采用普通投影屏幕或采用全息透明投影屏 幕，都是对图像进行一体式呈现， 由此克服了多个平板电视组合呈现时电视边 框导致的缺陷， 实现了无缝呈现。
并且， 优选地， 投影屏幕 A4 ( B4 )通过采用全息透明投影屏幕， 可为参 会者提供深度呈现的感觉。
另外， 优选地， 通过同步单元八16 ( 81 6 )的控制， 使全景摄像机 A7 ( B7 ) 拍摄本地 A (远端 B ) 图像时不受投影仪 A9 ( B9 )投影图像的影响， 由此可避 免垂直视角的产生， 从而可使对端参会者获得良好的目艮神交流享受。 当然， 还 可采用全景摄像机 A7 ( B7 )被放置到投影屏幕 A4 ( B4 )上方获取场景的全景 图像。 在这种设置下， 可省略同步单元 A16 ( B16 ), 从而筒化了设计并降低了 成本， 此时， 为了减少垂直视角， 可以将全景摄像机 A7 ( B7 ) 的反光镜放置 在下方， 彩色摄像机放置在上方。 此外， 优选地， 将投影仪 A9 ( B9 )设置在暗箱 A5 ( B5 ) 中， 可使投影仪 A9免受环境光照的影响。 此外， 也可采用前投技术将投影仪 A9 ( B9 )放置在 投影屏幕 A4 ( B4 ) 的前方、 吊在会议桌的上方或放置在下方进行投影， 这种 方案的好处是在采用全息透明投影时， 投影仪 A9 ( B9 ) 的光线不会给用户造 成干扰。
下面介绍本发明实施例 2。
参见图 11 , 为本发明 Telepresence系统实施例 2会议室布局俯视图。 参 见图 12 , 为与图 11对应的本发明 Telepresence系统实施例 2示意图 1。 在 实施例 2中， 去掉了暗箱设计， 将投影屏幕 4直接安放在座椅 1 A, 1 B, 1 C 的对面。 在实际设计时， 可以将该投影屏幕设计成可升降式结构， 不使用时将 投影屏幕 4升上去， 或是隐藏到桌子 2里面； 使用的时候将投影屏幕 4降下 来，或是再从桌子 2里面升出来。这样做的好处是在不开视频会议时还可以用 该桌子 2召开普通会议。 因为投影屏幕 4很薄， 因此扬声器可以采用很薄的平 板扬声器，放置在投影屏幕 4的下方； 或者采用振动模块直接贴在屏幕上， 使 屏幕成为扬声器。为了获得较好的垂直眼对眼效果，全景摄像机 7可以隐藏到 投影屏幕 4后面的背景墙壁 14里， 通过特殊的装饰使全景摄像机 7和背景墙 壁 14融为一体。 投影仪 9和全景摄像机 7需要进行同步， 同步方式和实施例 1 所述相同， 仍采用同步单元 A16进行同步控制。 该实施实例中的投影仪 9 可以采用前投方式（图 11 中所示）或背投方式。
实施例 2的另一种设计方案是将全景摄像机 7放置在投影屏幕 A的上方， 参 见图 13 , 为本发明 Telepresence系统实施例 2示意图 2。 在具体应用时， 可以 设计一个具有一定厚度的用于支撑的边框， 将投影屏幕 A4嵌入到边框中， 然 后将全景摄像机 A7放置在边框的前面或嵌入到边框里面。 在这种设置下， 不 需要同步单元来进行全景摄像机 A7和投影仪 A9的采集和投影同步控制。 不过 由于全景摄像机 7没有放置在用户 A15的视线位置上， 因此垂直眼对眼的效果 会变差。 一般来说， 如果垂直视角差< 7° , 则垂直眼对眼的效果可以接受。 为 了减少垂直视角， 可以将全景摄像机 7的反光镜放置在下方， 彩色摄像机放置 在上方。
在实施例 2中， 参看图 11 , 由于将投影屏幕 4直接安放在座椅的对面， 在 不使用时可收回投影屏幕， 因此该方案可实现与传统会议的兼容， 即在不需要 召开 Telepresence会议时将相关的设备隐藏， 将会议室用于召开普通会议。
下面介绍本发明实施例 3。
参见图 14, 为本发明 Telepresence系统实施例 3会议室布局俯视图， 参 见图 15为本发明 Telepresence系统实施例 3示意图。 实施例 3对方案进一 步进行了筒化，不使用投影屏幕而是使用多块大尺寸的电视 30A, 30B和 30C 进行拼接构成显示系统。 所用电视可以是 LCD电视、 PDP电视或 DLP背投 电视。 全景摄像机 A7放置在显示器的上方拍摄本地场景。 因为采用了电视呈 现， 无法得到深度效果， 因此用户背后的背景幕 A13和暗箱的背景幕 A6可以 不用进行特殊装饰。
可见， 实施例 3在现有 Telepresence系统上升级， 只需更换普通摄像机 为全景摄像机， 即可解决全景呈现效果不好的问题。
下面介绍本发明实施例 4。
参见图 16, 为本发明 Telepresence系统实施例 4示意图， 该实施例中， 使用半反半透镜实现深度呈现。全息透明投影屏幕被取消，取而代之的是一块 半反半透镜 A21。 该半反半透镜加装在暗箱的前面， 投影屏幕 A22位于半反 半透镜的斜上方， 和半反半透镜呈一定角度放置。 投影屏幕 A22采用背投方 式成像。投影仪 A9投出的图像首先经过反射镜 A23的反射后在投影屏幕 A22 处成像， A22处的图像通过半反半透镜 A21形成虚像 A101 , 使本地用户 A15 感觉在某个深度上看到了图像。全景摄像机 A7通过半反半透镜 A21采集用户 的图像， 并和暗箱 A5的背景 A6融为一体。 远端用户后面是一个暗背景， 使 本地特殊装饰的背景 A6能够透过除了远端用户身体明亮区域之外的暗区域被 本地用户 A15看到。 由于本地用户看到的远端用户图像 A1 01和本地背景 A6 之间有一个物理距离， 因此本地用户看到的远端用户图像好像在背景前面一 样。 需要说明的是， 反射镜 A23是可选的， 在不使用反射镜 A23时， 投影仪 A9可以采用前投方案。
可见， 实施例 4在采用全景摄像机 A7实现全景呈现以及使用投影实现无 缝显示问题的基础上， 采用半反半透镜 A21可解决深度呈现、 目艮神交流问题。 下面介绍本发明实施例 5。
参见图 17, 为本发明 Telepresence系统实施例 5示意图。 该实施例采用 了一个透明的光传导元件 A25,该元件有一个光重定向区域 A25,用于输入图 像。 A24作为一个波导设备， 完全通过内反射的方式将参会者 A15的图像传 导到位于暗箱 A5底部的全景摄像机 A7位置。在图中可以看到，入射光线 A102 在光导元件的两个内表面 A26和 A27之间进行了多次内反射，最后在暗箱 A5 底部作为出射光线 A1 03射出， 被全景摄像机 A7采集。 该光传导元件放置在 投影屏幕 A4前面， 输入区域 A25覆盖了投影屏幕 A4表面。 A25必须是足够 透明的， 以便不给用户 A15带来干扰。 输入区域的内表面 A26和 A27可以采 用全息光栅（holographically derived grating ) 实现， 元件采用玻璃或塑料组 成的透明面板。
可见， 实施例 5在采用全景摄像机 A7实现全景呈现、 以及通过投影屏幕 A4实现无缝显示的基础上， 通过光传导元件 A25实现眼神交流， 并且， 优选 地， 投影屏幕 A4采用全息透明投影屏幕， 可实现深度呈现的效果。 下面介绍本发明实施例 6。 该实施例采用偏光片的方式实现支持垂直眼对眼效果的全景 Tslsprsssncs系统。
为了理解顺畅， 首先筒要介绍公知的偏振光原理。
光波是横波， 即光波矢量的振动方向垂直于光的传播方向。 通常， 光源发 出的光波， 其光波矢量的振动在垂直于光的传播方向上作无规则取向，但统计 平均来说， 在空间所有可能的方向上， 光波矢量的分布可看作是机会均等的， 它们的总和与光的传播方向是对称的， 即光矢量具有轴对称性、 均勾分布、 各 方向振动的振幅相同， 这种光就称为自然光。 偏振光是指光矢量的振动方向 不变， 或具有某种规则地变化的光波。 按照其性质,偏振光又可分为平面偏振 光（线偏光）、 圆偏振光和椭圆偏振光、 部分偏振光几种。 如果光波电矢量的 振动方向只局限在一确定的平面内， 则这种偏振光称为平面偏振光， 若轨迹在 传播过程中为一直线，故又称线偏振光。如果光波电矢量随时间作有规则地改 变， 即电矢量末端轨迹在垂直轨迹在传播过程中为一直线， 故又称线偏振光。 如果光波电矢量随时间作有规则地改变，即电矢量末端轨迹在垂直于传播方向 的平面上呈圆形或椭圆形,则称为圆偏振光或椭圆偏振光。 如果光波电矢量的 振动在传播过程中只是在某一确定的方向上占有相对优势，这种偏振光就称为 部分偏振光。偏振片是用人工方法制成的薄膜，是用特殊方法使选择性吸收很 强的微粒晶体在透明胶层中作有规则排列而制成的，它允许透过某一电矢量振 动方向的光（此方向称为偏振化方向）， 而吸收与其垂直振动的光， 即具有二 向色性。
参见图 18,为本发明 Telepresence系统实施例 6示意图。在该实施例中， 投影仪 A9和全景摄像机 A7的镜头前放置了线性偏光片， 而且投影仪 A9的 线性偏光片的偏光角度和全景摄像机 A7的线性偏光片的偏光角度有一个差值 (因为全景摄像机前面的偏振片的偏振方向和投影仪所投出的光的偏振方向 不同， 具有一个差值， 根据偏振光的原理， 此时投影仪投出的光无法通过全景 摄像机前面的偏振片进入摄像机中， 因此存在此差值。）， 在理想的情况下， 该 差值为 90度，即投影仪 A9的偏光方向和全景摄像机 A7的偏光方向是垂直的。 例如， 投影仪 A9的偏光方向是垂直的， 而全景摄像机 A7的偏光方向是水平 的。 如果摄像机前面不加偏光片， 可以采用特殊材料制作的投影幕 A4, 该投 影幕为由全景摄像机 A7的偏振片材料 A41和其它材料 A42交织而成的一个 半透明幕。 这样， 输入场景的圆偏振光， 在经过投影屏幕 A4后， 变成了水平 线偏振光， 能够被全景摄像机 A7采集； 而投影仪 A9投出的光线是垂直线偏 振光， 其无法通过摄像机 A7的水平线偏振片， 因而不会被摄像机 A7采集到。 这样全景摄像机 A7的拍摄和投影仪 A9的投影就不会产生干扰。
可见， 实施例 6在采用全景摄像机 A7实现全景呈现、 以及通过投影屏幕 A4实现无缝显示的基础上， 通过在摄像机和投影仪前面加装偏光片实现眼神 交流。
下面介绍本发明实施例 7。
实施例 7旨在解决之前方案中用户后面暗背景的布置问题。在之前的方案 中， 若要实现深度感呈现， 用户后面的背景需要被布置成固定的黑色背景， 如 采用黑色的幕布， 或将墙面涂成黑色等。这种背景布置方式在某些会议室中可 能无法被用户接受，例如用户觉得在身后放置一个黑色背景和会议室装修方案 设计并不协调。 参见图 19 , 为本发明 Telepresence系统实施例 7示意图 1。 采用一个背 景投影仪 A50,将需要显示的用户背景投影在一个纯黑色的投影幕布 A13上。 该背景投影仪 A50连接到同步单元 A1 6上，该同步单元 A16协调全景摄像机 A7的采集和背景投影仪 A50的投影。这样系统的时分工作方式分为两种模式， 背景投影模式和摄像机采集模式。 在背景投影模式下， 背景投影仪 A50将背 景投影到黑色幕布 A13上，此时全景摄像机 A7处于未激活模式，不进行采集； 在摄像机采集模式下， 背景投影仪 A50处于未激活模式， 不进行投影， 而全 景摄像机 A7拍摄场景。 此时全景摄像机 A7拍摄到的用户 A15的背景是一个 黑色的背景。 只要切换的速度足够快， 用户看到的本地背景就不是黑色的， 而 是背景投影仪 A50投出的图像。 该图像是可更换的， 因此可以和会议室装修 方案协调一致。
参见图 20 , 为本发明 Telepresence系统实施例 7示意图 2。 在图 20中， 在 用于投影背景的背景投影仪 A50的前面加装线性偏振片， 这样投影到背景墙上 的光线是线偏振光。 在全景摄像机 A7前面也加上一个线性偏振片， 其偏振的 角度和背景投影仪 A50前面的偏振片的偏振角度垂直。 这样背景投影仪 A50投 出的背景光无法被全景摄像机 A7采集， 而前景的人反射的会议室灯光是圆偏 振的， 能够被摄像机 A7拍到。 这样人后面的背景在拍摄的图像上就是黑色的， 从而解决了暗背景问题。
参见图 21 , 为本发明 Telepresence系统实施例 7示意图 3。 在图 21中， 用 户背后采用一个大尺寸的背景显示器 A51显示背景图像。 该背景显示器 A51连 接到同步单元 A1 6上， 该同步单元 A16协调全景摄像机 A7的采集和背景显示器 A51的显示。 系统的时分工作方式分为两种模式， 背景显示模式和摄像机采集 模式。 在背景显示模式下， 背景显示器 A51显示正常的背景图像， 此时全景摄 像机 A7处于未激活模式， 不进行采集； 在摄像机采集模式下， 背景显示器 A51 显示一个纯黑色的背景图像， 而全景摄像机 A7进行拍摄。 此时全景摄像机 A7 拍摄到的用户 A15的背景是一个黑色的背景。 只要切换的速度足够快， 用户看 到的 A51显示的图像就不是黑色的。
参见图 22, 为本发明 Telepresence系统实施例 7示意图 4。 在图 22中， 在 人的背后放置大的背景显示器 A51并且在该背景显示器 A51的前面以及全景摄 像机 A7的前面分别加装线性偏振片，如果背景显示器 A51采用 LCD显示器， 则 由于 LCD发出的背景光本身就是偏振光， 因此可以仅在全景摄像机 A7前面加 上一个线性偏振片， 其偏振的角度和背景显示器 A51发出的背景偏振光的偏振 角度垂直。 这样背景显示器 A51的背景光就无法被全景摄像机 A7采集， 而前景 的人反射的是会议室灯光是圆偏振的， 能够被全景摄像机 A7拍到。 这样人后 面的背景在拍摄的图像上就是黑色的， 从而解决了暗背景问题。
可见， 实施例 7通过背景投影仪 A50或者背景显示器 A51 , 将需要显示的 用户背景投影在一个纯黑色的投影幕布 A13上，从而解决用户后面暗背景的布 置问题。 该实施例 7可与实施例 1 ~6结合应用。 综上所述，本发明实施例在现有 Telepresence系统上升级，只需更换普通 摄像机为全景摄像机， 即可拍摄到本地会议室的全景， 为对端参会者提供会议 全景， 由此解决了目前 Telepresence系统存在的全景呈现效果不好的问题， 并 与现有 Telepresence系统容易实现兼容。
优选地， 采用普通投影屏幕或全息透明投影屏幕， 都是对全景图像进行一 体式呈现， 由此克服了多个平板电视组合呈现的缺陷， 实现了无缝呈现。
优选地，通过采用全息透明投影屏幕和半反半透镜， 可为参会者提供深度 呈现的感觉。
优选地，通过同步单元的控制，使全景摄像机拍摄本地图像时不受投影仪 投影图像的影响，由此可避免因摄像机无法摆放在用户视线位置上而导致的视 差， 从而可使对端参会者获得良好的眼神交流享受。 另外， 也可采用半反半透 镜或者光传导元件或者线性偏振片的方案实现眼神交流。
优选地，通过布置特殊的暗背景，使用背景投影仪或者背景显示器的方式， 在用户后面布置一个暗的背景，使用户图像和背景图像分离显示，从而实现具 有深度感的呈现效果。
除了上述介绍的远程呈现系统， 本发明实施例还提供一种远程呈现方法， 参见图 23 , 该方法实施例包括：
S2301 : 获取本地的全景图像以及音频， 其中， 通过全景摄像机拍摄不同 视角的图像，并通过一拼接单元将所述全景摄像机拍摄到的不同视角的低分辨 率的图像拼接为高分辨率的全景图像； S2302:通过网络将本地的全景图像以及音频传输给远端分别进行显示和 播放。
其中， 全景摄像机采用平面反射镜虚拟共光心模式、 汇聚多摄像机模式或 密集摄像机阵列模式进行拍摄。
优选地， 按照时间顺序交替采集图像以及音频。 优选地， 该方法还包括： 通过立体摄像机采集本地的三维视频，并通过网络传输给远端通过辅助显示设 备进行显示。 优选地， 在显示全景图像之前， 还包括： 将全景图像进行几何校 正和边缘融合。优选地，该方法还可包括以下步骤：接收远端参会者位置信息； 根据所述位置信息对接收到的音频进行声场的重建和再现。优选地， 该方法还 可包括以下步骤： 对本地获取的音频数据和视频数据进行唇音同步。
此外， 本发明实施例还提供一种远程呈现系统中的视频采集设备。
该视频采集设备与所述远程呈现系统中的视频显示设备、 音频采集设备、 音频播放设备以及音视频通信设备结合工作， 其中， 所述音视频通信设备将本 地视频采集设备采集的图像以及音频采集设备采集的音频通过网络传输给远 端， 由远端的视频显示设备和音频播放设备分别对图像和音频进行显示和播 放， 与现有技术相比， 本发明实施例中的视频采集设备为全景摄像机， 所述全 景摄像机拍摄到的不同视角的低分辨率的图像通过一拼接单元拼接为高分辨 率的全景图像。
其中， 所述拼接单元是独立的设备， 或是作为全景摄像机的一部分， 或是 作为所述音视频通信设备的一部分。其中， 所述全景摄像机采用平面反射镜虚 拟共光心模式、 汇聚多摄像机模式或密集摄像机阵列模式进行拍摄。 以上所述仅为本发明的较佳实施例而已， 并不用以限制本发明， 凡在本发 明的精神和原则之内， 所做的任何修改、 等同替换、 改进等， 均应包含在本发 明保护的范围之内。

Claims

1、 一种远程呈现系统， 包括视频采集设备、 视频显示设备、 音频采集设 备、音频播放设备以及音视频通信设备， 所述音视频通信设备将本地视频采集 设备采集的图像以及音频采集设备采集的音频通过网络传输给远端，由远端的 视频显示设备和音频播放设备分别对图像和音频进行显示和播放， 其特征在 于， 所述视频采集设备为全景摄权像机； 所述系统还包括拼接单元， 所述拼接单 元用于将所述全景摄像机拍摄到的不同视角的低分辨率的图像拼接为高分辨 利
率的全景图像。 2
1
2、 根据权利要求 1所述系统， 其特要征在于， 所述视频显示设备包括至少 一个平板显示器；或者，所述视频显示设备包括求投影屏幕以及至少一个投影仪， 所述投影屏幕为非透明投影屏幕， 或半透明投影屏幕， 或透明投影屏幕。
3、 根据权利要求 2所述系统， 其特征在于， 所述投影仪放置在投影屏幕 的前方或者后方。
4、 根据权利要求 2所述系统， 其特征在于， 所述投影屏幕作为一个暗箱 的前表面， 放置在参会者对面；
或者， 所述投影屏幕布置在一个会议桌上， 位于参会者对面， 并且可以自 动或手动的方式升起或下降到会议桌内部。
5、 根据权利要求 2所述系统， 其特征在于， 当所述投影屏幕为透明投影 屏幕时， 所述全景摄像机放置于所述投影屏幕上方； 或者， 放置于所述投影屏 幕的后方且位于参会者视线的同等或近似高度上。
6、 根据权利要求 5所述系统， 其特征在于， 当所述全景摄像机放置于所 述透明投影屏幕后方时， 所述系统还包括同步单元， 所述同步单元输出同步信 号控制所述全景摄像机和所述投影仪交替工作。
7、 根据权利要求 2所述系统， 其特征在于， 当所述视频显示设备为平板 显示器或非透明投影屏幕或半透明投影屏幕时，所述全景摄像机放置于所述投 影屏幕的上方。
8、 根据权利要求 2所述系统， 其特征在于， 所述系统还包括至少一个半 反半透镜， 该半反半透镜加装在所述系统的暗箱的前面， 所述投影屏幕位于半 反半透镜的斜上方， 和该半反半透镜呈一定角度放置， 该半反半透镜形成所述 显示设备所呈现全景图像的虚像。
9、 根据权利要求 2所述系统， 其特征在于， 当所述投影屏幕为透明投影 屏幕时， 或者， 在所述系统加装了半反半透镜时， 在所述投影屏幕后面或者半 反半透镜形成的虚像后面布置一个具有深度提示效果的背景，并且在所述参会 者的后面布置一个暗背景。
10、根据权利要求 8所述系统， 其特征在于， 当所述投影屏幕为透明投影 屏幕时， 或者， 在所述系统加装了半反半透镜时， 在所述投影屏幕后面或者半 反半透镜形成的虚像后面布置一个具有深度提示效果的背景，并且在所述参会 者的后面布置一个暗背景。
11、根据权利要求 2所述系统， 其特征在于， 当所述视频显示设备包括投 影屏幕以及至少一个投影仪时，所述系统还包括放置于所述投影屏幕靠近参会 者一侧前面的透明光传导元件。
12、根据权利要求 2所述系统， 其特征在于， 当所述视频显示设备包括投 影屏幕以及至少一个投影仪时，所述系统还包括放置于所述投影仪和全景摄像 机镜头前面的线性偏光片；所述投影仪偏振片的偏振角度和所述全景摄像机偏 振片的偏振角度具有一个差值。
13、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括黑色的背景幕和背景投影仪，所述背景投影仪与所述全景摄像机在所述系 统的同步单元的控制下交替工作。
14、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括背景投影仪，并在所述背景投影仪和所述全景摄像机镜头前面分别加装线 性偏振片，所述背景投影仪偏振片的偏振角度和所述全景摄像机偏振片的偏振 角度具有一个差值。
15、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括放置于参会者后方的背景显示器，所述背景显示器与所述全景摄像机在同 步单元的控制下交替工作。
16、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括放置于参会者后方的背景显示器，并在所述背景显示器和所述全景摄像机 镜头前面分别加装线性偏振片，所述背景显示器的偏振角度和所述全景摄像机 偏振片的偏振角度具有一个差值；
或者， 所述系统还包括放置于参会者后方的液晶显示器， 在所述全景摄像 机镜头前面加装线性偏振片，所述液晶显示器发出光的偏振角度和所述全景摄 像机偏振片的偏振角度具有一个差值。
17、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括至少一个立体摄像机和辅助显示设备，所述音视频通信设备将本地立体摄 像机采集的三维视频通过网络传输给远端的辅助显示设备显示。
18、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述音频采 集设备为至少一个麦克风阵列， 所述音频播放设备为至少一个扬声器阵列。
19、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述全景摄 像机采用平面反射镜虚拟共光心模式、汇聚多摄像机模式或密集摄像机阵列模 式进行拍摄。
20、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述拼接单 元是独立的设备， 或是作为全景摄像机的一部分， 或是作为所述音视频通信设 备的一部分。
21、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述系统还 包括图像校正 /融合单元， 本地的图像校正 /融合单元用于将远端传输过来的图 像进行几何校正和边缘融合，经过几何校正和边缘融合的图像经本地的视频显 示设备进行显示。
22、 根据权利要求 21所述系统， 其特征在于， 当所述显示设备包括两个 及以上投影仪时， 所述图像校正 /融合单元还用于将全景图像分割为与所述投 影仪数量相当个的图像， 并将分割并处理后的图像分别输出给所述投影仪显 示。
23、 根据权利要求 1至 12任意一项所述系统， 其特征在于， 所述音视频 通信设备接收远端音视频通信设备传输过来的远端参会者位置信息，该位置信 息由远端音视频通信设备通过远端音频采集设备检测得到；
所述音频播放设备根据所述位置信息对接收到的音频进行声场的重建和 再现。
24、 一种远程呈现方法， 其特征在于， 包括：
获取本地的全景图像以及音频， 其中， 通过全景摄像机拍摄不同视角的图 像，并通过一拼接单元将所述全景摄像机拍摄到的不同视角的低分辨率的图像 拼接为高分辨率的全景图像；
通过网络将本地的全景图像以及音频传输给远端分别进行显示和播放。
25、 根据权利要求 24所述方法， 其特征在于， 所述采集本地的图像以及 音频是按照时间顺序交替进行的。
26、 根据权利要求 24所述方法， 其特征在于， 还包括： 通过立体摄像机 采集本地的三维视频， 并通过网络传输给远端通过辅助显示设备进行显示。
27、 根据权利要求 24所述方法， 其特征在于， 所述全景摄像机采用平面 反射镜虚拟共光心模式、 汇聚多摄像机模式或密集摄像机阵列模式进行拍摄。
28、 根据权利要求 24所述方法， 其特征在于， 在显示全景图像之前， 还 包括：
将全景图像进行几何校正和边缘融合。
29、 根据权利要求 24所述方法， 其特征在于， 还包括：
接收远端参会者位置信息；
根据所述位置信息对接收到的音频进行声场的重建和再现。
30、 根据权利要求 24所述方法， 其特征在于， 还包括：
对本地获取的音频数据和视频数据进行唇音同步。
31、一种远程呈现系统中的视频采集设备，所述远程呈现系统还包括视频 显示设备、 音频采集设备、 音频播放设备以及音视频通信设备， 所述音视频通 信设备将本地视频采集设备采集的图像以及音频采集设备采集的音频通过网 络传输给远端，由远端的视频显示设备和音频播放设备分别对图像和音频进行 显示和播放， 其特征在于， 所述视频采集设备为全景摄像机， 所述全景摄像机 拍摄到的不同视角的低分辨率的图像通过一拼接单元拼接为高分辨率的全景 图像。
32、 根据权利要求 31所述视频采集设备， 其特征在于， 所述拼接单元是 独立的设备，或是作为全景摄像机的一部分，或是作为所述音视频通信设备的 一部分。 行拍摄。
PCT/CN2009/071745 2009-05-12 2009-05-12 远程呈现系统、方法及视频采集设备 WO2010130084A1 (zh)

Priority Applications (7)

Application Number Priority Date Filing Date Title
EP09842025A EP2290969A4 (en) 2009-05-12 2009-05-12 TELEPRESENCE SYSTEM, METHOD AND VIDEO CAPTURE DEVICE
PCT/CN2009/071745 WO2010130084A1 (zh) 2009-05-12 2009-05-12 远程呈现系统、方法及视频采集设备
RU2011130860/07A RU2518218C2 (ru) 2009-05-12 2009-05-12 Система дистанционного присутствия, способ дистанционного присутствия и устройство сбора видеоданных
BRPI0924076-4A BRPI0924076B1 (pt) 2009-05-12 Sistema de telepresença e método de telepresença
CN2009801168492A CN102084650B (zh) 2009-05-12 2009-05-12 远程呈现系统、方法及视频采集设备
US12/888,769 US8692861B2 (en) 2009-05-12 2010-09-23 Telepresence system, telepresence method, and video collection device
US13/716,508 US8836750B2 (en) 2009-05-12 2012-12-17 Telepresence system, telepresence method, and video collection device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2009/071745 WO2010130084A1 (zh) 2009-05-12 2009-05-12 远程呈现系统、方法及视频采集设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/888,769 Continuation US8692861B2 (en) 2009-05-12 2010-09-23 Telepresence system, telepresence method, and video collection device

Publications (1)

Publication Number Publication Date
WO2010130084A1 true WO2010130084A1 (zh) 2010-11-18

Family

ID=43084602

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2009/071745 WO2010130084A1 (zh) 2009-05-12 2009-05-12 远程呈现系统、方法及视频采集设备

Country Status (5)

Country Link
US (2) US8692861B2 (zh)
EP (1) EP2290969A4 (zh)
CN (1) CN102084650B (zh)
RU (1) RU2518218C2 (zh)
WO (1) WO2010130084A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102685445A (zh) * 2012-04-27 2012-09-19 华为技术有限公司 网真视频图像输送方法、设备及网真系统
CN102761763A (zh) * 2012-06-26 2012-10-31 中国航空工业第六一八研究所 一种不少于四通道的主动立体视景装置
CN103843324A (zh) * 2011-06-15 2014-06-04 梅巴菲科姆公司 使用倒置望远镜摄像机的视频会议系统
US9143724B2 (en) 2011-07-06 2015-09-22 Hewlett-Packard Development Company, L.P. Telepresence portal system
CN105578199A (zh) * 2016-02-22 2016-05-11 北京佰才邦技术有限公司 虚拟现实全景多媒体处理系统、方法及客户端设备
CN105681681A (zh) * 2016-01-16 2016-06-15 深圳算云科技有限公司 一种多视频流的视频压缩方法及系统
WO2017120308A1 (en) * 2016-01-05 2017-07-13 360fly, Inc. Dynamic adjustment of exposure in panoramic video content

Families Citing this family (91)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100306825A1 (en) 2009-05-27 2010-12-02 Lucid Ventures, Inc. System and method for facilitating user interaction with a simulated object associated with a physical location
US8694553B2 (en) 2010-06-07 2014-04-08 Gary Stephen Shuster Creation and use of virtual places
CN102790872B (zh) * 2011-05-20 2016-11-16 南京中兴软件有限责任公司 一种视频会议的实现方法及系统
US8698874B2 (en) * 2011-06-10 2014-04-15 Microsoft Corporation Techniques for multiple video source stitching in a conference room
US9477141B2 (en) 2011-08-31 2016-10-25 Cablecam, Llc Aerial movement system having multiple payloads
US10469790B2 (en) * 2011-08-31 2019-11-05 Cablecam, Llc Control system and method for an aerially moved payload system
US9337949B2 (en) 2011-08-31 2016-05-10 Cablecam, Llc Control system for an aerially moved payload
US9250510B2 (en) * 2012-02-15 2016-02-02 City University Of Hong Kong Panoramic stereo catadioptric imaging
CN104350740B (zh) * 2012-03-19 2018-04-20 索尼移动通信株式会社 利用无线外围视频会议装置进行视频会议
US20150070462A1 (en) * 2012-03-28 2015-03-12 Nokia Corporation Method, Apparatus and Computer Program Product for Generating Panorama Images
US11330171B1 (en) * 2012-05-25 2022-05-10 Gn Audio A/S Locally adaptive luminance and chrominance blending in a multiple imager video system
US8804321B2 (en) 2012-05-25 2014-08-12 Steelcase, Inc. Work and videoconference assembly
CN102761732B (zh) * 2012-07-25 2018-04-27 鲁懿齐 一种视频会议目光对视交流系统
US9232310B2 (en) * 2012-10-15 2016-01-05 Nokia Technologies Oy Methods, apparatuses and computer program products for facilitating directional audio capture with multiple microphones
CN103034330B (zh) * 2012-12-06 2015-08-12 中国科学院计算技术研究所 一种用于视频会议的眼神交互方法及系统
CN103905668B (zh) * 2012-12-27 2016-12-28 中强光电股份有限公司 具有视讯功能的电话机及其视讯会谈的方法
CN105142447B (zh) * 2013-01-16 2017-03-08 范德维德股份有限公司 试衣间镜子
US9325943B2 (en) 2013-02-20 2016-04-26 Microsoft Technology Licensing, Llc Providing a tele-immersive experience using a mirror metaphor
WO2015017304A1 (en) * 2013-07-30 2015-02-05 Kodak Alaris Inc. System and method for creating navigable views of ordered images
US9343043B2 (en) * 2013-08-01 2016-05-17 Google Inc. Methods and apparatus for generating composite images
US9426300B2 (en) 2013-09-27 2016-08-23 Dolby Laboratories Licensing Corporation Matching reverberation in teleconferencing environments
CN103546672B (zh) * 2013-11-07 2016-09-07 苏州君立软件有限公司 一种图像采集系统
KR20150068298A (ko) * 2013-12-09 2015-06-19 씨제이씨지브이 주식회사 다면 영상 생성 방법 및 시스템
CN103716541B (zh) * 2013-12-23 2019-08-02 宇龙计算机通信科技(深圳)有限公司 一种拍摄大分辨率照片的方法及终端
CN104765636B (zh) * 2014-01-02 2018-05-04 华为技术有限公司 一种远程桌面图像的合成方法和装置
EP3109744B1 (en) * 2014-02-17 2020-09-09 Sony Corporation Information processing device, information processing method and program
US9883138B2 (en) 2014-02-26 2018-01-30 Microsoft Technology Licensing, Llc Telepresence experience
JP6485711B2 (ja) * 2014-04-16 2019-03-20 ソニー株式会社 音場再現装置および方法、並びにプログラム
CN104023203B (zh) * 2014-04-17 2017-10-27 深圳英飞拓科技股份有限公司 一种基于摄像机的图像生成方法和装置
CN107077059A (zh) 2014-07-22 2017-08-18 巴科公司 采用偏振反射屏幕的显示系统和方法
KR20170088329A (ko) 2014-07-22 2017-08-01 바르코 인코포레이티드 색상의 파장 다중화를 이용한 디스플레이 시스템 및 방법
EP3172620A4 (en) * 2014-07-22 2018-03-28 Barco Inc. Display systems and methods employing time mutiplexing of projection screens and projectors
US11258983B2 (en) 2014-09-25 2022-02-22 Steve H. McNelley Immersive communication terminals
US10841535B2 (en) 2014-09-25 2020-11-17 Steve H. McNelley Configured transparent communication terminals
US11099465B2 (en) 2014-09-25 2021-08-24 Steve H. McNelley Communication stage and display systems
US11750772B2 (en) 2014-09-25 2023-09-05 Steve H. McNelley Rear illuminated transparent communication terminals
US10129506B2 (en) * 2014-09-25 2018-11-13 Steve H. McNelley Advanced transparent projection communication terminals
RU2696952C2 (ru) 2014-10-01 2019-08-07 Долби Интернешнл Аб Аудиокодировщик и декодер
US9726968B2 (en) 2014-10-27 2017-08-08 Barco, Inc. Display systems and methods employing screens with an array of micro-lenses or micro-mirrors
CN104284130A (zh) * 2014-10-29 2015-01-14 四川智诚天逸科技有限公司 一种视频通信设备
TWI688789B (zh) * 2014-11-20 2020-03-21 美商英特爾股份有限公司 虛擬影像產生器及投影虛擬影像的方法
CN104469340B (zh) * 2014-12-01 2017-03-15 深圳凯澳斯科技有限公司 一种立体视频共光心成像系统及其成像方法
US10216982B2 (en) 2015-03-12 2019-02-26 Microsoft Technology Licensing, Llc Projecting a virtual copy of a remote object
US9858719B2 (en) * 2015-03-30 2018-01-02 Amazon Technologies, Inc. Blended reality systems and methods
US9554207B2 (en) 2015-04-30 2017-01-24 Shure Acquisition Holdings, Inc. Offset cartridge microphones
US9565493B2 (en) 2015-04-30 2017-02-07 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US10778855B2 (en) * 2015-06-19 2020-09-15 Line Corporation System and method for creating contents by collaborating between users
JP6733675B2 (ja) * 2015-08-21 2020-08-05 ソニー株式会社 プロジェクションシステム、および機器ユニット、ならびに機器制御方法
EP3148188A1 (en) * 2015-09-24 2017-03-29 Airbus Operations GmbH Virtual windows for airborne verhicles
EP3148186A1 (en) * 2015-09-24 2017-03-29 Airbus Operations GmbH Virtual windows for airborne vehicles
US10834373B2 (en) * 2016-03-11 2020-11-10 Hyperloop Transportation Technologies, Inc. Augmented windows
WO2017205642A1 (en) * 2016-05-25 2017-11-30 Livit Media Inc. Methods and systems for live sharing 360-degree video streams on a mobile device
JP6883225B2 (ja) * 2016-08-17 2021-06-09 ソニーグループ株式会社 表示制御装置、表示制御方法およびプログラム
JP6844171B2 (ja) * 2016-09-23 2021-03-17 カシオ計算機株式会社 投影装置、投影方法及びプログラム
US10264302B2 (en) * 2016-09-30 2019-04-16 Ricoh Company, Ltd. Communication management apparatus, method and computer-readable storage medium for generating image data identification information
CN106331956A (zh) * 2016-11-04 2017-01-11 北京声智科技有限公司 集成远场语音识别和声场录制的系统和方法
CN108616742B (zh) * 2016-12-21 2020-04-07 深圳市掌网科技股份有限公司 一种3d全景拍摄系统以及方法
US10367948B2 (en) 2017-01-13 2019-07-30 Shure Acquisition Holdings, Inc. Post-mixing acoustic echo cancellation systems and methods
WO2018147329A1 (ja) * 2017-02-10 2018-08-16 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 自由視点映像生成方法及び自由視点映像生成システム
CN107024828A (zh) * 2017-03-29 2017-08-08 深圳市未来媒体技术研究院 一种可共光心的摄像机装置、全景无缝拼接组件及方法
TWI672677B (zh) * 2017-03-31 2019-09-21 鈺立微電子股份有限公司 用以融合多深度圖的深度圖產生裝置
WO2018184735A1 (de) * 2017-04-06 2018-10-11 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Konzept zur virtuellen teilnahme an live-ereignissen
CN107087208B (zh) * 2017-05-25 2020-07-07 深圳市酷开网络科技有限公司 一种全景视频播放方法、系统及存储装置
US10334360B2 (en) * 2017-06-12 2019-06-25 Revolabs, Inc Method for accurately calculating the direction of arrival of sound at a microphone array
CN107463248A (zh) * 2017-06-20 2017-12-12 昆明理工大学 一种基于动态捕捉与全息投影的远程交互方法
CN110019886A (zh) * 2017-08-28 2019-07-16 富泰华工业（深圳）有限公司 全景影像生成装置及方法
CN109598674B (zh) * 2017-09-30 2023-05-02 杭州海康威视数字技术股份有限公司 一种图像拼接方法及装置
CN107613243A (zh) * 2017-11-02 2018-01-19 深圳市裂石影音科技有限公司 一种基于语音跟踪的全景视频录制设备及录制方法
RU2686576C1 (ru) 2017-11-30 2019-04-29 Самсунг Электроникс Ко., Лтд. Компактное устройство голографического дисплея
CN109978760B (zh) * 2017-12-27 2023-05-02 杭州海康威视数字技术股份有限公司 一种图像拼接方法及装置
DE112019001215T5 (de) * 2018-03-08 2020-11-19 Sony Corporation Informationsverarbeitungsgerät, Informationsverarbeitungsverfahren, Informationsverarbeitungssystem und Programm
CN110324559B (zh) * 2018-03-28 2021-11-30 北京富纳特创新科技有限公司 视频通信装置及方法
CN110324555B (zh) * 2018-03-28 2021-02-26 北京富纳特创新科技有限公司 视频通信装置及方法
CN108513117A (zh) * 2018-05-09 2018-09-07 北京邦邦共赢网络科技有限公司 一种基于全息投影的影像投射方法及装置
WO2019231632A1 (en) 2018-06-01 2019-12-05 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
HUP1800193A1 (hu) * 2018-06-06 2019-12-30 Vivien Solutions Kft Rendszer egymástól távol lévõ személyek közötti vizuális kollaboráció, különösen virtuális ügyfélszolgálat biztosítására, valamint eljárás vizuális kollaboráció, különösen virtuális ügyfélszolgálat létrehozására
US11297423B2 (en) 2018-06-15 2022-04-05 Shure Acquisition Holdings, Inc. Endfire linear array microphone
CN110892715A (zh) * 2018-08-29 2020-03-17 深圳市大疆创新科技有限公司 视频的处理方法、装置、显示系统及存储介质
US11310596B2 (en) 2018-09-20 2022-04-19 Shure Acquisition Holdings, Inc. Adjustable lobe shape for array microphones
US10904479B2 (en) * 2019-03-12 2021-01-26 Light Field Lab, Inc. Video communication including holographic content
JP2022526761A (ja) 2019-03-21 2022-05-26 シュアー アクイジッション ホールディングス インコーポレイテッド 阻止機能を伴うビーム形成マイクロフォンローブの自動集束、領域内自動集束、および自動配置
US11303981B2 (en) 2019-03-21 2022-04-12 Shure Acquisition Holdings, Inc. Housings and associated design features for ceiling array microphones
US11558693B2 (en) 2019-03-21 2023-01-17 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality
US11445294B2 (en) 2019-05-23 2022-09-13 Shure Acquisition Holdings, Inc. Steerable speaker array, system, and method for the same
WO2020243471A1 (en) 2019-05-31 2020-12-03 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
US11297426B2 (en) 2019-08-23 2022-04-05 Shure Acquisition Holdings, Inc. One-dimensional array microphone with improved directivity
US11552611B2 (en) 2020-02-07 2023-01-10 Shure Acquisition Holdings, Inc. System and method for automatic adjustment of reference gain
US11706562B2 (en) 2020-05-29 2023-07-18 Shure Acquisition Holdings, Inc. Transducer steering and configuration systems and methods using a local positioning system
US11159748B1 (en) * 2020-07-14 2021-10-26 Guy Cohen Paz Studio in a box
CN116918351A (zh) 2021-01-28 2023-10-20 舒尔获得控股公司 混合音频波束成形系统
CN113452909A (zh) * 2021-06-10 2021-09-28 高玉宗 一种全景呈现系统及其应用

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6977676B1 (en) * 1998-07-08 2005-12-20 Canon Kabushiki Kaisha Camera control system
CN1917623A (zh) * 2005-08-17 2007-02-21 索尼株式会社 摄像机控制器和远程会议系统
US20070299912A1 (en) * 2006-06-26 2007-12-27 Microsoft Corporation, Corporation In The State Of Washington Panoramic video in a live meeting client

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4928301A (en) 1988-12-30 1990-05-22 Bell Communications Research, Inc. Teleconferencing terminal with camera behind display screen
US5115266A (en) * 1989-11-08 1992-05-19 Troje Gerald J Optical system for recording or projecting a panoramic image
US5495576A (en) * 1993-01-11 1996-02-27 Ritchey; Kurtis J. Panoramic image based virtual reality/telepresence audio-visual system and method
US5990934A (en) * 1995-04-28 1999-11-23 Lucent Technologies, Inc. Method and system for panoramic viewing
US5745305A (en) * 1995-04-28 1998-04-28 Lucent Technologies Inc. Panoramic viewing apparatus
US5801758A (en) * 1995-06-21 1998-09-01 Apple Computer, Inc. System and method for image capture and display utilizing time sharing across a single, two-way optical path
US5793527A (en) * 1995-06-30 1998-08-11 Lucent Technologies Inc. High resolution viewing system
US8199185B2 (en) * 1995-09-20 2012-06-12 Videotronic Systems Reflected camera image eye contact terminal
US6795106B1 (en) * 1999-05-18 2004-09-21 Intel Corporation Method and apparatus for controlling a video camera in a video conferencing system
US7015954B1 (en) * 1999-08-09 2006-03-21 Fuji Xerox Co., Ltd. Automatic video system using multiple cameras
WO2001011880A1 (en) * 1999-08-10 2001-02-15 Peter Mcduffie White Communications system
GB9918704D0 (en) * 1999-08-10 1999-10-13 White Peter M Device and method for eye to eye communication overa network
US20020075295A1 (en) * 2000-02-07 2002-06-20 Stentz Anthony Joseph Telepresence using panoramic imaging and directional sound
US8035612B2 (en) * 2002-05-28 2011-10-11 Intellectual Ventures Holding 67 Llc Self-contained interactive video display system
ATE483327T1 (de) * 2001-08-15 2010-10-15 Koninkl Philips Electronics Nv 3d videokonferenzsystem
US6583808B2 (en) * 2001-10-04 2003-06-24 National Research Council Of Canada Method and system for stereo videoconferencing
US6783247B2 (en) * 2002-03-14 2004-08-31 White Peter Mcduffie Life-size communications systems with front projection
US6813074B2 (en) * 2002-05-31 2004-11-02 Microsoft Corporation Curved-screen immersive rear projection display
CN100434967C (zh) 2003-05-06 2008-11-19 皇家飞利浦电子股份有限公司 通过全息屏幕同时产生同一个图像的多个互相隔开的全息帧的投影装置
US20050041286A1 (en) * 2003-08-18 2005-02-24 White Peter Mcduffie Front projection screen and systems using same
US7057637B2 (en) * 2004-04-21 2006-06-06 White Peter Mcduffie Reflected backdrop for communications systems
US8208007B2 (en) * 2004-04-21 2012-06-26 Telepresence Technologies, Llc 3-D displays and telepresence systems and methods therefore
US8456506B2 (en) * 2004-08-03 2013-06-04 Applied Minds, Llc Systems and methods for enhancing teleconferencing collaboration
US7768544B2 (en) * 2005-01-21 2010-08-03 Cutler Ross G Embedding a panoramic image in a video stream
CN101142047B (zh) * 2005-03-19 2011-06-08 罗姆股份有限公司 钻卡头
RU2322771C2 (ru) 2005-04-25 2008-04-20 Святослав Иванович АРСЕНИЧ Стереопроекционная система
US7679639B2 (en) * 2006-04-20 2010-03-16 Cisco Technology, Inc. System and method for enhancing eye gaze in a telepresence system
US8773495B2 (en) * 2007-10-12 2014-07-08 Polycom, Inc. Integrated system for telepresence videoconferencing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6977676B1 (en) * 1998-07-08 2005-12-20 Canon Kabushiki Kaisha Camera control system
CN1917623A (zh) * 2005-08-17 2007-02-21 索尼株式会社 摄像机控制器和远程会议系统
US20070299912A1 (en) * 2006-06-26 2007-12-27 Microsoft Corporation, Corporation In The State Of Washington Panoramic video in a live meeting client

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
LEI ZHONG-FENG ET AL: "Video Image Stitching and its Application", CHINA CABLE TELEVISION, 2003, pages 70 - 72, XP008161538 *
MA LI ET AL, A HIGH RESOLUTION PANORAMIC VIDEO MONITORING SYSTEM BASED ON VIDEO MOSAICING JOURNAL OF IMAGE AND GRAPHICS, vol. 13, no. 12, December 2008 (2008-12-01), pages 2291 - 2296, XP008161537 *
See also references of EP2290969A4 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103843324A (zh) * 2011-06-15 2014-06-04 梅巴菲科姆公司 使用倒置望远镜摄像机的视频会议系统
CN103843324B (zh) * 2011-06-15 2017-03-15 泰莱奥飞斯菲科姆公司 使用倒置望远镜摄像机的视频会议系统
US9143724B2 (en) 2011-07-06 2015-09-22 Hewlett-Packard Development Company, L.P. Telepresence portal system
CN102685445A (zh) * 2012-04-27 2012-09-19 华为技术有限公司 网真视频图像输送方法、设备及网真系统
CN102761763A (zh) * 2012-06-26 2012-10-31 中国航空工业第六一八研究所 一种不少于四通道的主动立体视景装置
WO2017120308A1 (en) * 2016-01-05 2017-07-13 360fly, Inc. Dynamic adjustment of exposure in panoramic video content
CN105681681A (zh) * 2016-01-16 2016-06-15 深圳算云科技有限公司 一种多视频流的视频压缩方法及系统
CN105578199A (zh) * 2016-02-22 2016-05-11 北京佰才邦技术有限公司 虚拟现实全景多媒体处理系统、方法及客户端设备

Also Published As

Publication number Publication date
CN102084650B (zh) 2013-10-09
BRPI0924076A2 (pt) 2021-01-26
RU2518218C2 (ru) 2014-06-10
US20110096136A1 (en) 2011-04-28
US20130100240A1 (en) 2013-04-25
RU2011130860A (ru) 2013-06-20
US8692861B2 (en) 2014-04-08
EP2290969A4 (en) 2011-06-29
US8836750B2 (en) 2014-09-16
EP2290969A1 (en) 2011-03-02
CN102084650A (zh) 2011-06-01

Similar Documents

Publication Publication Date Title
WO2010130084A1 (zh) 远程呈现系统、方法及视频采集设备
US10447967B2 (en) Live teleporting system and apparatus
US8508576B2 (en) Remote presenting system, device, and method
US8860775B2 (en) Remote presenting system, device, and method
US20020063780A1 (en) Teleconferencing system
US20070171275A1 (en) Three Dimensional Videoconferencing
US8289367B2 (en) Conferencing and stage display of distributed conference participants
US9143727B2 (en) Dual-axis image equalization in video conferencing
US20150208036A1 (en) Video conference display method and device
US7190388B2 (en) Communication terminal and system
WO2013060295A1 (zh) 一种视频处理方法和系统
WO2011003234A1 (zh) 光学立体放映装置、系统及方法
JP2016072844A (ja) 映像システム
BRPI0924076B1 (pt) Sistema de telepresença e método de telepresença

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200980116849.2

Country of ref document: CN

REEP Request for entry into the european phase

Ref document number: 2009842025

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2009842025

Country of ref document: EP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09842025

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2552/KOLNP/2011

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2011130860

Country of ref document: RU

Kind code of ref document: A

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: PI0924076

Country of ref document: BR

ENP Entry into the national phase

Ref document number: PI0924076

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20110705