CN101662693A - Method, device and system for sending and playing multi-viewpoint media content - Google Patents
Method, device and system for sending and playing multi-viewpoint media content Download PDFInfo
- Publication number
- CN101662693A CN101662693A CN200810146721A CN200810146721A CN101662693A CN 101662693 A CN101662693 A CN 101662693A CN 200810146721 A CN200810146721 A CN 200810146721A CN 200810146721 A CN200810146721 A CN 200810146721A CN 101662693 A CN101662693 A CN 101662693A
- Authority
- CN
- China
- Prior art keywords
- information
- media content
- viewpoint
- audio
- viewpoint media
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/172—Processing image signals image signals comprising non-image signal components, e.g. headers or format information
- H04N13/178—Metadata, e.g. disparity information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
Abstract
The embodiment of the invention discloses a method, a device and a system for playing and sending multi-viewpoint media content, which relate to the media content playing technology, and solve the problem that an audio signal is possibly not matched with a video image corresponding to a switched viewpoint after viewpoint switching. The invention adopts the technical scheme that the method for playing the multi-viewpoint media content comprises the following steps: receiving the multi-viewpoint media content; generating switched viewpoint information during the viewpoint switching; generating avideo signal and an audio signal corresponding to the viewpoint information according to the viewpoint information and the multi-viewpoint media content; and synchronously outputting the video signaland the audio signal. The method, the device and the system provided by the embodiment of the invention for playing and sending the multi-viewpoint media content can be applied to a system with multi-viewpoint media content playing function.
Description
Technical field
The present invention relates to the communications field, relate in particular to a kind of transmission of multi-viewpoint media content and player method, Apparatus and system.
Background technology
Multi-viewpoint media content is meant the media content of being made up of multi-view point video information and audio-frequency information.Wherein, described multi-view point video information is meant uses a plurality of video cameras, from different perspectives Same Scene is taken synchronously the video information of a plurality of video flowings compositions of acquisition.At the broadcast end of multi-viewpoint media content, the beholder can watch described multi-viewpoint media content from different perspectives by selecting different viewpoints.
But broadcast end at multi-viewpoint media content, the broadcast direction of sound source is changeless, promptly described multi-viewpoint media content is carried out after viewpoint switches, may there be differential seat angle between vision signal that the beholder watches and the audio signal of hearing, makes vision signal and audio signal not match.For example: as shown in Figure 1, beholder P watches the same personage the same scene from three different viewpoints (corresponding angle is respectively ∠ α, ∠ β and ∠ γ), obtain vision signal A, B and the C of described three viewpoint correspondences, in Fig. 1, sound source is positioned at S place (angle is ∠ α), when beholder's selected angle is the viewpoint of ∠ α when watching, vision signal A is identical with the angle of audio signal (S sends from sound source), and this moment, vision signal A and audio signal were complementary; When beholder's selected angle is the viewpoint of ∠ β or ∠ γ when watching, there is differential seat angle between vision signal B or C and the audio signal (S sends from sound source), vision signal and audio signal do not match.
Summary of the invention
Embodiments of the invention provide a kind of transmission of multi-viewpoint media content and player method, Apparatus and system, can after switching viewpoint the broadcast direction of vision signal and audio signal be mated.
For achieving the above object, embodiments of the invention adopt following technical scheme:
A kind of player method of multi-viewpoint media content comprises: receive multi-viewpoint media content; When carrying out the viewpoint switching, generate the view information after switching; According to described view information and multi-viewpoint media content, generate vision signal and the corresponding audio signal corresponding with this view information; Export described vision signal and audio signal synchronously.
A kind of sending method of multi-viewpoint media content comprises: according to the video information of many viewpoints, obtain the three-dimensional information of described video information; According to the audio-frequency information that receives from a plurality of diverse locations, obtain the sound source position information of described audio-frequency information; The video information of described many viewpoints and the three-dimensional information of this video information and the sound source position information of described audio-frequency information and this audio-frequency information are encoded, send after generating multi-viewpoint media content.
A kind of playing device of multi-viewpoint media content comprises:
The media content receiving element is used to receive multi-viewpoint media content;
The view information generation unit is used for generating the view information after switching when carrying out the viewpoint switching;
The signal generation unit is used for the view information according to described view information generation unit generation, and the multi-viewpoint media content of media content receiving element reception, generates vision signal and the corresponding audio signal corresponding with this view information;
Output unit is used for exporting synchronously vision signal and the audio signal that described signal generation unit generates synchronously.
A kind of dispensing device of multi-viewpoint media content comprises:
The video information process unit is used for obtaining the three-dimensional information of described video information according to multi-view point video information;
The audio-frequency information processing unit is used for basis from the audio-frequency information that a plurality of diverse locations receive, and obtains the sound source position information of described audio-frequency information;
The multi-viewpoint media content generation unit, the three-dimensional information that is used for described multi-view point video information that the video information of described many viewpoints and described video information process unit are obtained, the sound source position information of the described audio-frequency information that obtains with described audio-frequency information and described audio-frequency information processing unit is encoded, and sends after generating multi-viewpoint media content.
A kind of Play System of multi-viewpoint media content comprises:
The multi-viewpoint media content dispensing device, be used for handling to the video information of many viewpoints of receiving and from the audio-frequency information that a plurality of diverse locations receive, obtain the sound source position information of the three-dimensional information and the described audio-frequency information of described video information, three-dimensional information with video information and this video information of described many viewpoints, encode with the sound source position information of described audio-frequency information and this audio-frequency information, send behind the generation multi-viewpoint media content;
The multi-viewpoint media content playing device, be used to receive the multi-viewpoint media content that described multi-viewpoint media content dispensing device sends, when carrying out the viewpoint switching, generate the view information after switching, according to this view information and the multi-viewpoint media content that receives, generate corresponding vision signal and audio signal, export described vision signal and audio signal synchronously.
The transmission of the multi-viewpoint media content that the embodiment of the invention provides and player method, Apparatus and system, owing to comprised the sound source position information of the three-dimensional information and the audio-frequency information of multi-view point video information in the multi-viewpoint media content that transmitting terminal sends, so playing end can be according to view information after switching and the multi-viewpoint media content that receives, generate vision signal and the audio signal corresponding with this view information, solved prior art because audio signal is changeless, after carrying out the viewpoint switching, audio signal and the unmatched problem of the switching pairing vision signal of backsight point.
Description of drawings
Fig. 1 watches the schematic diagram of multi-viewpoint media content from three different points of view for beholder in the prior art;
The sending method flow chart one of the multi-viewpoint media content that Fig. 2 provides for the embodiment of the invention;
The sending method flowchart 2 of the multi-viewpoint media content that Fig. 3 provides for the embodiment of the invention;
The player method flow chart of the multi-viewpoint media content that Fig. 4 provides for the embodiment of the invention;
The dispensing device structural representation one of the multi-viewpoint media content that Fig. 5 provides for the embodiment of the invention;
The dispensing device structural representation two of the multi-viewpoint media content that Fig. 6 provides for the embodiment of the invention;
The playing device structural representation one of the multi-viewpoint media content that Fig. 7 provides for the embodiment of the invention;
The playing device structural representation two of the multi-viewpoint media content that Fig. 8 provides for the embodiment of the invention;
The Play System structural representation one of the multi-viewpoint media content that Fig. 9 provides for the embodiment of the invention;
The Play System structural representation two of the multi-viewpoint media content that Figure 10 provides for the embodiment of the invention.
Embodiment
As shown in Figure 2, the sending method of the multi-viewpoint media content that the embodiment of the invention provides comprises:
In the present embodiment, the video information of described many viewpoints is taken by a shooting unit and is obtained, this shooting unit comprises the video camera that is positioned at different points of view more than, described step 201 can be carried out three-dimensional information to the video information of described many viewpoints and be handled, obtain the three-dimensional information of this multi-view point video information, wherein, this three-dimensional information can comprise: the parallax information between the depth information of described multi-view point video information and the adjacent viewpoint video information etc.;
In the present embodiment, the described audio-frequency information that receives from a plurality of diverse locations obtains by a microphone array, this microphone array comprises a plurality of microphones that are positioned at diverse location, described step 202 can be handled the described array signal process techniques such as audio-frequency information use wave beam formation that obtain by microphone array, obtains the sound source position information of described audio-frequency information;
In the present embodiment, may comprise more than one sound source signal in the described audio-frequency information, at this moment, the sound source position information of the audio-frequency information that described step 202 obtains is the sound source position information of each sound source signal correspondence;
The embodiment of the invention provide the sending method of multi-viewpoint media content, owing to comprise the sound source position information of the three-dimensional information and the audio-frequency information of multi-view point video information in the multi-viewpoint media content of its transmission, so the vision signal and the audio signal that generate switching backsight point correspondence for the broadcast end after viewpoint is switched provide condition.
When the sending method of the multi-viewpoint media content that provides when the embodiment of the invention is applied in the bilateral system, as be applied in the meeting-place, as shown in Figure 3, before step 202 as shown in Figure 2, can also comprise:
Described step 200a and 200b can be positioned at before the described step 201, also can be positioned at after the described step 201, and in the present embodiment, as shown in Figure 3, described step 200a and 200b are positioned at before the step 201.
The sending method of the multi-viewpoint media content that the embodiment of the invention provides because the audio-frequency information that receives has been carried out the echo cancelltion processing, makes in bilateral system, and the audio signal of playing the end broadcast can not produce the audio-frequency information that transmitting terminal receives to be disturbed.
As shown in Figure 4, the player method of the multi-viewpoint media content that the embodiment of the invention provides comprises:
In the present embodiment, described step 401 can receive the multi-viewpoint media content transmitting terminal by network and send multi-viewpoint media content; Described multi-viewpoint media content can comprise: the three-dimensional information of video information and this video information (as: depth information or parallax information etc.), with sound source position information of audio-frequency information and this audio-frequency information etc., wherein, described video information is taken the video flowing that obtains by an above viewpoint and is formed, described audio-frequency information comprises at least one source of sound information, and the sound source position information of described audio-frequency information is meant the positional information of each source of sound;
On the theory, the video information that comprises in the described multi-viewpoint media content should be taken the video flowing that obtains by all viewpoints and be formed, yet, for the consideration of taking cost, in fact the video information that comprises in the described multi-viewpoint media content only is made up of the video flowing that several crucial viewpoints are taken acquisition, and for example: described video information is formed by taking the video flowing that obtains from front, left surface, right flank and the back of scenery;
In sum, in the present embodiment, described step 403 specifically is the video information of two adjacent crucial viewpoints of the viewpoint after utilization and the switching and the parallax information between this video information, uses the virtual view composition algorithm, the vision signal of the viewpoint correspondence after the synthetic described switching;
In the present embodiment, described step 403 generates and the step of view information corresponding audio signal can comprise: at first, according to the later view information of switching that obtains in the described step 402, and the sound source position information of described multi-viewpoint media content sound intermediate frequency information, generate sound source position information with this view information corresponding audio information; Then, the audio-frequency information according to comprising in the sound source position information of the audio-frequency information of described generation and the described multi-viewpoint media content uses the wavefront synthetic technology, generates and this view information corresponding audio signal; Certainly, the viewpoint corresponding audio signal after described step 403 also can adopt other three-dimensional audio play-back technologies that are similar to wavefront synthetic technology generations and switch, is not given unnecessary details other situations herein;
When comprising more than one source of sound in the described audio-frequency information, described step 403 need for each source of sound generate respectively with switch after the corresponding sound source position information of viewpoint;
Further, the player method of the multi-viewpoint media content that the embodiment of the invention provides after described step 403, can also comprise: the viewpoint corresponding audio signal after described and the switching is carried out the step that echo cancelltion is handled.
The player method of the multi-viewpoint media content that the embodiment of the invention provides, can be according to view information after switching and the multi-viewpoint media content that receives, generate vision signal and the audio signal corresponding with this view information, solved prior art because audio signal is changeless, after carrying out the viewpoint switching, there is differential seat angle between audio signal and the pairing vision signal of the switching backsight point position, the audio signal and the unmatched problem of vision signal that make broadcast, realize the purpose that audio signal and vision signal are switched synchronously, improved the sense of reality and telepresenc that the user watches described multi-viewpoint media content.
The sending method of the multi-viewpoint media content that provides with the invention described above embodiment accordingly, as shown in Figure 5, the embodiment of the invention also provides a kind of dispensing device of multi-viewpoint media content, comprising:
Video information process unit 501 is used for obtaining the three-dimensional information of described video information according to multi-view point video information;
Audio-frequency information processing unit 502 is used for basis from the audio-frequency information that a plurality of diverse locations receive, and obtains the sound source position information of described audio-frequency information;
Multi-viewpoint media content generation unit 503, the three-dimensional information that is used for described multi-view point video information that the video information of described many viewpoints and described video information process unit 501 are obtained, the sound source position information of the described audio-frequency information that obtains with described audio-frequency information and described audio-frequency information processing unit 502 is encoded, and sends after generating multi-viewpoint media content.
Further, as shown in Figure 6, the dispensing device of the multi-viewpoint media content that the embodiment of the invention provides can also comprise:
Audio signal acquiring unit 504 is used to obtain the audio signal of the multi-viewpoint media content of broadcast;
Echo cancelltion processing unit 505 is used for the audio signal of the broadcast of obtaining according to described echo cancelltion information receiving unit 504, the described audio-frequency information that receives from a plurality of diverse locations is carried out echo cancelltion handle;
Described audio-frequency information processing unit 502 is used for that also described echo cancelltion processing unit 505 is handled later audio-frequency information and handles, and obtains the three-dimensional information of this audio-frequency information.
The embodiment of the invention provide the dispensing device of multi-viewpoint media content, owing to comprise the sound source position information of the three-dimensional information and the audio-frequency information of multi-view point video information in the multi-viewpoint media content of its transmission, so the vision signal and the audio signal that generate switching backsight point correspondence for the broadcast end after viewpoint is switched provide condition.
As shown in Figure 7, the playing device of the multi-viewpoint media content that the embodiment of the invention provides comprises:
Media content receiving element 701 is used to receive multi-viewpoint media content;
In the present embodiment, described media content receiving element 701 can pass through network interface, and receiving end/sending end is through handling later multi-viewpoint media content from the network; Described multi-viewpoint media content can comprise: the three-dimensional information of video information and this video information (as: depth information or parallax information etc.), with sound source position information of audio-frequency information and this audio-frequency information etc., wherein, described video information is taken the video flowing that obtains by an above viewpoint and is formed, described audio-frequency information comprises at least one source of sound information, and the sound source position information of described audio-frequency information is meant the positional information of each source of sound;
View information generation unit 702 is used for generating the view information after switching when carrying out the viewpoint switching;
Further, as shown in Figure 8, described view information generation unit 702 can comprise:
Handover information acquiring unit 7021 is used to obtain the viewpoint handover information;
Further, as shown in Figure 8, described signal generation unit 703 comprises audio-frequency information generation unit 7031, and this audio signal generation unit 7031 can comprise:
Positional information generation unit 70311, be used for view information according to described view information generation unit 702 generations, and the sound source position information of the audio-frequency information that comprises in the described multi-viewpoint media content, generate sound source position information with described view information corresponding audio information;
Further, as shown in Figure 8, the playing device of described multi-viewpoint media content can also comprise:
Echo cancelltion processing unit 705 is used for carrying out the echo cancelltion processing with described with view information corresponding audio signal.
The playing device of the multi-viewpoint media content that the embodiment of the invention provides, can be according to view information after switching and the multi-viewpoint media content that receives, generate vision signal and the audio signal corresponding with this view information, solved prior art because audio signal is changeless, after carrying out the viewpoint switching, there is differential seat angle between audio signal and the pairing vision signal of the switching backsight point position, make the unmatched problem of audio signal and vision signal, realize the purpose that audio ﹠ video switches synchronously, improved the sense of reality and telepresenc that the user watches described multi-viewpoint media content.
As shown in Figure 9, the Play System of the multi-viewpoint media content that the embodiment of the invention provides comprises:
Multi-viewpoint media content dispensing device 901, be used for handling to the video information of many viewpoints of receiving and from the audio-frequency information that more than one diverse location receives, obtain the sound source position information of the three-dimensional information and the described audio-frequency information of described video information, three-dimensional information with video information and this video information of described many viewpoints, encode with the sound source position information of described audio-frequency information and this audio-frequency information, send behind the generation multi-viewpoint media content;
Multi-viewpoint media content playing device 902, be used to receive the multi-viewpoint media content that described multi-viewpoint media content dispensing device 901 sends, when carrying out the viewpoint switching, generate the view information after switching, according to this view information and the multi-viewpoint media content that receives, generate corresponding vision signal and audio signal, export described vision signal and audio signal synchronously.
Further, when the Play System of the multi-viewpoint media content that provides when the embodiment of the invention was an intercommunication system, as the meeting-place, as shown in figure 10, the Play System of described multi-viewpoint media content can also comprise:
Described multi-viewpoint media content dispensing device 901 also is used for the audio signal according to described echo cancelltion device 903 transmissions, the audio-frequency information that receives from a plurality of diverse locations is carried out echo cancelltion handle.
The Play System of the multi-viewpoint media content that the embodiment of the invention provides, can be according to view information after switching and the multi-viewpoint media content that receives, generate vision signal and the audio signal corresponding with this view information, solved prior art because audio signal is changeless, after carrying out the viewpoint switching, there is differential seat angle between audio signal and the pairing vision signal of the switching backsight point position, make the unmatched problem of audio signal and vision signal, realize the purpose that audio ﹠ video switches synchronously, improved the sense of reality and telepresenc that the user watches described multi-viewpoint media content.
One of ordinary skill in the art will appreciate that all or part of step that realizes in the foregoing description method is to instruct relevant hardware to finish by program, described program can be stored in the computer-readable recording medium, as ROM/RAM, magnetic disc or CD etc.
The above; only be the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, anyly is familiar with those skilled in the art in the technical scope that the present invention discloses; can expect easily changing or replacing, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion by described protection range with claim.
Claims (15)
1, a kind of player method of multi-viewpoint media content is characterized in that, comprising:
Receive multi-viewpoint media content;
When carrying out the viewpoint switching, generate the view information after switching;
According to described view information and multi-viewpoint media content, generate vision signal and the corresponding audio signal corresponding with this view information;
Export described vision signal and audio signal synchronously.
2, the player method of multi-viewpoint media content according to claim 1, it is characterized in that, described multi-viewpoint media content comprises: the sound source position information of the video information of many viewpoints and the three-dimensional information of this video information and audio-frequency information and this audio-frequency information.
3, the player method of multi-viewpoint media content according to claim 2 is characterized in that, the view information after described generation is switched comprises:
Obtain the viewpoint handover information;
According to the three-dimensional information of described viewpoint handover information and video information, generate the view information after switching.
4, the player method of multi-viewpoint media content according to claim 2 is characterized in that, and is described according to described view information and multi-viewpoint media content, generates with this view information corresponding audio signal to comprise:
According to the sound source position information of described view information and described audio-frequency information, generate sound source position information with described view information corresponding audio information;
According to described audio-frequency information and with the sound source position information of described view information corresponding audio information, generate and described view information corresponding audio signal.
5, the player method of multi-viewpoint media content according to claim 1, it is characterized in that, described according to described view information and multi-viewpoint media content, generate after the vision signal and corresponding audio signal corresponding, also comprise: carry out the echo cancelltion processing with view information corresponding audio signal described with this view information.
6, a kind of sending method of multi-viewpoint media content is characterized in that, comprising:
According to the video information of many viewpoints, obtain the three-dimensional information of described video information;
According to the audio-frequency information that receives from a plurality of diverse locations, obtain the sound source position information of described audio-frequency information;
The video information of described many viewpoints and the three-dimensional information of this video information and the sound source position information of described audio-frequency information and this audio-frequency information are encoded, send after generating multi-viewpoint media content.
7, the sending method of multi-viewpoint media content according to claim 6 is characterized in that, described method also comprises:
Obtain the audio signal of the multi-viewpoint media content of broadcast;
According to the audio signal of the multi-viewpoint media content of the broadcast of obtaining, the described audio-frequency information that receives from a plurality of diverse locations is carried out echo cancelltion handle.
8, a kind of playing device of multi-viewpoint media content is characterized in that, comprising:
The media content receiving element is used to receive multi-viewpoint media content;
The view information generation unit is used for generating the view information after switching when carrying out the viewpoint switching;
The signal generation unit is used for the view information according to described view information generation unit generation, and the multi-viewpoint media content of media content receiving element reception, generates vision signal and the corresponding audio signal corresponding with this view information;
Output unit is used for exporting synchronously vision signal and the audio signal that described signal generation unit generates synchronously.
9, the playing device of multi-viewpoint media content according to claim 8 is characterized in that, described view information generation unit comprises:
The handover information acquiring unit is used to obtain the viewpoint handover information;
First generation unit is used for the viewpoint handover information that obtains according to described handover information acquiring unit, and the three-dimensional information of the video information that comprises in the described multi-viewpoint media content, generates the view information after switching.
10, the playing device of multi-viewpoint media content according to claim 8 is characterized in that, described signal generation unit comprises the audio signal generation unit, and this audio signal generation unit comprises:
The positional information generation unit, be used for view information according to described view information generation unit generation, and the sound source position information of the audio-frequency information that comprises in the described multi-viewpoint media content, generate sound source position information with described view information corresponding audio information;
Second generation unit, be used for the audio-frequency information that comprises according to described multi-viewpoint media content and positional information generation unit that generate with sound source position information described view information corresponding audio information, generate and described view information corresponding audio signal.
11, the playing device of multi-viewpoint media content according to claim 8 is characterized in that, also comprises:
The echo cancelltion processing unit is used for carrying out the echo cancelltion processing with described with view information corresponding audio signal.
12, a kind of dispensing device of multi-viewpoint media content is characterized in that, comprising:
The video information process unit is used for the video information according to many viewpoints, obtains the three-dimensional information of described video information;
The audio-frequency information processing unit is used for basis from the audio-frequency information that a plurality of diverse locations receive, and obtains the sound source position information of described audio-frequency information;
The multi-viewpoint media content generation unit, the three-dimensional information that is used for described multi-view point video information that the video information of described many viewpoints and described video information process unit are obtained, the sound source position information of the described audio-frequency information that obtains with described audio-frequency information and described audio-frequency information processing unit is encoded, and sends after generating multi-viewpoint media content.
13, the dispensing device of multi-viewpoint media content according to claim 12 is characterized in that, also comprises:
The audio signal acquiring unit is used to obtain the audio signal of the multi-viewpoint media content of broadcast;
The echo cancelltion processing unit is used for the audio signal of the broadcast of obtaining according to described audio signal acquiring unit, the described audio-frequency information that receives from a plurality of diverse locations is carried out echo cancelltion handle;
Described audio-frequency information processing unit also is used for the later audio-frequency information of described echo cancelltion processing unit processes is handled, and obtains the three-dimensional information of this audio-frequency information.
14, a kind of Play System of multi-viewpoint media content is characterized in that, comprising:
The multi-viewpoint media content dispensing device, be used to dock the video information of many viewpoints and handle from the audio-frequency information that a plurality of diverse locations receive, obtain the sound source position information of the three-dimensional information and the described audio-frequency information of described video information, three-dimensional information with video information and this video information of described many viewpoints, encode with the sound source position information of described audio-frequency information and this audio-frequency information, send behind the generation multi-viewpoint media content;
The multi-viewpoint media content playing device, be used to receive the multi-viewpoint media content that described multi-viewpoint media content dispensing device sends, when carrying out the viewpoint switching, generate the view information after switching, according to this view information and the multi-viewpoint media content that receives, generate corresponding vision signal and audio signal, export described vision signal and audio signal synchronously.
15, the Play System of multi-viewpoint media content according to claim 14 is characterized in that, also comprises:
The echo cancelltion device is used to receive the audio signal that described multi-viewpoint media content playing device generates, and this audio signal is sent to the multi-viewpoint media content dispensing device;
Described multi-viewpoint media content dispensing device also is used for the audio signal according to described echo cancelltion device transmission, the audio-frequency information that receives from a plurality of diverse locations is carried out echo cancelltion handle.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200810146721.8A CN101662693B (en) | 2008-08-27 | 2008-08-27 | Method, device and system for sending and playing multi-viewpoint media content |
| PCT/CN2009/073547 WO2010022658A1 (en) | 2008-08-27 | 2009-08-26 | Method, apparatus and system for playing and transmitting multi-view media content |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200810146721.8A CN101662693B (en) | 2008-08-27 | 2008-08-27 | Method, device and system for sending and playing multi-viewpoint media content |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101662693A true CN101662693A (en) | 2010-03-03 |
| CN101662693B CN101662693B (en) | 2014-03-12 |
Family
ID=41720839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200810146721.8A Active CN101662693B (en) | 2008-08-27 | 2008-08-27 | Method, device and system for sending and playing multi-viewpoint media content |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN101662693B (en) |
| WO (1) | WO2010022658A1 (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223560A (en) * | 2011-05-04 | 2011-10-19 | 友达光电股份有限公司 | Audio and video playing system and method related to double-image application |
| CN102984560A (en) * | 2011-09-07 | 2013-03-20 | 华为技术有限公司 | Method and device used for playing video from breaking point |
| CN103873846A (en) * | 2014-03-24 | 2014-06-18 | 中国人民解放军国防科学技术大学 | Video synchronization playing method for stakeholder viewpoint real three-dimensional display system based on sliding window |
| CN103905809A (en) * | 2012-12-27 | 2014-07-02 | 索尼公司 | Information processing apparatus and recording medium |
| WO2014183533A1 (en) * | 2013-12-04 | 2014-11-20 | 中兴通讯股份有限公司 | Image processing method, user terminal, and image processing terminal and system |
| CN106792142A (en) * | 2016-12-23 | 2017-05-31 | 惠州Tcl移动通信有限公司 | The audio frequency playing method and system of a kind of mobile terminal |
| CN108566514A (en) * | 2018-04-20 | 2018-09-21 | Oppo广东移动通信有限公司 | Image combining method and device, equipment, computer readable storage medium |
| CN109905719A (en) * | 2013-03-15 | 2019-06-18 | 谷歌有限责任公司 | Generate the video with multiple viewpoints |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10235010B2 (en) | 2016-07-28 | 2019-03-19 | Canon Kabushiki Kaisha | Information processing apparatus configured to generate an audio signal corresponding to a virtual viewpoint image, information processing system, information processing method, and non-transitory computer-readable storage medium |
| CN114390354A (en) * | 2020-10-21 | 2022-04-22 | 西安诺瓦星云科技股份有限公司 | Program production method, device and system and computer readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007088730A1 (en) * | 2006-01-31 | 2007-08-09 | Yamaha Corporation | Voice conference device |
| KR20080065766A (en) * | 2007-01-10 | 2008-07-15 | 광주과학기술원 | Device for tranceiving multi-view video and 3d audio, method for tranceiving the same |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100284768B1 (en) * | 1998-04-06 | 2001-03-15 | 윤종용 | Audio data processing apparatus in mult-view display system |
| JP2005159592A (en) * | 2003-11-25 | 2005-06-16 | Nippon Hoso Kyokai <Nhk> | Contents transmission apparatus and contents receiving apparatus |
| KR100954033B1 (en) * | 2007-05-07 | 2010-04-20 | 광주과학기술원 | A Method and Apparatus for View-dependent Multi-channel Audio Processing for a Multi-view Camera System |
-
2008
- 2008-08-27 CN CN200810146721.8A patent/CN101662693B/en active Active
-
2009
- 2009-08-26 WO PCT/CN2009/073547 patent/WO2010022658A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007088730A1 (en) * | 2006-01-31 | 2007-08-09 | Yamaha Corporation | Voice conference device |
| KR20080065766A (en) * | 2007-01-10 | 2008-07-15 | 광주과학기술원 | Device for tranceiving multi-view video and 3d audio, method for tranceiving the same |
Non-Patent Citations (1)
| Title |
|---|
| ILKWON PARK: "Interactive Multi-view and View-dependent Audio under MPEG-21 DIA(Digital Item Adaption)", 《3DTV CONFERENCE 2007》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223560A (en) * | 2011-05-04 | 2011-10-19 | 友达光电股份有限公司 | Audio and video playing system and method related to double-image application |
| CN102984560B (en) * | 2011-09-07 | 2017-06-20 | 华为技术有限公司 | The method and apparatus that video is played from breakpoint |
| CN102984560A (en) * | 2011-09-07 | 2013-03-20 | 华为技术有限公司 | Method and device used for playing video from breaking point |
| CN103905809B (en) * | 2012-12-27 | 2017-09-29 | 索尼公司 | Message processing device and recording medium |
| CN103905809A (en) * | 2012-12-27 | 2014-07-02 | 索尼公司 | Information processing apparatus and recording medium |
| CN109905719A (en) * | 2013-03-15 | 2019-06-18 | 谷歌有限责任公司 | Generate the video with multiple viewpoints |
| CN109905719B (en) * | 2013-03-15 | 2021-05-07 | 谷歌有限责任公司 | Generating video with multiple viewpoints |
| WO2014183533A1 (en) * | 2013-12-04 | 2014-11-20 | 中兴通讯股份有限公司 | Image processing method, user terminal, and image processing terminal and system |
| CN104994369A (en) * | 2013-12-04 | 2015-10-21 | 中兴通讯股份有限公司 | Image processing method, user terminal, and image processing terminal and system |
| CN104994369B (en) * | 2013-12-04 | 2018-08-21 | 南京中兴软件有限责任公司 | A kind of image processing method, user terminal, image processing terminal and system |
| CN103873846B (en) * | 2014-03-24 | 2015-09-23 | 中国人民解放军国防科学技术大学 | Based on many viewpoint real three-dimensional display system audio video synchronization player methods of sliding window |
| CN103873846A (en) * | 2014-03-24 | 2014-06-18 | 中国人民解放军国防科学技术大学 | Video synchronization playing method for stakeholder viewpoint real three-dimensional display system based on sliding window |
| CN106792142A (en) * | 2016-12-23 | 2017-05-31 | 惠州Tcl移动通信有限公司 | The audio frequency playing method and system of a kind of mobile terminal |
| CN108566514A (en) * | 2018-04-20 | 2018-09-21 | Oppo广东移动通信有限公司 | Image combining method and device, equipment, computer readable storage medium |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101662693B (en) | 2014-03-12 |
| WO2010022658A1 (en) | 2010-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101662693B (en) | Method, device and system for sending and playing multi-viewpoint media content | |
| KR102127955B1 (en) | Method and apparatus for playback of a higher-order ambisonics audio signal | |
| US9113034B2 (en) | Method and apparatus for processing audio in video communication | |
| CN101651841B (en) | Method, system and equipment for realizing stereo video communication | |
| JP5763184B2 (en) | Calculation of parallax for 3D images | |
| EP2329653B1 (en) | Refined depth map | |
| US20120224025A1 (en) | Transport stream structure including image data and apparatus and method for transmitting and receiving image data | |
| EP2136602A1 (en) | Communication terminal and information system | |
| JP2007195091A (en) | Synthetic image generating system | |
| WO2009076853A1 (en) | A three dimensional video communication terminal, system and method | |
| CN101610421A (en) | Video communication method, Apparatus and system | |
| WO2009143735A1 (en) | Method, device and system for 3d video communication | |
| CN105516639A (en) | Headset device, three-dimensional video call system and three-dimensional video call implementing method | |
| WO2011110107A1 (en) | System and method for implementing stereoscopic video communication in instant messaging | |
| CN101047872B (en) | Stereo audio vedio device for TV | |
| CN103748872A (en) | Receiver-side adjustment of stereoscopic images | |
| CN205320191U (en) | Helmet and three -dimensional video phone system | |
| CN102572493A (en) | Image processing apparatus and image processing method | |
| JP2014022947A (en) | Stereoscopic video transmission apparatus, stereoscopic video transmission method, and stereoscopic video processing apparatus | |
| JPH06113336A (en) | Three-dimension multi-point video conference system | |
| CN205829854U (en) | A kind of tele-conferencing system | |
| KR102260653B1 (en) | The image generation system for providing 3d image | |
| Tanimoto | Ftv (free-viewpoint television) for ray and sound reproducing in 3d space | |
| JP4424640B2 (en) | Stereo video signal generation method and transmission method, and system thereof | |
| KR20220135939A (en) | Transmission apparatus, receiving apparatus and method for providing 3d image by using point cloud data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 518129 Building 2, B District, Bantian HUAWEI base, Longgang District, Shenzhen, Guangdong. Patentee after: Huawei terminal (Shenzhen) Co.,Ltd. Address before: 518129 Building 2, B District, Bantian HUAWEI base, Longgang District, Shenzhen, Guangdong. Patentee before: HUAWEI DEVICE Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190110 Address after: 523808 Southern Factory Building (Phase I) Project B2 Production Plant-5, New Town Avenue, Songshan Lake High-tech Industrial Development Zone, Dongguan City, Guangdong Province Patentee after: HUAWEI DEVICE Co.,Ltd. Address before: 518129 Building 2, B District, Bantian HUAWEI base, Longgang District, Shenzhen, Guangdong. Patentee before: Huawei terminal (Shenzhen) Co.,Ltd. |
|
| TR01 | Transfer of patent right |