KR20100066289A - Method and apparatus for providing realistic immersive multimedia services - Google Patents

Abstract

PURPOSE: A method and an apparatus for providing realistic immersive multimedia services are provided to select a stereoscopic image area corresponding to a sound source of a stereoscopic audio using connection information between the stereoscopic audio and the stereoscopic image, thereby matching a specific object with an audio. CONSTITUTION: A stereoscopic audio and a stereoscopic image connected with the stereoscopic audio are obtained(302). Connection information between the stereoscopic audio and the stereoscopic image is obtained(304). An area corresponding to a fixed sound source of the stereoscopic audio is selected from a played stereoscopic image using the connection information(306). The connection information includes identification information for each sound source of the stereoscopic audio and identification information of each area included in the stereoscopic image.

Description

METHOD AND APPARATUS FOR PROVIDING SENSITIVE MULTIMEDIA SERVICES {METHOD AND APPARATUS FOR PROVIDING REALISTIC IMMERSIVE MULTIMEDIA SERVICES}

The present invention relates to a method and apparatus for providing immersive multimedia service using stereoscopic audio and stereoscopic images, and to a method and apparatus for reproducing sensory multimedia contents including stereoscopic audio and stereoscopic images.

The present invention is derived from a study conducted as part of the IT source technology development project of the Ministry of Knowledge Economy [Task Management No .: 2008-F-011-01, Title: Next-Generation DTV Core Technology Development (Standardization)]

Techniques for creating a 3D stereoscopic image or a stereoscopic image include a method of using left and right binocular stereoscopic images as texture stereoscopic images, and a method of using a stereoscopic image representing texture stereoscopic images and depth information. One of such stereoscopic imaging techniques may be stereoscopic stereoscopic images. A stereoscopic stereoscopic image is a stereoscopic image using left and right binocular stereoscopic images having a constant distance, that is, a viewpoint, and exhibits a stereoscopic feeling according to the distance between cameras and the distance of an object from the camera.

Viewers watching a stereoscopic image can feel a stereoscopic effect through a sense of distance, such as an object popping forward or entering backward, which was not seen in a 2D stereoscopic image. Such stereoscopic images are currently being applied to the broadcasting field, and some have started commercial 3DTV broadcasting services. In the mobile communication field, a terminal having a function of viewing a stereoscopic image by mounting a stereoscopic camera has been released, and a stereoscopic image service for transmitting related contents through a mobile communication network is expected to be provided. In the future, it is expected that 3D video communication will be possible, and related technologies are currently being developed. Such stereoscopic image related services are expected to be commonly used in all kinds of terminals capable of stereoscopic display and stereoscopic representation as well as in the above-mentioned fields.

In the case of a stereoscopic image, a three-dimensional space may be formed in the screen. That is, when a specific stereoscopic image object exists in the stereoscopic image, not only the X and Y axes but also a Z axis forming a sense of depth exist, it is possible to arrange any specific stereoscopic image object in the front and rear space as well as the left and right. Accordingly, it is possible to move a specific object in the screen from left to right or from top to bottom, and to move or arrange the object in front of or behind the screen.

This feature is more useful to users when the screen is used in formats such as MPEG-4 BIFS, where the screen consists of one or more image objects, including background video, to interact with the user, rather than simple viewing 3D video with almost no local interaction. It can provide a good spatial stereo effect. In the case of a service such as BIFS, interaction with a user may provide effects such as increasing and decreasing the depth of a desired object and adjusting brightness or color of the object as needed.

Meanwhile, in the audio field, interest in stereoscopic sound has recently increased due to the development of digital audio signal processing technology. The three-dimensional sound refers to a sound in which spatial information is added to the sound so that a listener who is not located in a space where a sound source is generated can perceive a sense of direction, distance, and space when the sound is heard. If the stereoscopic sound reproduction apparatus is used, it is possible to obtain the same effect as if the corresponding sound is heard in the field even when not in the field. The stereophonic sound may be variously applied to games, virtual reality, and multimedia contents including movies. In addition, in all fields that require sound, such as teleconference, realistic acoustic communication, distance education and broadcasting, and defense using a communication network, stereoscopic sound technology is recognized as an important technology to increase the realism and immersion.

Stereoscopic sound technology is a collective name for creating realistic stereoscopic sound, which includes sound image localization, sound field control, effect sound generation, crosstalk removal, etc. It may be divided into externalization and virtual surround technology.

The sound image positioning method refers to a technique of positioning a sound source at a desired virtual position, and the sound field control method is a technique for virtually creating a space of a sound source such as a concert hall or a recording studio. In addition, the sound effect control technology generates sound effects such as a melody or chorus, and the virtual surround technology is a method of reproducing sound of 4 or 5.1 channels in a 2-channel speaker environment.

The stereo sound gives the spatial arrangement information to the auditory, and as a result, the stereoscopic image information can be identified, thereby obtaining the sound source position, the surrounding environment, the position of the listener, the movement of the object, the shape of the sound image, and the like. The stereo sound is divided into a multichannel surround method and a binaural two channel stereo method according to a reproduction method.

In general, the transmission path of sound is a spatial transmission system that causes phenomena such as reflection, diffraction, and scattering by materials, and reflection and diffraction by the human brain and ears. It is divided into head transmission system that causes phenomenon such as resonance. The main factors that perceive the spatiality of the sound transmitted to the ear are the time difference, the level difference, and the difference between the two sounds reaching the ears.

The binaural type refers to a signal recorded by installing microphones on both ears of a listener in a space where a sound source is generated, and when the signal is reproduced through headphones, a sound image is reproduced as if directly heard in the field. can do. Here, the sound source refers to the position of the object that actually generates the sound, and the sound image refers to the sound source of the sensory sense perceived by humans. The spatial characteristics of the sound source and the sound image do not necessarily coincide with each other, and the more the sound source and the sound image match, the better sound quality stereoscopic sound is realized. The binaural signal includes not only the position and direction of the sound source but also spatial cues related to the space surrounding the sound source, that is, the sound field.

Stereophonic sound reproduction method can be divided into stereophonic sound reproduction method by two channels and stereophonic sound reproduction method by multiple channels according to the number of channels to be reproduced. This technology refers to the technology of reproducing the stereoscopic sound generated by the sound image control and the sound field control in the two-channel headphone environment or the two speaker environment using the perceptual characteristics. Most of the stereoscopic sound technologies developed to date are two-channel reproduction methods.

The binaural type two-channel stereophonic sound generation method is divided into a recording method and a filter processing method which is widely used recently. In the recording method, a microphone is inserted into both ears of a listener to record a scene sound and play the same. The filter processing method is a method of reproducing stereo sound by passing mono filters through a head related transfer function (FRTF) and a room transfer function (RTF). The head transfer function generates impulse signals such as white noise from speakers arranged at various angles in the form of spheres around the dummy head in an anechoic chamber, and generates impulse responses measured by microphones mounted on both ears of the dummy head. Say what you have done. The head transfer function corresponding to the desired position can be selected from this DB, and the sound image can be positioned at the corresponding position through the simple sound source and the convolution operation.

In the multi-channel surround system, 5.1 channels of the Dolby Digital system are taken as examples, and the stereo, stereo, stereo, and subwoofers are generally provided as shown in FIG. 1. Such a system uses a psychoacoustic phenomenon called auditory masking, which is called perceptual digital audio coding, which is mainly used in the film field, and is also used as a multichannel sound.

According to the present invention, by selecting a stereoscopic image region corresponding to a sound source of stereoscopic audio using linkage information between stereoscopic audio and stereoscopic images, it is possible to match audio with a specific object located in the stereoscopic image rather than simply synchronizing audio and stereoscopic images. It is an object of the present invention to provide a method and apparatus.

In addition, the present invention provides a method and apparatus that can provide a more realistic stereoscopic audio and stereoscopic video services by matching stereoscopic audio with a specific object in the stereoscopic image to give a stereoscopic image effect corresponding to the audio to the object. There is a purpose.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention, which are not mentioned above, can be understood by the following description, and more clearly by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

In order to achieve the above object, the present invention provides a method of reproducing stereoscopic audio and stereoscopic images, the method comprising: acquiring stereoscopic audio and stereoscopic images associated with the stereoscopic audio, acquiring linkage information between stereoscopic audio and stereoscopic images, and interlinking information; Selecting a region corresponding to a predetermined sound source of stereoscopic audio in the stereoscopic image to be reproduced, wherein the linkage information is identification information for each sound source of stereoscopic audio and identification of each region included in the stereoscopic image. It is characterized by including information.

The present invention also provides a stereo audio and stereoscopic image reproducing apparatus, comprising: an audio and stereoscopic image acquisition unit for acquiring stereoscopic images linked to stereoscopic audio and stereoscopic audio; A stereoscopic image processing unit for selecting a region corresponding to a predetermined sound source of stereoscopic audio in the stereoscopic image to be reproduced by using the interlinked information, wherein the interlinked information is included in the identification information and the stereoscopic image of each sound source of the stereoscopic audio; It is another feature to include identification information of each area.

In addition, in the stereoscopic audio and stereoscopic image providing method, the present invention provides a method for transmitting stereoscopic audio and stereoscopic audio associated with stereoscopic audio, and stereoscopic audio for selecting an area corresponding to a predetermined sound source of stereoscopic audio when stereoscopic video is played back. And transmitting linkage information between stereoscopic images, wherein the linkage information includes identification information of each sound source of stereoscopic audio and identification information of each region included in the stereoscopic image.

In addition, the present invention is a stereoscopic audio and stereoscopic image providing apparatus, stereoscopic audio, stereoscopic audio associated with stereoscopic audio, stereoscopic audio and stereoscopic image for selecting a region corresponding to a predetermined sound source of stereoscopic audio when stereoscopic video is reproduced And a data transmitter for transmitting the association information, wherein the association information includes identification information of each sound source of stereoscopic audio and identification information of each region included in the stereoscopic image.

According to the present invention as described above, by selecting the stereoscopic image region corresponding to the sound source of the stereoscopic audio using the linkage information between the stereoscopic audio and the stereoscopic image, a specific object located in the stereoscopic image, not merely to synchronize the audio and stereoscopic image. There is an advantage that can match audio with.

In addition, the present invention has the advantage of providing a more realistic stereoscopic audio and stereoscopic video services by matching the stereoscopic audio and a specific object in the stereoscopic image to give the stereoscopic image effect to the corresponding object.

The above objects, features, and advantages will be described in detail with reference to the accompanying drawings, and thus, those skilled in the art may easily implement the technical idea of the present invention. In describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

The present invention relates to a technique for selecting a region corresponding to a predetermined sound source of stereoscopic audio in a stereoscopic video to be reproduced by using stereoscopic audio and linkage information of stereoscopic video associated with the stereoscopic audio. The linkage information includes identification information of each sound source of stereoscopic audio and identification information of each region included in the stereoscopic image. For example, when viewing a symphony concert scene played by an orchestra using stereoscopic audio, when a performance of a particular part of the orchestra (for example, a violin) is highlighted or enlarged, a specific region or object of the stereoscopic image corresponding thereto ( For example, the effect of changing the color of the violin player) or increasing the three-dimensional feeling is provided.

To this end, the linkage information includes identification information of each sound source of stereoscopic audio and identification information of each region or object included in the stereoscopic image. Further, through linkage information between the included sound source identification information and the region or object identification information of the stereoscopic image, it is possible to determine which region or object of the stereoscopic image corresponding to the corresponding sound source is played.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

1 is a view for explaining a method of reproducing and providing stereoscopic audio and stereoscopic video according to an embodiment of the present invention.

As shown in FIG. 1, a viewer can watch a scene 104 where an orchestra plays music through a 5.1 channel audio environment 102. At this time, the orchestra playing screen 104 may be a stereoscopic image composed of three-dimensional objects.

In FIG. 1, since the performance screen 104 is a three-dimensional image composed of three-dimensional objects, a three-dimensional effect is given to each object, for example, every object displayed on the screen such as a player or a musical instrument. Therefore, the viewer can three-dimensionally view the position of each player as in a real performance hall. In addition, the 5.1-channel audio environment 102 allows viewers to enjoy stereoscopic sound as if they were in a real venue. The process of reproducing and viewing such audio and stereoscopic images is conventional, and is merely to reproduce stereoscopic audio and stereoscopic images associated therewith.

In an embodiment of the present invention, instead of simply playing audio and stereoscopic images as in the related art, when a specific sound source of stereoscopic audio is emphasized, a stereoscopic image object corresponding to the stereoscopic image object is emphasized so that a viewer can enjoy a more realistic screen. do.

If the right audio is emphasized or the sound becomes loud as shown in FIG. 1 during the stereoscopic image reproduction in the 5.1-channel audio environment 102, the corresponding region, that is, the right rear part of the screen, is selected in the playing scene 104. . When the area is selected in this way, the three-dimensional effect is adjusted so that the area is more prominent than other areas. For example, when the playing scene 104 is a stereoscopic image composed of one or more objects and a depth map, the corresponding area (or object) may be more highlighted by increasing or decreasing the depth value.

In order to provide such an effect, another embodiment of the present invention may use a stereoscopic image composed of a texture and a depth image. Here, the depth stereoscopic image is a stereoscopic image providing a depth sense in space, and generally expresses the depth sense using a depth value of 0 to 255. The depth value can be adjusted by the user or an application program in the terminal, and through this, the depth can be changed as desired. By providing such depth control in conjunction with stereoscopic audio, the user can more realistically recognize a stereoscopic image area (or object) corresponding to a predetermined sound source of stereoscopic audio.

In another embodiment of the present invention, instead of adjusting the depth of the selected area, a method of reliably recognizing the corresponding area to the viewer by adjusting the color or contrast of the corresponding area may be used.

In one embodiment of the present invention, the 3D object constituting the stereoscopic image is composed of a 3D object that can adjust the depth (depth) or disparity (disparity). Here, the 3D object includes a 3D image object (eg, JPEG, PNG, or MNG format) that supports an alpha map, including a 3D video having an arbitrary size. The 3D objects constituting the stereoscopic image are located in a 3D space on the screen, and each 3D object is associated with a specific audio channel (or sound source).

According to an embodiment of the present invention, when stereoscopic audio is played, the depth map of the 3D object corresponding to a predetermined sound source of the audio is adjusted to enlarge or reduce the depth of the 3D object, thereby increasing the stereoscopic audio. Can provide an effect. Here, the adjustment means changing the depth by adjusting the depth map or disparity value.

In order to select a predetermined sound source of stereoscopic audio and a region of a reproduction stereoscopic image corresponding thereto, linkage information between stereoscopic image and audio is required. In one embodiment of the present invention, the meta information between the 3D object and the stereoscopic audio is used as the linking information. This meta information may include information such as a meta information identifier, identifiers of 3D objects and audio channels (or audio objects), depth adjustment information, adjustment methods, depth effects, event start time, event duration and event end time. .

First, the meta information identifier is an identifier for identifying data (packets, characters and codes capable of providing identification information including a file) constituting the meta information.

The object identifier is information for identifying the associated audio and stereoscopic image. If the MPEG-2 system is used, the object identifier can be an ES ID. If the MPEG-2 system is not used, the object identifier uses a separate CRDI, URL, and URI. Stereoscopic images can be identified.

The depth adjustment information may be composed of an adjustment value, a minimum value, and a maximum value, and the adjustment value means a basic value for adjusting the existing depth map. The minimum value means the minimum value to be guaranteed when adjusting the depth value, and the maximum value means the maximum value to be guaranteed when adjusting the depth value.

The depth adjustment method is a method for reflecting the depth adjustment value, and generally, a method such as addition, multiplication, linear increase, or linear decrease may be applied. If the adjustment method is added, the depth of the corresponding area may be adjusted by adding the depth adjustment value to the value of the existing depth map. When the depth adjustment method is a linear increase, the depth adjustment value may be linearly increased from the existing value, and conversely, in the case of a linear decrease, the adjustment value may be linearly decreased from the existing value.

The event start time means a start time to which the depth adjustment of the 3D object of the reproduced stereoscopic image corresponding to the predetermined sound source of the corresponding audio is applied. As a time display method, UTC (Universal Time coordinated) value can be used for general broadcasting, and Internet standard time can be applied for Internet. In addition, the event duration refers to the duration in which the depth adjustment of the 3D object is applied, and the time when the depth adjustment of the 3D object ends.

Such meta information may be provided in various forms depending on the application. For example, when stored as a file through a storage medium such as a DVD, it may be provided as a separate track. When provided through a broadcasting network, it may be composed of specific metadata and transmitted as a stream or may be provided as meta information in a video stream. In the case of a communication network through the Internet, it may be separately provided as an individual file, and may be included in separate metadata, video, or audio stream in the same manner as a broadcasting network providing method. That is, in the preferred embodiment of the present invention, the linkage information may be provided in one integrated file format together with the stereoscopic audio and the stereoscopic image, or may be provided as a separate file from the stereoscopic audio and the stereoscopic image.

Until now, the stereoscopic audio and stereoscopic image provision according to an embodiment of the present invention have been described with reference to FIG. 1. In FIG. 1, an embodiment of the present invention has been described through a specific 3D stereoscopic image for convenience, but the present invention is not limited to such a specific stereoscopic image.

2 is a block diagram showing the configuration of a stereoscopic audio and stereoscopic image reproducing apparatus according to an embodiment of the present invention.

In FIG. 2, the stereoscopic audio and stereoscopic image reproducing apparatus 202 includes an audio and stereoscopic image acquisition unit 204, a linkage information acquisition unit 206, and a stereoscopic image processing unit 208.

The stereoscopic audio and stereoscopic image reproducing apparatus 202 serves to acquire stereoscopic audio and stereoscopic images associated with the stereoscopic audio. Such audio and stereoscopic images may be obtained in real time through a broadcasting network or a communication network, or may be obtained from a file previously stored by a recording medium or the like. That is, the acquisition of audio and stereoscopic images can be made through any method or medium.

The linkage information acquisition unit 204 acquires linkage information between the stereoscopic audio and the stereoscopic image obtained by the stereoscopic audio and the stereoscopic image reproducing apparatus 202. The association information may be obtained through various methods or media as well as audio and stereoscopic images. For example, when audio and stereoscopic images are stored in a DVD recording medium, the linkage information acquisition unit 204 may acquire linkage information (eg, metadata) stored in a track separate from the audio and stereoscopic images. .

The stereoscopic image processing unit 208 serves to select a region corresponding to a predetermined sound source of stereoscopic audio in the stereoscopic image to be reproduced using the interlocking information acquired by the interlocking information acquisition unit 204. As mentioned above, the association information includes identification information of each sound source of stereoscopic audio and identification information of each region (or object) included in the stereoscopic image. In addition, information about which sound source corresponds to which stereoscopic image area stored by a content provider or an editor may also be included. The stereoscopic image processing unit 208 selects a region corresponding to a predetermined sound source of stereoscopic audio from the reproduced stereoscopic image using the linkage information, and adds stereoscopic information (including depth information or disparity information) or color sense for the selected region. Adjust the contrast.

Here, the linkage information includes not only the identification information of the audio and stereoscopic images mentioned above, but also adjustment values for performing adjustment of stereoscopic information, color or contrast of the selected area, and time information (such as adjustment start time, Adjustment execution time, adjustment end time), and the like.

On the other hand, although not shown in Figure 2, in order to reproduce the stereoscopic audio and stereoscopic images according to an embodiment of the present invention, such stereoscopic audio and stereoscopic images should be provided. The stereoscopic audio and stereoscopic image providing apparatus according to an embodiment of the present invention is stereoscopic audio, stereoscopic image associated with the stereoscopic audio, stereoscopic image for selecting a region corresponding to a predetermined sound source of stereoscopic audio when the stereoscopic image is reproduced. It may include a data transmission unit for transmitting the linkage information between the audio and stereoscopic image.

In addition, the stereoscopic audio and stereoscopic image providing apparatus according to an embodiment of the present invention may generate stereoscopic audio / video scene configuration data and transmit the stereoscopic audio / video scene configuration data through the data transmission unit. The linkage information according to an embodiment of the present invention relates to one stereoscopic image linked to one stereoscopic audio, and a plurality of objects are usually present in the stereoscopic image. Therefore, it is possible to generate scene configuration data for a specific scene using at least one stereoscopic audio, at least one stereoscopic image, and at least one associated information. For example, in a particular scene, if an image object A 'is associated with an audio object called A and an image object B' is associated with an audio object called B, the A and B audio objects are simultaneously played in the scene. The depth of A 'and B' can be adjusted simultaneously. In this way, it is possible to integrate a plurality of stereoscopic audio, stereoscopic images, and linkage information for a specific scene to generate and transmit one stereoscopic audio / video scene configuration data.

3 is a flowchart illustrating a process of stereoscopic audio and stereoscopic image reproduction according to an embodiment of the present invention.

First, stereoscopic audio and a stereoscopic image associated with the stereoscopic audio are obtained (302). As mentioned above, stereoscopic audio and stereoscopic images can be obtained by any method or medium. In addition to acquiring stereoscopic audio and stereoscopic images, linkage information between stereoscopic audio and stereoscopic images is acquired (304).

Then, using the obtained linkage information, a region (or object) corresponding to a predetermined sound source of stereoscopic audio is selected in the stereoscopic image to be reproduced (302). By adjusting stereoscopic information (depth or disparity information), color or contrast of the selected area, it is possible to provide a viewer with a more realistic stereoscopic image of the corresponding sound source. In the case of 3D information, by increasing or decreasing the adjustment value of the depth or disparity information, the corresponding region or object can be made to protrude or appear more. Also, by changing the color, the area can be seen to be distinctly different from other areas or objects around it. Alternatively, the same effect can be obtained by adjusting the contrast of the corresponding area.

4 is a flowchart illustrating a stereoscopic audio and stereoscopic image providing process according to an exemplary embodiment of the present invention.

First, stereoscopic audio and stereoscopic images associated with the stereoscopic audio are transmitted (402). When the stereoscopic image is reproduced, linkage information between the stereoscopic audio and the stereoscopic image is selected (404) for selecting an area corresponding to a predetermined sound source of stereoscopic audio. Here, the association information includes identification information of each sound source of stereoscopic audio and identification information of each region included in the stereoscopic image.

Then, the stereoscopic audio / video scene configuration data is generated using the plurality of stereoscopic audio, stereoscopic images, and associated information (406), and the generated stereoscopic audio / video scene configuration data is transmitted (408).

According to the present invention, by selecting the stereoscopic image area corresponding to the sound source of the stereoscopic audio using the linkage information between the stereoscopic audio and the stereoscopic image, it is not only to synchronize the audio and stereoscopic image, but also to locate a specific object and audio located in the stereoscopic image. There is an advantage to match.

Further, according to the present invention, by matching a stereoscopic audio with a specific object in the stereoscopic image, a stereoscopic image effect corresponding to the audio is provided to the corresponding object, thereby providing a more realistic stereoscopic audio and stereoscopic video service.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by.

1 is a view for explaining a method of playing and providing stereoscopic audio and stereoscopic images according to an embodiment of the present invention.

2 is a block diagram showing the configuration of a stereoscopic audio and stereoscopic image reproducing apparatus according to an embodiment of the present invention.

3 is a flowchart illustrating a process of stereoscopic audio and stereoscopic image reproduction according to an embodiment of the present invention.

4 is a flowchart illustrating a stereoscopic audio and stereoscopic image providing process according to an exemplary embodiment of the present invention.

Claims (14)

Obtaining stereoscopic audio and stereoscopic images associated with the stereoscopic audio; Obtaining linkage information between the stereoscopic audio and the stereoscopic image; And Selecting an area corresponding to a predetermined sound source of the stereoscopic audio from the stereoscopic image to be reproduced using the linkage information; The link information includes identification information of each sound source of the stereoscopic audio and identification information of each region included in the stereoscopic image. The method of claim 1, Adjusting stereoscopic information of the selected area; And the stereoscopic information is depth information or disparity information of the stereoscopic image. The method of claim 1, And adjusting the color or contrast of the selected area. The method according to claim 1 or 2, The linkage information is And at least one of an adjustment value for adjusting the stereoscopic information, color or contrast, and time information for adjusting the stereoscopic information, color or contrast. An audio and stereoscopic image acquisition unit for acquiring stereoscopic audio and stereoscopic images associated with the stereoscopic audio; A linkage information acquisition unit for acquiring linkage information between the stereoscopic audio and the stereoscopic image; And A stereoscopic image processing unit which selects a region corresponding to a predetermined sound source of the stereoscopic audio from the stereoscopic image to be reproduced using the linkage information; The link information includes identification information of each sound source of the stereoscopic audio and identification information of each region included in the stereoscopic image. The method of claim 5, The stereoscopic image processing unit Adjust stereoscopic information of the selected area, And the stereoscopic information is depth information or disparity information of the stereoscopic image. The method of claim 5, The stereoscopic image processing unit And adjusting the color or contrast of the selected area. The method according to claim 6 or 7, The linkage information is And at least one of an adjustment value for adjusting the stereoscopic information, color or contrast, and time information for adjusting the stereoscopic information, color or contrast. Transmitting stereoscopic audio and stereoscopic images associated with the stereoscopic audio; And Transmitting linking information between the stereoscopic audio and the stereoscopic image for selecting an area corresponding to a predetermined sound source of the stereoscopic audio when the stereoscopic image is reproduced, The link information includes identification information of each sound source of the stereoscopic audio and identification information of each region included in the stereoscopic image. The method of claim 9, And the stereoscopic audio, the stereoscopic image, and the linkage information are configured as a single unified file. The method of claim 9, The link information comprises a stereo audio and a stereoscopic image and a separate file, stereoscopic audio and stereoscopic image providing method. The method of claim 9, Generating stereoscopic audio / video scene configuration data using the stereoscopic audio, the stereoscopic image, and the linkage information; And Transmitting the stereoscopic audio / video scene composition data. Stereo audio and stereoscopic image providing method further comprising. 3D audio, a stereoscopic image associated with the stereoscopic audio, and a data transmission unit configured to transmit linkage information between the stereoscopic audio and the stereoscopic image for selecting a region corresponding to a predetermined sound source of the stereoscopic audio when the stereoscopic image is reproduced. and, The link information includes identification information of each sound source of the stereoscopic audio and identification information of each region included in the stereoscopic image. The method of claim 13, The apparatus may further include a scene configuration unit configured to generate stereoscopic audio / video scene configuration data using the stereoscopic audio, the stereoscopic image, and the linkage information. And the data transmission unit transmits the stereoscopic audio / video scene configuration data.

Images