KR101534295B1 - Method and Apparatus for Providing Multiple Viewer Video and 3D Stereophonic Sound - Google Patents

Abstract

Provided are a method and an apparatus for providing a multi-viewer image and 3D stereophonic sound. According to an embodiment of the present invention, a stereophonic image having one or more corresponding objects disposed in a virtual space, and a multi-channel image are displayed in a predetermined channel display area, a multi-channel sound signal corresponded to the multi-channel image is outputted, and generation of a reconfigured stereophonic image and outputting of each sound signal included in the multi-channel sound signal are controlled to be outputted based on changed information in case that the changed information for a corresponding object is recognized, thereby exhibiting 3D spatial sensation, and providing surround effect having 3D effect and a realistic sensation without a separate extra speaker for realizing 3D sound.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for providing a multi-viewer image and a 3D stereophonic sound,

The present embodiment relates to a method and apparatus for providing a multi-viewer image and 3D stereo sound.

It should be noted that the following description merely provides background information related to the present embodiment and does not constitute the prior art.

After the phonograph, the first acoustical player called gramophone, was invented, the sound system is a digital recorder-type CD, through a record player method that reproduces the sound of an LP (long play) A digital record player method that reproduces the sound of the digital audio signal.

1 is a block diagram of a conventional acoustic equipment system. The conventional acoustic equipment system includes a wired microphone 10 for recording a performance sound source, wireless microphones 11A and 11B, an antenna splitter 12 connected to wireless microphones 11A and 11B via antennas ANT, (11A, 11B) An audio mixer connected to a wireless receiver (13), a wireless receiver (13) and a wired microphone (10) for converting an analog sound signal recorded for each channel into a digital sound signal, A speaker selector 15 for selecting a speaker, a speaker controller 16 for adjusting the volume of the selected speaker, a power amplifier 17 for amplifying a sound signal, a speaker 18 for outputting an amplified sound signal, A graphic equalizer 19 for adjusting the frequency level so that the recorded sound signal is close to the original sound, a power amplifier 20 for amplifying the sound signal whose frequency level is adjusted, and a speaker 21 for outputting the amplified sound signal The.

In recent years, as the technology of a sound system has been developed, stereo sound technology has been researched and developed in which a sound reproduced in a sound system is implemented as if it is heard in a field. In addition, with the launch of 3DTV, 3D stereoscopic images, which provide a sense of realism and immersion, are activated as in the case of 3D movies. However, in the case of a conventional sound system, more than two speakers are required to implement 3D sound (3D sound), and binaural three-dimensional audio using a head related transfer function (HRTF) There is a problem in that it can not properly implement the high reduction of the sound, the sense of distance, and the sense of space.

Accordingly, there is a need for a technique capable of realizing 3D sound without providing additional speaker for realizing 3D sound in a sound system.

In this embodiment, a stereoscopic image and a multi-channel image in which one or more corresponding objects are arranged in a virtual space are displayed in a predetermined channel display area and a multi-channel sound signal is output. When variation information about a corresponding object is recognized, And outputs a sound signal of each channel included in the multi-channel sound signal to output a 3D spatial feeling, and a separate additional speaker for realizing 3D sound is provided And to provide a method and an apparatus for providing a multi-viewer image and a 3D stereophonic sound that can provide a stereoscopic and realistic surround effect without providing a stereoscopic effect.

A 3D renderer that receives one or more object data and generates and provides a stereoscopic image in which one or more corresponding objects corresponding to the one or more object data in the virtual space are arranged; A video output unit receiving the multi-channel image and the stereoscopic image, and displaying at least one of the multi-channel image and the stereoscopic image in a predetermined channel display area; An audio output unit receiving and outputting a multi-channel sound signal corresponding to the multi-channel image; And controlling the output of each sound signal included in the multi-channel sound signal based on the variation information when the variation information for one of the one or more corresponding objects is recognized, And a stereoscopic sound controller for outputting a 3D stereophonic sound.

According to another aspect of the present invention, there is provided a stereoscopic image display apparatus including a data processing device that receives one or more object data, generates a stereoscopic image in which one or more corresponding objects corresponding to the one or more object data in the virtual space are arranged, Provision process; A display step of receiving a multi-channel image and the stereoscopic image and displaying at least one of the multi-channel image and the stereoscopic image in a preset channel display area; An outputting step of receiving and outputting a multi-channel sound signal corresponding to the multi-channel image; And a control step of controlling the output of each sound signal included in the multi-channel sound signal based on the variation information when variation information on any one of the one or more corresponding objects is recognized, A computer-readable recording medium on which a program is recorded.

According to the present embodiment, the multi-viewer image and the 3D stereophonic sound player display the stereoscopic image and the multi-channel image in which one or more corresponding objects in the virtual space are arranged in a predetermined channel display area, When the variation information for the corresponding object is recognized, the output of each sound signal included in the reconstructed stereoscopic image and the multi-channel sound signal reconstructed from the stereoscopic image is adjusted and output based on the variation information to display 3D spatial feeling, It is possible to provide a stereoscopic and realistic surround effect without providing a separate additional speaker for realizing the surround effect.

1 is a block diagram schematically showing a conventional acoustic equipment system.
FIG. 2 is a block diagram schematically illustrating a multi-viewer image and a 3D stereophonic sound player system according to the present embodiment.
3 is a flowchart illustrating a method of providing a multi-viewer image and a 3D stereo sound according to the present embodiment.
4 is an exemplary diagram for explaining a time difference and a difference in intensity of a sound pressure of sound waves heard in two ears based on a sound source in a human auditory system.
5 is a diagram for explaining a method of displaying a multi-channel image and a stereoscopic image according to the present embodiment in a channel display area.
6 is a diagram illustrating an example of a stereoscopic image according to the present embodiment.
7 is a diagram illustrating an example of a reconstructed stereoscopic image according to the present embodiment.
8 to 14 are views illustrating an example of a multi-viewer image implemented using a multi-viewer image and a 3D stereophonic sound player according to an exemplary embodiment of the present invention.

Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.

FIG. 2 is a block diagram schematically illustrating a multi-viewer image and a 3D stereophonic sound player system according to the present embodiment.

The multi-viewer image and 3D stereophonic player system according to the present embodiment shown in FIG. 2 includes a transmission server, a multi-viewer image, and a 3D stereophonic player apparatus. Here, the components included in the multi-viewer image and the 3D stereophonic player system are not limited thereto.

The transmission server collects video, sound, and object data, and transmits the collected video, sound, and object data to a multi-viewer video and 3D stereo player device (hereinafter, referred to as a player device). 1, a multi-viewer image and a 3D stereophonic sound player system are provided with a transmission server to transmit image, sound, and object data collected by a player device, but the present invention is not limited thereto. For example, the player device may be provided with video, sound, and object data stored from discs and music played through a playback device (not shown) in the player device. The transmission server includes at least one camera C1, C2, C3, a video encoder 100, an image packetizer 101, at least one microphone M1, M2, M3, M4, M5, an audio encoder 102, A speech packetizer 103, a multiplexer 104, a data encoder 107, and an OD / BIFS generator 108. [

That is to say, the transmission server receives a half (2/4) picture taken by Live through a plurality of cameras, a video encoder 100 for encoding an image, a plurality of microphones, an audio encoder 102 for encoding sound, and an OD / BIFS generator 108. / 3/4 channel image, multi-channel sound, and object data, multiplexes it by the multiplexer 104, and transmits it to the player device. On the other hand, the object data transmitted from the OD / BIFS generator 108 includes information on objects placed in the stereoscopic image generated through the 3D renderer of the player device, and binary format information therefor. At this time, the OD / BIFS generator 108 defines a two-dimensional or three-dimensional binary format for the object using MPEG4 Binary Format for Scene (BIFS).

Meanwhile, the multi-viewer image and the 3D stereophonic sound player system according to the present embodiment may use MPEG-4 as a video codec and use an MPEG audio codec or an MP3 / AAC codec as an audio codec. In addition, the multi-viewer video and 3D stereophonic player system can use a mono or stereo codec, MPEG-4 AAC (Advanced Audio Coding) v2 or MPEG-D USAC (Unified Speech and Audio) to compress and restore individual channel signals or object signals with high efficiency / Audio Coding) can be used. It can be used in a form that is not separated from the A / V decoder in order to reproduce 3D Audio, and various channel conversion tools standardized in MPEG Surround or SAOC (Spatial Audio Object Coding) can be used Do.

The multiviewer image and the 3D stereophonic sound player apparatus according to the present embodiment (hereinafter, referred to as a player apparatus) output and provide the collected multi-channel image, multi-channel sound signal, and object data, And displays the multi-channel image and the stereoscopic image in at least one or more channel display areas set in advance, thereby providing a multi-viewer image. In addition, the player apparatus generates and provides a reconstructed stereoscopic image reconstructing the stereoscopic image based on the variation information when the variation information for the corresponding object in the stereoscopic image is recognized, By outputting the output, it is possible to display the 3D spatial feeling to the user and to provide a stereoscopic and realistic surround effect.

The player apparatus includes a demultiplexer 200, a channel distributor 201, a demultiplexer 202, a video decoder 203, an audio decoder 204, a 3D renderer 207, a stereo sound adjuster 208, 209). Here, the constituent elements included in the player apparatus are not necessarily limited thereto.

The demultiplexer 200 receives a packet transmission stream transmitted from a transmission server using a wired / wireless communication network, and demultiplexes the received packet transmission stream to extract a video stream, a voice stream, and object data. Thereafter, the demultiplexer 200 transmits the extracted video stream, voice stream, and object data to the channel distributor 201.

The channel distributor 201 distributes the multi-channel audio signal included in the video stream, the multi-channel audio signal included in the audio stream, and the multi-channel audio signal included in the video stream to the at least one video channel, the 3D renderer 207 and the audio channel, And distributes the object data. Meanwhile, the player apparatus according to the present embodiment provides the channel unit 201 with one channel, two channels, and four channels by using the display unit 209 so that the channel distributor 201 can receive And distributes the multi-channel image included in the video stream to at least one or more video channels according to the channel selection information.

The multi-channel video and multi-channel audio signals distributed using the channel distributor 201 are tee-packetized using the depacketizer 202 and are decoded by the video decoder 203 and the audio decoder 240, And is output to the display unit 209. A detailed description of the object data distributed to the 3D renderer 207 will be described later in the process of describing the 3D renderer 207. [

The 3D renderer 207 receives one or more object data distributed from the channel distributor 201 and generates and provides a stereoscopic image in which one or more corresponding objects corresponding to one or more object data in the implemented virtual space are arranged. On the other hand, the object data includes information on objects placed in the stereoscopic image and two-dimensional to three-dimensional binary format information. That is, the 3D renderer 207 arranges one or more three-dimensional corresponding objects corresponding to one or more object data based on the object data at predetermined positions in the three-dimensional virtual space (X, Y, Z) . Then, the stereoscopic image generated using the 3D renderer 207 is transmitted to the display unit 208, and the display unit 209 displays the stereoscopic image together with the multi-channel image to the user. Meanwhile, the at least one corresponding object disposed in the stereoscopic image is implemented in a form in which the motion can be adjusted according to the touch signal and the mouse signal input from the user, whereby the stereoscopic image is output as the output of each sound signal included in the multi- As a user interface (UI) for controlling the operation of the system.

Meanwhile, a corresponding object disposed in the stereoscopic image generated using the 3D renderer 207 according to the present embodiment includes a listener object and a plurality of virtual speaker objects. At this time, the listener object can be designed as a dummy head modeled as a human head and ear structure. In addition, the number of the plurality of virtual speaker objects may be determined according to the number of channels of the multi-channel sound signal included in the sound stream. For example, the number of virtual speaker objects is preferably equal to the number of channels of the multi-channel sound signal, but the present invention is not limited thereto. In the present embodiment, the number of virtual speaker objects arranged in the stereoscopic image is not limited to a specific number.

The 3D renderer 207 generates and provides a reconstructed stereoscopic image in which variation information is reflected on a stereoscopic image when variation information on one of the one or more corresponding objects implemented in the stereoscopic image is recognized. That is, the 3D renderer 207 generates and provides a reconstructed stereoscopic image reconstructed based on the variation information, the size and size of the virtual space, the arrangement, arrangement type and size of the corresponding object, and the like. Accordingly, the 3D renderer 207 can provide a more realistic 3D spatial feeling to the user.

The 3D renderer 207 generates and provides a reconstructed stereoscopic image in which the placement of the listener object in the virtual space is reconstructed based on the placement variation information when the placement variation of one or more corresponding objects implemented in the stereoscopic image, do. Meanwhile, the 3D renderer 207 may generate a reconstructed stereoscopic image in which the arrangement form of the plurality of virtual speaker objects is further changed based on the arrangement variation information about the listener object. For example, the 3D renderer 207 may determine the layout of a plurality of virtual speaker objects such that a plurality of virtual speaker objects are oriented in the direction in which the listener object moved from the front is located, based on the layout variation information for the listener object A reconstructed stereoscopic image in which the angle is further varied may be generated.

The 3D renderer 207 may be configured such that when a variation in the arrangement form of at least one virtual speaker object among a plurality of virtual speaker objects is recognized, a layout form of the virtual speaker object in which the variation occurs based on the arrangement form variation information is re- And generates and provides an image. At this time, the arrangement form of the virtual speaker object is preferably, but not necessarily, the arrangement angle of the virtual speaker object. Likewise, when the variation of the placement of at least one virtual speaker object among a plurality of virtual speaker objects is recognized, the 3D renderer 207 acquires the reconstructed stereoscopic image in which the placement of the virtual speaker object, And provide them.

Likewise, the reconstructed stereoscopic image reconstructed using the 3D renderer 207 is transmitted to the display unit 209. The display unit 209 reflects this in real time and displays it to the user, thereby providing a more realistic 3D spatial feeling to the user There is an effect that can be done.

Meanwhile, the 3D renderer 207 may include an individual volume controller for individually adjusting the volume of the sound signal corresponding to the plurality of virtual speaker objects in the stereoscopic image, a volume controller for simultaneously adjusting the volume of the sound signal corresponding to the plurality of virtual speaker objects An EQ controller for adjusting a frequency of a sound signal corresponding to a plurality of virtual speaker objects, an adjustment button for controlling movement of a corresponding object, and a multi-channel sound signal output using a display unit 209 are recorded Some or all of the recording buttons for performing the recording may be further implemented. Therefore, the user of the player device can adjust the output of each sound signal included in the multi-channel sound signal based on the input signal inputted using the controller and buttons, as well as the variation information on the corresponding object implemented in the stereoscopic image have.

When the variation information for any one of the one or more corresponding objects disposed in the stereoscopic image generated using the 3D renderer 207 is recognized, the stereophonic adjuster 208 generates a multi-channel sound signal based on the variation information And adjusts and outputs the output of each included acoustic signal. At this time, it is preferable that the output of each acoustic signal adjusted using the stereo controller 208 is the amplitude of the acoustic signal, but it is not necessarily limited thereto. The stereo sound adjuster 208 is interlocked with the audio decoder 204 and the audio output unit 212 to receive a multi-channel sound signal and adjust the output of each sound signal included in the multi- have.

The stereo sound adjuster 208 may be configured to adjust the position of each of the sound signals included in the multi-channel sound signal based on the placement variation information when a variation in the placement of any one of the one or more corresponding objects disposed in the stereoscopic image, Adjust the output and output. In other words, the stereo sound adjuster 208 increases the output of the acoustic signal corresponding to the virtual speaker object closer to the listener object as the arrangement of the listener object changes, and the output of the acoustic signal corresponding to the more distant virtual speaker object And outputs it. On the other hand, if placement variations of the listener object occur, as noted above, the layout for the plurality of virtual speaker objects may also be further varied, wherein the stereo sound adjuster 208 not only includes placement variation information for the listener object And arranging variation information on a plurality of virtual speaker objects are further considered to control the output of each sound signal included in the multi-channel sound signal.

In addition, the stereophonic controller 208 may be configured such that, when a variation in the arrangement form of at least one virtual speaker object among a plurality of virtual speaker objects is recognized, And outputs it. That is, when it is determined that variation in the placement angle of any one virtual speaker object has occurred based on the placement form variation information, the stereo sound adjuster 208 generates a virtual speaker object corresponding to the virtual speaker object Adjusts the amplitude of the sound signal and outputs it. For example, the stereo sound adjuster 208 may cause the effect that the sound signal reproduced in the forward direction is reproduced in a direction corresponding to the varying arrangement angle by adjusting the amplitude of the sound signal corresponding to the virtual speaker object in which the variation occurs .

On the other hand, the stereo sound adjuster 208 may adjust the output of each sound signal included in the multi-channel sound signal, for example, the sound signals corresponding to the plurality of virtual speaker objects, based on the variation information about the corresponding object implemented in the stereoscopic image However, it is also possible to adjust the output of each sound signal included in the multi-channel sound signal based on the input signal of the regulator and the button, which are further implemented in the stereoscopic image.

The stereo sound adjuster 208 according to the present embodiment adjusts the output of each sound signal included in the multi-channel sound signal based on the variation information about the corresponding object implemented in the stereo image, It is possible to provide a stereoscopic and realistic surround effect without adding additional speakers.

The display unit 209 receives and outputs a multi-channel image, a stereoscopic image, and a multi-channel sound signal. The display unit 209 includes a video output unit 210, an audio output unit 211, and a synchronization unit 212.

The video output unit 210 receives the multi-channel image and the stereoscopic image, and displays at least one of the multi-channel image and the stereoscopic image in a predetermined channel display area. That is, the video output unit 210 displays at least one of a multi-channel image and a stereoscopic image in one channel display area of one channel, two channels, and four channels according to the channel selection information received from the user. For example, when the four channels are selected, the video output unit 210 displays a multi-channel image on three channels and a three-dimensional image on the remaining one channel, respectively. In addition, the video output unit 210 displays a multi-channel image on one channel and a stereoscopic image on the remaining one channel when two channels are selected. In addition, the video output unit 210 displays a stereoscopic image on one channel when one channel is selected.

Meanwhile, when the variation information for the corresponding object implemented in the displayed stereoscopic image is recognized, the video output unit 210 transmits the sensed variation information to the 3D renderer 207 and the stereo sound adjuster 208. Then, the video output unit 210 receives the reconstructed reconstructed stereoscopic image based on the variation information from the 3D renderer 207, and displays the reconstructed stereoscopic image in real time.

The audio output unit 211 receives and outputs a multi-channel sound signal corresponding to a multi-channel image. The audio output unit 211 according to the present embodiment outputs a received multi-channel sound signal using a multi-channel surround method and a binaural type 2-channel stereo method.

The audio output unit 211 receives the multi-channel sound signal whose output of each sound signal is controlled by the stereo sound controller 208 when the corresponding object implemented in the stereoscopic image changes, . Accordingly, the audio output unit 211 according to the present embodiment operates to provide a stereoscopic and realistic surround effect without providing an additional speaker for implementing 3D sound.

The synchronization unit 213 synchronizes and outputs the image displayed using the video output unit 210 and the sound signal output using the audio output unit 211. [ Similarly, when the corresponding object implemented in the stereoscopic image changes, the synchronization unit 213 outputs the reconstructed stereoscopic image output from the video output unit 210 and the output of each sound signal output from the audio output unit 211 And outputs the adjusted multi-channel sound signals in real time in synchronization. Accordingly, the user of the player apparatus according to the present embodiment can simultaneously receive a stereoscopic and multi-viewer image and stereoscopic sound having a realistic surround effect.

3 is a flowchart illustrating a method of providing a multi-viewer image and a 3D stereo sound according to an embodiment of the present invention.

A multi-viewer image and a 3D stereophonic sound player device (hereinafter, referred to as a player device) receive one or more object data, and generate stereoscopic images in which one or more corresponding objects corresponding to one or more object data in the virtual space are arranged (S302). In step S302, the player device arranges one or more three-dimensional corresponding objects corresponding to one or more object data based on the object data at predetermined positions in the (X, Y, Z) coordinates of the three-dimensional virtual space, And generates an image. The stereoscopic image functions as a UI for controlling the output of each sound signal included in the multi-channel sound signal. On the other hand, the corresponding object in the stereoscopic image generated in step S302 includes a listener object and a plurality of virtual speaker objects.

The player device receives the multi-channel image and the stereoscopic image, and displays at least one of the multi-channel image and the stereoscopic image in a preset channel display area (S304). In step S304, the player apparatus displays the multi-channel image collected from the transmission server or the music and the stereoscopic image generated using step S302 according to the channel selection information received from the user through one of channel 1, channel 2, and channel 4 Area. A method in which the player device displays multi-channel images and stereoscopic images in a predetermined channel display area is the same as described above, and a detailed description thereof will be omitted.

The player device receives and outputs the multi-channel sound signal corresponding to the multi-channel image (S306). On the other hand, the player apparatus synchronously outputs the image displayed using step S304 and the sound signal output using step S306.

The player apparatus determines whether the corresponding object placed in the stereoscopic image is changed (S308), and if the variation information about the corresponding object is recognized, a reconstructed stereoscopic image in which the variation information is reflected in the stereoscopic image is generated and provided (S310) . In step S310, the player device generates and provides a reconstructed stereoscopic image reconstructed based on the variation information, the size and size of the virtual space, the arrangement, arrangement and size of the corresponding object, and the like. Meanwhile, the method in which the player device generates the reconstructed stereoscopic image based on the variation information about the corresponding object is the same as described above, and a detailed description thereof will be omitted.

When the variation information for the corresponding object is recognized through the step S308, the player apparatus adjusts and outputs the output of each sound signal included in the multi-channel sound signal based on the variation information (S312). On the other hand, a method in which the player apparatus adjusts the output of each sound signal included in the multi-channel sound signal based on the variation information for the corresponding object is the same as described above, and a detailed description thereof will be omitted. That is, when the variation of the arrangement of any one of the one or more corresponding objects, for example, the listener object, in the stereoscopic image is recognized, the player device outputs the output of each sound signal included in the multi- And outputs it.

Further, when a variation in the arrangement form of at least one virtual speaker object among a plurality of virtual speaker objects is recognized, the player device adjusts the output of the sound signal corresponding to the virtual speaker object in which the variation occurs based on the arrangement form variation information And outputs it. Meanwhile, the player apparatus synchronously outputs the reconstructed stereoscopic image provided using step S310 and the multi-channel sound signal output using step S312, thereby providing a multi-viewer image and a 3D stereoscopic sound.

As described above, the method for providing the multi-viewer image and the 3D stereophonic sound described in FIG. 3 may be implemented by a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, Optical disk, a storage device, etc.).

4 is an exemplary diagram for explaining a time difference and a difference in intensity of a sound pressure of sound waves heard in two ears based on a sound source in a human auditory system.

There are interaural time difference (ITD) and interaural level difference (ILD) between two ears based on sound sources. Due to the nature of the human ear, the perception of bass and treble is different. In addition, the source of sound is called a sound source, and the intensity of the sound generated from the sound source becomes inversely proportional to the square of the distance, as the distance r becomes longer.

In order to reflect the characteristics of the auditory system, the multi-viewer image and the 3D stereophonic sound player according to the present embodiment generate and provide a stereoscopic image in which one or more corresponding objects in the virtual space are arranged, It is possible to provide stereoscopic and realistic surround effects by adjusting and outputting the output of each sound signal included in the multi-channel sound signal based on the variation information.

5 is a diagram for explaining a method of displaying a multi-channel image and a stereoscopic image according to the present embodiment in a channel display area.

As shown in FIG. 5, the multi-channel image and the stereoscopic image according to the present embodiment are displayed in at least one channel display area set according to the channel selection information received from the user. That is, the player apparatus according to the present embodiment provides one channel of one channel, two channels, and four channels so that it can be selected. For example, when four channels are selected, And displays stereoscopic images, respectively. When the two channels are selected, the player device displays a multi-channel image in one channel and a stereoscopic image in the remaining one channel, respectively. Further, when one channel is selected, the player device displays a stereoscopic image on one channel. Meanwhile, the stereoscopic image displayed in the channel display area functions as a UI for controlling the output of each sound signal included in the multi-channel sound signal.

6 is a diagram illustrating an example of a stereoscopic image according to the present embodiment. 6 (a) is an illustration of an example of a stereoscopic image according to the first embodiment, and FIG. 6 (b) is an example of an example of a stereoscopic image according to a second embodiment.

As shown in FIGS. 6A and 6B, the stereoscopic image according to the present embodiment is implemented in a form in which one or more corresponding objects corresponding to one or more object data in the virtual space implemented are disposed. That is, the player device generates a stereoscopic image by arranging one or more three-dimensional corresponding objects corresponding to one or more object data based on the object data at predetermined positions in the (X, Y, Z) coordinates of the three-dimensional virtual space.

Meanwhile, the corresponding object disposed in the stereoscopic image according to the present embodiment includes a listener object and a plurality of virtual speaker objects. At this time, the listener object may be designed as a dummy head modeled as a human head and ear structure, and the number of the plurality of virtual speaker objects may be determined according to the number of channels of the multi-channel sound signal included in the voice stream.

Meanwhile, in the stereoscopic image according to the present embodiment, an individual volume controller, an overall volume controller, an EQ controller, and a recording button sensor for recording multi-channel sound signals are provided for controlling the output of sound signals corresponding to a plurality of virtual speaker objects An adjustment button for adjusting the movement of the object may be further implemented.

7 is a diagram illustrating an example of a reconstructed stereoscopic image according to the present embodiment. 7A is a diagram illustrating an example of a reconstructed stereoscopic image according to the first embodiment, and FIG. 7B is a diagram illustrating an example of a reconstructed stereoscopic image according to the second embodiment.

As shown in FIG. 7 (a), a reconstructed stereoscopic image generated when one or more corresponding objects implemented in a stereoscopic image, for example, a layout variation of a listener object is recognized, The batch is reconstructed.

7 (b), the reconstructed stereoscopic image, which is generated when a change in placement of one or more corresponding objects, for example, a listener object, implemented in a stereoscopic image is recognized, In addition to the placement, the placement angles of the plurality of virtual speaker objects may also be further reconstructed.

8 to 14 are views illustrating an example of a multi-viewer image implemented using a multi-viewer image and a 3D stereophonic sound player according to an exemplary embodiment of the present invention.

8 to 11 are views illustrating an example of a multi-viewer image according to the first embodiment of the present invention. As shown in FIGS. 8 to 11, the multi-channel image and the stereoscopic image according to the present embodiment are displayed in at least one channel display area set according to the channel selection information received from the user. 8 to 11 illustrate a case where a multi-channel image and a stereoscopic image are displayed in four channel display areas, respectively. That is, a multi-channel image is displayed on three channels and a stereoscopic image is displayed on the remaining one channel.

12 to 14 are views illustrating an example of a multi-viewer image according to a second embodiment of the present invention.

As shown in FIGS. 12 to 14, the multi-channel image and the stereoscopic image according to the present embodiment can be displayed in a panorama format. 12 to 14 illustrate a case where a multi-channel image and a stereoscopic image are displayed in four channel display areas. In this case, a multi-channel image as well as a corresponding object in the 3D virtual space are additionally displayed. This provides a more realistic 3D multi-viewer image in the case where a multi-viewer image is implemented according to the first embodiment of the present invention, Can be maximized.

The foregoing description is merely illustrative of the technical idea of the present embodiment, and various modifications and changes may be made to those skilled in the art without departing from the essential characteristics of the embodiments. Therefore, the present embodiments are to be construed as illustrative rather than restrictive, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

200: Demultiplexer 201: Channel distributor
202: Decode packetizer 203: Video decoder
204: Audio decoder 207: 3D renderer
208: stereo sound adjuster 209:
210: video output unit 211: audio output unit
212:

Claims (17)

A method for generating a stereoscopic image in which a plurality of virtual speaker objects including a first virtual speaker object and a second virtual speaker object in a virtual space and a listener object are arranged based on the object data, A renderer;
A video output unit receiving the multi-channel image and the stereoscopic image, and displaying at least one of the multi-channel image and the stereoscopic image in a predetermined channel display area;
An audio output unit receiving and outputting a multi-channel sound signal corresponding to the multi-channel image; And
Wherein the listener object is associated with the first virtual speaker object, the first virtual speaker object, the first virtual speaker object, and the second virtual speaker object, the second virtual speaker object being associated with the audio output unit and being arranged in the virtual space generated by a user input signal input using a user input device, The sound of the sound signal corresponding to the first virtual speaker object is increased and the sound of the sound signal corresponding to the second virtual speaker object is decreased and outputted Stereo sound conditioner
And a 3D stereophonic sound player. The method according to claim 1,
The 3D renderer includes:
Wherein the virtual space, the plurality of virtual speaker objects, and the listener object are implemented in 3D form. The method according to claim 1,
Wherein the 3D renderer generates and provides a reconstructed stereoscopic image in which the variation information of the listener object is reflected on the stereoscopic image when a change in the placement direction and movement of the listener object disposed in the virtual space is recognized, Viewer video and 3D stereophonic player device. The method according to claim 1,
Wherein the number of the plurality of virtual speaker objects implemented in the stereoscopic image is determined according to the number of channels of the multi-channel sound signal. delete The method according to claim 1,
The 3D renderer includes:
And generates a reconstructed stereoscopic image in which the placement form of the plurality of virtual speaker objects is reconstructed based on a change in the placement direction of the listener object and movement. The method according to claim 1,
The 3D renderer includes:
And generating a reconstructed stereoscopic image in which the arrangement form of any one of the virtual speaker objects is reconstructed based on the arrangement variation information when a variation in the arrangement form of at least one virtual speaker object among the plurality of virtual speaker objects is recognized And a 3D stereophonic sound player. The method according to claim 1,
Wherein the 3D renderer includes:
An individual volume adjuster for individually adjusting the volume of the acoustic signal corresponding to the plurality of virtual speaker objects;
A total volume controller for simultaneously adjusting the volume of the acoustic signal corresponding to the plurality of virtual speaker objects;
An EQ adjuster for adjusting a frequency of an acoustic signal corresponding to the plurality of virtual speaker objects,
An adjustment button for adjusting movement of the plurality of virtual speaker objects and the listener object; And
And a recording button for recording a multi-channel sound signal output by using the audio output unit is further implemented. The method according to claim 1,
Wherein the video output unit comprises:
And displays the one or more images on one channel display area of one channel, two channels, and four channels based on the channel selection information received from the user. delete delete The method according to claim 1,
Wherein the stereophonic sound conditioner adjusts the sound intensity of the sound signal corresponding to the virtual speaker object based on the arrangement variation information when a variation in the arrangement form of at least one virtual speaker object among the plurality of virtual speaker objects is recognized, And outputting the output of the 3D stereophonic sound player. In the data processing device,
A stereoscopic image display method comprising: receiving one or more object data; generating a stereoscopic image in which a plurality of virtual speaker objects including a first virtual speaker object and a second virtual speaker object in a virtual space and a listener object are arranged based on the object data, Image providing process;
A display step of receiving a multi-channel image and the stereoscopic image and displaying at least one of the multi-channel image and the stereoscopic image in a preset channel display area;
An outputting step of receiving and outputting a multi-channel sound signal corresponding to the multi-channel image; And
The listener object is brought close to the first virtual speaker object and the second virtual speaker object is moved in accordance with the change and movement of the arrangement direction of the listener object disposed in the virtual space generated by the input user input signal using the user input device, A program for realizing a control process of increasing the sound of the sound signal corresponding to the first virtual speaker object and outputting the sound of the sound signal corresponding to the second virtual speaker object when the sound signal is distant from the object, Recorded on a computer readable recording medium. 14. The method of claim 13,
And generating and providing a reconstructed stereoscopic image in which the variation information of the listener object is reflected on the stereoscopic image, when the change in the placement direction of the listener object and the movement are recognized, and providing the program to the computer Lt; / RTI > delete delete 14. The method of claim 13,
Wherein the adjusting step adjusts the output of the sound signal corresponding to the one virtual speaker object based on the arrangement variation information when the variation of the arrangement type is recognized in at least one virtual speaker object among the plurality of virtual speaker objects And outputting the result of the adjustment. The computer-readable recording medium records the program.

Images