KR101834925B1 - A system and method for 3d virtual studio broadcasting that synchronizes video and audio signals by converting object position change into vector - Google Patents

Abstract

The present invention relates to a virtual studio broadcast editing and transmitting device for converting an object location change into a vector to synchronize an image signal with a sound signal and a broadcast editing and transmitting method using the same. According to the present invention, the broadcast editing and transmitting device comprises an input unit, a control unit and an output unit. According to the present invention, the broadcast editing and transmitting method comprises the following steps of: receiving a photographed image and a recorded voice in real time; editing a broadcast image and a broadcast voice in real time; and outputting the broadcast image and the broadcast voice. The broadcast voice, which corresponds to the broadcast image, can also be edited in real time by edition of the broadcast image.

Description

TECHNICAL FIELD [0001] The present invention relates to a virtual studio broadcasting editing and transmitting apparatus and a method of using the same, and more particularly, to a virtual studio broadcast editing and transmission apparatus and a method of using the same,

The present invention relates to a virtual studio broadcast editing and transmitting apparatus and a broadcasting editing and transmitting method using the same, and more particularly, to a virtual studio broadcast editing and transmitting apparatus for synchronizing video and sound signals by converting object position changes into vectors, The output of the broadcast sound can be automatically edited by simply editing the broadcast image without adjusting it, and the change of the object position in which the three-dimensional broadcast sound is automatically outputted according to the broadcast image is converted into a vector And more particularly, to a virtual studio broadcast editing and transmission device synchronizing video and audio signals and a method using the same.

3D Virtual Studio is designed to reproduce realistic broadcasting studio by composing with 3D realistic object, data image and so on on 3D graphic screen created by computer, and it can realize various interior and presentation with low cost and time. It is easy to see how virtual studios are utilized in weather forecasts, election campaigns, and sports broadcasts.

However, when an object or a data image synthesized in a virtual studio is moved in a virtual studio or a screen is switched, the output of the corresponding broadcast audio is manually adjusted according to the timing at which the object and data images move or the screen is switched Accordingly, the timing at which the broadcast video and the broadcast voice are reproduced is inaccurate, and the time, cost, and manpower required for the broadcast production are additionally required. In addition, when a video and a voice are edited after completion of photographing such as a movie, a TV drama, etc., it is predicted how the object will move, and the sound for the object can be adjusted according to the predicted path. However, There is a problem in that it is difficult to control the output of the recording voice corresponding to the object because it is difficult to predict the moving path and the like in the case of the object.

The object of the present invention is to provide a virtual studio broadcast editing and transmission apparatus and a virtual studio broadcast editing and synchronizing system, which synchronize video and sound signals by converting a change of an object position into a vector in which a voice corresponding to a broadcast video is automatically edited in real- And a broadcasting editing and transmitting method using the same.

Another object to be solved by the present invention is to provide a method and apparatus for editing a broadcast image corresponding to a broadcast image in real time so that a broadcast sound corresponding to the broadcast image is outputted as a 3D sound, And a virtual studio broadcasting editing and transmitting device synchronized with a sound signal, and a broadcasting editing and transmitting method using the same.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a virtual studio broadcast editing and transmitting apparatus for converting an object position change into a vector and synchronizing an image and a sound signal according to an embodiment of the present invention includes an image input unit for receiving a captured image in real time, An input unit including a voice input unit for receiving a recorded voice corresponding to a photographed image in real time, a memory for storing a photographed image and a recorded voice received through the input unit and storing one or more data images to be used for editing, A control unit including a broadcasting image editing unit for receiving the photographed image and the data image from one and editing the broadcasting image in real time, and a broadcasting voice editing unit for editing the broadcasting voice corresponding to the broadcasting image in real time, and an image output unit for outputting the broadcasting image, A plurality of audio output units for outputting broadcast audio And the broadcast video editor edits and displays the photographed image in the broadcast image, the object and the data image in the photographed image in three-dimensional coordinates, and the broadcast voice editor compares the movement of the photographed image, And editing the broadcast voice in real time using the voice corresponding to the recorded voice and the data base based on the change of the three-dimensional coordinates indicating the movement of the data video.

According to another aspect of the present invention, a broadcast image editing unit maps a broadcast image to a three-dimensional virtual space, assigns three-dimensional coordinates to an object image and a data image in a photographed image in a virtual space, The voice editing unit can edit the output of the broadcast voice for each of the plurality of audio output units in real time based on the three-dimensional vector representing the movement of the shot image, the movement of the object in the shot image, and the movement of the data image in the virtual space have.

According to another aspect of the present invention, the broadcast voice editing unit may be configured such that when an object or a material image in a photographed image, a photographed image moves along an x-axis in a virtual space, The output of the audio output unit corresponding to the up and down output of the plurality of audio output units is adjusted in real time when the object or the data image in the captured image and the object image move along the y axis in the virtual space, The output of the audio output unit corresponding to the forward and backward output of the plurality of audio output units can be adjusted in real time when the object or the data image in the image and the shot image move along the z axis in the virtual space.

According to another aspect of the present invention, there is provided a broadcast sound editing apparatus including: a broadcast sound editing unit for, when an object in a photographed image moves in a virtual space, for each of a plurality of sound output units based on a change in three- Broadcast voice can be controlled in real time.

According to still another aspect of the present invention, a voice input unit receives a recorded voice in real time from a voice input device moving together with an object in a photographed image, and the broadcast voice editing unit, based on a change in position of the voice input device in a virtual space, It is possible to output the recorded voice to each of the plurality of voice output units.

According to still another aspect of the present invention, a voice input unit receives a plurality of voice recordings in real time from a plurality of voice input devices fixed at a plurality of positions at a capturing place where a captured video is captured, Dimensional coordinates to a plurality of voice input devices fixed in the voice input device and detects movement of the object in a virtual space based on a real time change of a plurality of recorded voice from a plurality of fixed voice input devices, It is possible to output the recorded voice to each of the plurality of voice output units based on the change of the three-dimensional coordinates of the object in the virtual space according to the movement of the object.

According to still another aspect of the present invention, a broadcast audio editing unit includes a plurality of audio output units based on a change in three-dimensional coordinates of a photographed image or a data image in a virtual space when a photographed image or a data image moves in a virtual space, It is possible to adjust the broadcast voice for each of them in real time.

According to another aspect of the present invention, the broadcast voice editing unit can mute a voice corresponding to a photographed image or a data image corresponding to a photographed image when a photographed image or a data image moves outside a virtual space.

According to another aspect of the present invention, a memory stores a plurality of local banks for storing positions of a captured image and a data image to be output as broadcast images in a virtual space, an input for selecting a specific local bank among a plurality of local banks, The broadcast voice editing unit broadcasts a broadcast signal for each of the plurality of audio output units based on the change of the three-dimensional coordinates of the photographed image and the change of the three-dimensional coordinates of the data image generated as the broadcast image is switched to the specific local bank. The voice can be adjusted in real time.

According to still another aspect of the present invention, when an input for selecting the switching from the first bank to the second bank among the plurality of local banks is received, the broadcast voice editor compares three-dimensional coordinates of the photographed image represented by the first bank and data Dimensional coordinate of the image and the three-dimensional coordinate of the data image by comparing the three-dimensional coordinate of the image, the three-dimensional coordinate of the photographed image represented by the second bank and the data image to generate a three-dimensional vector for each of the photographed image and the data image, It is possible to adjust the broadcast voice for each of the voice output units of the mobile terminal 100 in real time.

According to an aspect of the present invention, there is provided a virtual studio broadcasting editing and transmitting method for synchronizing video and audio signals by converting object position changes into vectors according to an embodiment of the present invention, A recording voice corresponding to a broadcast image and a broadcast image on the basis of a recorded image, a recorded voice, and a data image stored in a memory, and outputting the broadcast image through a video output unit And outputting the broadcast audio through a plurality of audio output units. The step of editing the broadcast video and the broadcast audio in real time comprises the steps of mapping the broadcast video into a three-dimensional virtual space, A step of representing an object and a data image in the image, the photographed image in three-dimensional coordinates, And editing the broadcast speech in real time based on the change of the three-dimensional coordinates indicating the movement of the object in the photographed image and the movement of the data image.

According to another aspect of the present invention, the step of editing the broadcast speech in real time based on the change of the three-dimensional coordinates of the object or the data image in the captured image and the captured image includes the steps of: When the object or the data image moves along the x-axis in the virtual space, the output of the audio output unit corresponding to the left and right output of the plurality of audio output units is adjusted in real time and the captured image, The output of the audio output unit corresponding to the upper and lower outputs of the plurality of audio output units is adjusted in real time and when the object or data image in the photographed image moves in the virtual space along the z axis , The output of the audio output unit corresponding to the forward and backward output of a plurality of audio output units is adjusted in real time, It may include the step.

According to an aspect of the present invention, there is provided a broadcast relay system for converting packet data including a plurality of broadcast signals and a broadcast image into a vector, A server for receiving packet data and broadcasting images from a virtual studio broadcasting editing and transmitting apparatus that synchronizes video and sound signals by converting object position changes into vectors, and a server for receiving packet data And a broadcast relay terminal for receiving the broadcast video and outputting the broadcast video to a video output device and a plurality of audio output devices, the packet data including time information, a plurality of channel information, and a plurality of broadcast audio corresponding to a plurality of channels, The broadcast relay terminal corresponds to a plurality of audio output apparatuses among a plurality of channels of packet data And outputs only a channel to be selected.

According to another aspect of the present invention, there is provided a virtual studio broadcast editing and transmission device for converting an object position change into a vector and synchronizing an image and a sound signal, And a voice corresponding to the recording voice and the data voice based on the change of the three-dimensional coordinates indicating the movement of the photographed image, the movement of the object in the photographed image, and the movement of the data image in the broadcast voice And a broadcast voice editor for editing the broadcast voice on a channel-by-channel basis.

The details of other embodiments are included in the detailed description and drawings.

According to the present invention, broadcast audio corresponding to a broadcast image can be automatically and edited in real time according to editing of a broadcast image, thereby saving manpower, time, and cost required for producing a broadcast image.

According to the present invention, a broadcast audio corresponding to a broadcast video can be edited in real time so that the broadcast audio corresponding to the broadcast video is outputted as a 3D sound through a plurality of audio output units, so that a viewer watching a broadcast video can feel an audiovisual stereoscopic effect .

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the specification.

FIG. 1 is a block diagram schematically illustrating the configuration of a virtual studio broadcast editing and transmitting device that synchronizes video and audio signals by converting object position changes into vectors according to an embodiment of the present invention. Referring to FIG.
FIG. 2A is a view illustrating a front portion of a virtual studio broadcast editing and transmission apparatus that synchronizes video and sound signals by converting object position changes into vectors according to an embodiment of the present invention.
FIG. 2B is a diagram illustrating a rear portion of a virtual studio broadcast editing and transmission apparatus that synchronizes video and audio signals by converting object position changes into vectors according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 3A is a main screen displaying a GUI menu for manipulating a control unit of a virtual studio broadcast editing and transmitting apparatus that converts an object position change into a vector and synchronizes video and sound signals according to an exemplary embodiment of the present invention.
FIG. 3B is a transmission screen in which a final broadcast image for manipulating a control unit of a virtual studio broadcast editing and transmitting apparatus synchronized with an image and a sound signal by converting an object position change into a vector according to an embodiment of the present invention is transmitted.
4 is a flowchart illustrating a virtual studio broadcast editing and transmission method of synchronizing video and audio signals by converting object position changes into vectors according to an embodiment of the present invention.
FIG. 5 is a view of a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and a virtual studio in a broadcast editing and transmission method using the same .
FIG. 6A is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and in a broadcasting editing and transmitting method using the virtual studio broadcasting editing and transmitting device, It is a scene of a broadcast video edited by combining with a virtual studio.
FIG. 6B shows a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and in a broadcast editing and transmission method using the same, Is mapped to a three-dimensional virtual space, and a three-dimensional coordinate is given to an object image and a data image in a photographed image, a photographed image, and the like in a virtual space.
FIG. 6C illustrates a virtual studio broadcast editing and transmission apparatus for synchronizing video and sound signals by converting object position changes into vectors according to an exemplary embodiment of the present invention, and a broadcast editing and transmission method using the same, A schematic diagram for explaining a process of editing, in real time, the output of a broadcast audio for each of a plurality of audio output units based on a three-dimensional vector indicating movement of a photographed image in a space, movement of an object in a shot image, to be.
FIG. 7 is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting object position changes into vectors according to another embodiment of the present invention, and in a broadcasting editing and transmitting method using the same, FIG. 1 is a schematic view for explaining a case where a recording voice is recorded from a plurality of voice input devices.
FIG. 8A is a diagram illustrating a virtual studio broadcast editing and transmitting apparatus and a broadcasting editing and transmitting method using the same according to another embodiment of the present invention. FIG. It is a scene of a broadcast image edited by synthesizing objects and data images in a virtual studio.
FIG. 8B is a diagram showing a mapping of the broadcast image of FIG. 8A into a three-dimensional virtual space.
8C is a view showing a moving image of the data image in the broadcast image of FIG. 8A.
FIG. 8D is a diagram showing a state in which the broadcast image of FIG. 8C is mapped into a three-dimensional virtual space.
Figure 9A is a schematic diagram of an example for the local bank selector of Figure 3A.
FIG. 9B illustrates a method for selecting and editing a first local bank in a virtual studio broadcast editing and transmitting apparatus and a broadcast editing and transmitting method using the virtual studio broadcast editing and transmitting apparatus in which an image and sound signals are synchronized by converting an object position change into a vector according to another embodiment of the present invention Fig. 8 is a diagram showing a virtual space when an input is received. Fig.
FIG. 9C is a diagram illustrating a virtual studio broadcast editing and transmitting apparatus and a broadcasting editing and transmitting method using the virtual studio broadcasting editing and synchronizing apparatus in which video and sound signals are synchronized by converting object position changes into vectors according to another embodiment of the present invention. Fig. 8 is a diagram showing a virtual space when an input is received. Fig.
FIG. 9D is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting an object position change into a vector according to another embodiment of the present invention, and a broadcasting editing and transmitting method using the same, Fig. 8 is a diagram showing a virtual space in a case where an input for selecting a virtual space is received.
FIG. 10 is a block diagram schematically illustrating the configurations of a real-time broadcast relay system using a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting object position changes into vectors according to various embodiments of the present invention.
FIG. 11A is a schematic diagram illustrating packet data transmitted from a virtual studio broadcast editing and transmitting apparatus in which an object position change is converted into a vector according to various embodiments of the present invention and the video and sound signals are synchronized.
11B is a diagram showing corresponding output points of each channel of the packet data.
Figs. 12A to 12B are diagrams exemplarily showing a speaker arrangement of a classroom and an auditorium.

In this specification, each block or each step may represent a part of a module, segment or code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, which is capable of reading information from, and writing information to, the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and the storage medium may reside within an application specific integrated circuit. The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

Also, the first, second, etc. are used to describe various components, but it goes without saying that these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical scope of the present invention.

Like reference numerals refer to like elements throughout the specification.

It is to be understood that each of the features of the various embodiments of the present invention may be combined or combined with each other partially or entirely and technically various interlocking and driving is possible as will be appreciated by those skilled in the art, It may be possible to cooperate with each other in association.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram schematically illustrating the configuration of a virtual studio broadcast editing and transmitting device that synchronizes video and audio signals by converting object position changes into vectors according to an embodiment of the present invention. Referring to FIG.

Referring to FIG. 1, a virtual studio broadcast editing and transmitting apparatus 100 for converting an object position change into a vector and synchronizing an image and a sound signal according to an embodiment of the present invention includes an input unit 110, a memory 150, A control unit 120, an output unit 130, and a graphics card 140.

The input unit 110 includes an image input unit for receiving the photographed image in real time and a voice input unit for receiving the recorded voice corresponding to the photographed image in real time. The input unit 110 receives a variety of images and sounds input to the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and sound signals, in real time. The image input unit of the input unit 110 may receive a captured image captured through a video input device such as the camera 11 and the webcam 13 in real time. For example, Can be received in real time. In addition, the voice input unit 112 of the input unit 110 can record voice through the voice input device such as the microphone 12 and can receive the voice corresponding to the photographed image in real time. For example, The recording voice corresponding to the photographed image of the object can be received in real time. In addition, the input unit 110 can receive another image and voice through an external storage device such as the USB 14 and an external electronic device such as the computer 15. [ Also, although not shown in the drawings, the input unit 110 may receive user input through user input means such as a keyboard, a mouse, a touch screen, and the like. The input unit 110 may be implemented in the form of an input port that can be connected to various input devices, external storage devices, external electronic devices, user input means, and the like as described above, but is not limited thereto.

The memory 150 stores the photographed image and the recorded voice received through the input unit 110, and stores one or more data images to be used for editing. The data image is a different image from the photographed image, and refers to an image recorded and edited separately from the image received through the image input device such as the camera 11, the web cam 13, and the like through the image input unit. That is, the data image may be defined as an image edited through another electronic device in advance, and the data image may include a voice corresponding to the data image. However, the present invention is not limited to this, and a data voice corresponding to the data image may be separately stored in the memory 150. [ Since audio for each object is not separately recorded in the data image, editing the data image may not require separate editing of the audio corresponding to the data image. The video and audio stored in the memory 150 can be used for editing for real-time broadcasting in the control unit 120. [ In addition, the memory 150 may store a virtual studio background and the like, which are necessary for producing a broadcast image, and may store video and audio edited by the controller 120.

The control unit 120 receives the recording image, the recording voice corresponding to the recording image, and the data image from at least one of the input unit 110 and the memory 150, and edits the broadcasting image in real time. At this time, the controller 120 receives the selection of the user transmitted through the input unit 110, synthesizes the photographed image and the recorded voice into the virtual studio, and displays various effects such as the data image, the subtitle 192, And edits it as a broadcast image. The control unit 120 may be implemented in the form of a processor for performing various image and sound processing operations in real time. In addition, the controller 120 may perform an operation for performing image and audio processing by applying an operation method such as Real Time Intelligence Format Convergence (RTIFC), multithreading, process load balancing, and floating point operation for high speed operation .

The control unit 120 includes a broadcast image editing unit 121 for editing a broadcast image in real time and a broadcast audio editor 122 for editing a voice corresponding to the broadcast image in real time.

The broadcast image editing unit 121 receives the photographed image and the data image from at least one of the input unit 110 and the memory 150 and edits the broadcast image in real time. At this time, the broadcast image editing unit 121 maps the broadcast image to a three-dimensional virtual space (VS), displays the photographed image in the virtual space (VS), the object and the data image in the photographed image in three- do. On the other hand, the broadcast audio editing unit 122 edits the broadcast audio corresponding to the broadcast video in real time. At this time, the broadcast voice editor 122 uses the three-dimensional vector representing the movement of the photographed image, the movement of the object in the photographed image, and the movement of the data image, .

The output unit 130 includes a video output unit 131 for outputting a broadcast image and a plurality of audio output units 132 for outputting broadcast audio. The output unit 130 converts the object position change into a vector and outputs the broadcast image and the broadcast voice produced by the virtual studio broadcast editing and transmitting apparatus 100 synchronized with the video and sound signals to the output apparatus. The video output unit 131 of the output unit 130 may output a broadcast image through a video output device such as a separate display 31 and the plurality of audio output units 132 of the output unit 130 may be separately It is possible to output the broadcast voice corresponding to the broadcast image through the voice output device such as the speaker 32 of the mobile terminal. In the virtual studio broadcast editing and transmission device 100, which converts an object position change according to an embodiment of the present invention into a vector and synchronizes video and audio signals, a plurality of audio output units 132, that is, It is possible to give a stereoscopic effect to the broadcast voice. That is, according to the configuration of the broadcast image, the broadcast sound corresponding to the broadcast image is output to the plurality of sound output units 132 differently, The editing and transmitting apparatus 100 can output a three-dimensional spatial sound output, and a viewer can feel a stereoscopic effect on the broadcast image and a stereoscopic effect on the broadcast voice. The output unit 130 may be implemented in the form of an output port that can be connected to a video output device such as the display 31 and a voice output device such as the speaker 32, but is not limited thereto.

The graphic card 140 may convert an image represented by digital data into an analog image signal and transmit the analog image signal to the output unit 130. The broadcast image edited in the form of digital data in the control unit 120 can not be output from an output device that handles analog data.

Although not shown in FIG. 1, the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes video and sound signals by converting object position changes into vectors, The display unit may further include a display unit for providing a screen, and the display unit may be implemented with various types of displays.

Hereinafter, with reference to FIG. 2A and FIG. 2B, it is assumed that the front position 160 of the virtual studio broadcast editing and transmitting device 100, which synchronizes the video and sound signals by converting the object position change into a vector according to an embodiment of the present invention, The rear portion 170 will be described.

FIG. 2A is a view illustrating a front portion of a virtual studio broadcast editing and transmission apparatus that synchronizes video and sound signals by converting object position changes into vectors according to an embodiment of the present invention. FIG. 2B is a diagram illustrating a rear portion of a virtual studio broadcast editing and transmission apparatus that synchronizes video and audio signals by converting object position changes into vectors according to an exemplary embodiment of the present invention. Referring to FIG.

2A, an external storage unit such as a USB 14 is connected to the front unit 160 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, A USB terminal 163 as an input unit 110 to be connected, a power switch 161 and a reset switch 162 for turning on / off the power, a CD-ROM 164 . However, the present invention is not limited to this, and various input units 110, switches, or output units may be connected to the front unit 160 of the virtual studio broadcast editing and transmitting apparatus 100, (130), and the like.

2B, various input unit 110 and output unit 130 are connected to the rear part 170 of the virtual studio broadcast editing and transmission device 100, which converts the object position change into a vector and synchronizes the video and sound signals, Can be disposed. For example, various input units 110 such as an audio output terminal 171, a video input terminal 172, a video output terminal 173, a USB terminal 174, and an extension module pack terminal 175 are connected to the rear portion 170, And an output unit 130 may be disposed.

The user can simultaneously input up to 16 voices and nine videos through the input unit 110 such as the video input terminal 172, the USB terminal 174 and the expansion module pack terminal 175, and the HDMI, HD-SDI , Webcam, and so on.

In addition, the user can simultaneously receive up to 16 sounds and three images through the output unit 130 such as the video output terminal 173, the audio output terminal 171, and the expansion module pack terminal 175. Here, the expansion module pack terminal 175 may be a sound card dedicated cable terminal capable of eight voice input / output. However, the number of output channels of the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes video and audio signals by converting the object position change into a vector, can be set to various values.

As described above, the various input units 110 (or 110) disposed in the front portion 160 and / or the rear portion 170 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and sound signals, ) Is transmitted to the control unit 120 and can be edited for real time broadcasting. The controller 120 can edit the video and audio by receiving the user's selection. Hereinafter, FIGS. 3A to 3B are referred to together to describe screens provided to the user for receiving user input.

FIG. 3A is a main screen displaying a GUI menu for manipulating a control unit of a virtual studio broadcast editing and transmitting apparatus that converts an object position change into a vector and synchronizes video and audio signals according to an exemplary embodiment of the present invention. FIG. 3B is a transmission screen in which a final broadcast image for manipulation of a control unit of a virtual studio broadcast editing and transmitting apparatus, which converts an object position change into a vector and synchronizes an image and a sound signal according to an embodiment of the present invention, is transmitted.

Referring to FIG. 3A, a main screen 180 displaying a GUI menu for manipulating the controller 120 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, A source viewer 181, an A / V player control 182a, an A / V mixer volume control 182b, a preview 183a, a program view 183b including an external source viewer 181a and an internal source viewer 181b. A transition bar 184a, a transition effect control 184b, an A / V preset control 185, a PIP object control 186, a local bank selector 187a, a global bank selector 187b, an A / V recorder 188 ), A CD player 189, and the like.

The source viewer 181 is composed of an external source viewer 181a and an internal source viewer 181b. The source viewer 181 confirms the image input state and is a screen that can be compared with a screen applied to the transmission screen 190. [ The external source viewer 181a converts the object position change into a vector and outputs the image input source screen input from the outside of the virtual studio broadcast editing and transmitting apparatus 100 synchronized with the video and sound signals to the HDMI, HD-SDI, A screen from a plurality of external image input sources such as a display device 13 may be displayed. In addition, a screen-captured screen can also be displayed on the external source viewer 181a. The internal source viewer 181b is a video input source screen from the inside of the virtual studio broadcast editing and transmitting apparatus 100 that converts the object position change into a vector and synchronizes video and sound signals, to be.

The A / V player control 182a is an internal source player menu screen, and each source player can correspond to an image of the internal source viewer 181b. Although a maximum of three source players are shown as supported in FIG. 3, it is not limited thereto.

The A / V mixer volume control 182b is a voice mixer control screen, and the user can control the voice output for each input source through the A / V mixer volume control 182b. For example, the A / V mixer volume control 182b allows volume control of the PC output, the microphone 12 output, the source output, the camera 11 output, etc., and controls the equalizer setting, .

The CD player 189 is a menu screen for controlling reproduction of a CD inserted in the CD-ROM 164 as shown in FIG. 2A.

The local bank selector 187a is a screen configured to select a 3D view setting in the virtual studio. For example, as shown in FIG. 3, a total of eight local banks can be preset, and in a manner of selecting one local bank LB of eight preset local banks LB, You can set up a 3D view within the.

The global bank selector 187b is a screen configured to select a virtual studio background. For example, as shown in FIG. 3, a total of eight global banks can be preset, and the user can select a virtual studio background by selecting one of the eight preset global banks.

The preview 183a is a preliminary screen to which the A / V setting and the 3D studio setting are applied. The program view 183b is a broadcasting transmission screen to which the A / V setting and the 3D studio setting are applied. A screen directly applied with the virtual studio background selected in the bank LB and the global bank selector 187b appears.

The transition bar 184a is a menu for switching screens of the preview 183a and the program view 183b. The transition effect control 184b is a menu for designating an effect to be used when the screens of the preview 183a and the program view 183b are switched to the transition bar 184a.

The A / V preset control 185 is a menu for changing the main setting of the 3D virtual studio. For example, the user can use the A / V preset control 185 to change major settings of the 3D virtual studio, such as subtitle 192, viewpoint conversion, lighting, and performance monitoring.

The PIP object control 186 is a menu for controlling enlargement, reduction, position movement, and rotation of a video image in a screen image. Finally, the A / V recorder 188 is a menu for storing the transmission screen 190 as an image file.

Next, referring to FIG. 3B, the transmission screen 190 is a screen for outputting a screen for final broadcast transmission in which editing is completed, and it is possible to confirm how an image is outputted in actual broadcasting. In addition, it is possible to check whether various broadcast editing effects set in the main screen 180 are applied. For example, the logo image 191 inserted using the PIP object control 186 menu or the A / V preset control 185 ) Can be used to check how the embedded subtitle 192 is applied.

In the following, based on the description of the virtual studio broadcast editing and transmitting device 100 that converts the object position change according to the above-described embodiment of the present invention into a vector and synchronizes the video and sound signals, The video and audio processing in the virtual studio broadcast editing and transmission device 100 and the broadcast editing and transmission method using the same will be described.

4 is a flowchart illustrating a virtual studio broadcast editing and transmission method of synchronizing video and audio signals by converting object position changes into vectors according to an embodiment of the present invention.

First, the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes video and audio signals by converting the object position change according to an embodiment of the present invention into a vector, The recording voice is received in real time (S100).

FIG. 5 is a view of a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and a virtual studio in a broadcast editing and transmission method using the same . FIG. 5 shows a virtual studio scene for a situation where a person OB broadcasts using a microphone 12 in front of a chroma key background (BA) for convenience of explanation.

Referring to FIG. 5, one or more image input devices, such as cameras C1, C2, C3, etc., may be used to capture an object OB such as a human in a virtual studio. Further, a voice input device such as a microphone 12a, which is directly held by a person or is attached or fixed to a person, may be used to record a person's voice, or a fixed microphone 12b fixed to a specific position in the virtual studio A voice input device can be used.

The video input unit of the input unit 110 of the virtual studio broadcast editing and transmission apparatus 100 that converts the object position change into a vector and synchronizes the video and sound signals according to an embodiment of the present invention is connected to the cameras C1, C2, C3) and the like, and the voice input unit of the input unit 110 can receive the microphone 12a fixed to the person and / or the microphone 12a fixed to the specific position Recorded from a voice input device such as the stationary microphone 12b, and can receive the recorded voice corresponding to the photographed image wirelessly or wired in real time.

In this case, the chroma key background BA means a background consisting of a single color such as green or blue, and the object OB is any object such as a person placed in front of the chroma key background BA, ≪ RTI ID = 0.0 > BA. ≪ / RTI > The captured image and the recorded voice received through the input unit 110 may be transmitted to the control unit 120 and stored in the memory 150. [

Then, the virtual studio broadcast editing and transmission apparatus 100 synchronizes the video and sound signals by converting the object position change into a vector according to an embodiment of the present invention, and then, based on the photographed image, the recorded voice, and the data image DV, The broadcast audio corresponding to the broadcast video BV and the broadcast video BV is edited in real time (S200).

The broadcast image editing unit 121 of the control unit 120 of the virtual studio broadcast editing and transmission apparatus 100 that synchronizes the image and sound signals by converting the object position change into a vector may be provided with at least one of the input unit 110 and the memory 150 And edits the broadcast video (BV) in real time. The broadcast sound editing unit 122 of the control unit 120 of the virtual studio broadcast editing and transmission apparatus 100 synchronizes the image and sound signals by converting the object position change into a vector and outputs the broadcast sound corresponding to the broadcast image BV In real time. At this time, the broadcast voice editing unit 122 corresponds to the recording voice and the data base DV based on the three-dimensional vector indicating the movement of the photographed image, the movement of the object OB in the photographed image, and the movement of the data base DV You can edit the broadcast voice in real time using the data voice. 6A to 6C are referred to for a more detailed description of the process of the controller 120 editing the broadcast video BV and the broadcast voice in real time using the photographed image, the recorded voice, and the data image DV do.

FIG. 6A is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and in a broadcasting editing and transmitting method using the virtual studio broadcasting editing and transmitting device, It is a scene of a broadcast video edited by combining with a virtual studio. FIG. 6B shows a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting an object position change into a vector according to an embodiment of the present invention, and in a broadcast editing and transmission method using the same, Is mapped to a three-dimensional virtual space, and a three-dimensional coordinate is given to an object image and a data image in a photographed image, a photographed image, and the like in a virtual space. FIG. 6C illustrates a virtual studio broadcast editing and transmission apparatus for synchronizing video and sound signals by converting object position changes into vectors according to an exemplary embodiment of the present invention, and a broadcast editing and transmission method using the same, A schematic diagram for explaining a process of editing, in real time, the output of a broadcast audio for each of a plurality of audio output units based on a three-dimensional vector indicating movement of a photographed image in a space, movement of an object in a shot image, to be. 6B and 6C, it is assumed that a person who is an object OB directly holding a microphone 12a or a microphone 12a attached to a person is used.

Referring to FIG. 6A, a broadcast image editing unit (not shown) of a controller 120 of a virtual studio broadcast editing and transmitting apparatus 100, which converts an object position change into a vector and synchronizes an image and a sound signal according to an embodiment of the present invention, 121 receives the photographed image and the data image DV and edits the broadcast image BV in real time. Specifically, the broadcast image editing unit 121 can edit the broadcast image BV in real time by synthesizing the material image DV on the photographed image of the object OB in the virtual studio. 6A, two data images DV1 and DV2 are included in the broadcast image BV for convenience of explanation, but the number of the data images DV is not limited thereto.

5, when a captured image is acquired by capturing an object OB in a virtual studio using a chroma background (BA), the broadcast image editing unit 121 edits the captured image and the data image DV, It is possible to perform chroma key processing on the photographed image before completing the broadcast video BV. In detail, the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes the video and sound signals by converting the object position change into a vector according to an embodiment of the present invention, It is possible to separate the object OB and the chroma key background BA by detecting the edge of the object OB with respect to the shot image and to combine only the object OB separated from the chroma key background BA into the virtual studio have.

The broadcast image editing unit 121 of the control unit 120 receives an external storage unit such as the USB 14 and an external electronic device such as the computer 15 through the input unit 110 and stores the received broadcast image in the memory 150 (DV) and the material image (DV) stored in the virtual studio (ST) can be synthesized in the virtual studio to edit the broadcast image (BV) as shown in FIG. 6A in real time.

Subsequently, the broadcast image editing unit 121 maps the broadcast image BV to a three-dimensional virtual space VS in step S210. Then, in the virtual space VS, the captured image, the object OB in the captured image, Dimensional coordinate to the data image DV (S220).

Referring to FIG. 6B, the broadcast image editing unit 121 may map the broadcast image BV to a three-dimensional virtual space (VS). In this case, the virtual space VS in the three-dimensional space is a space in which the broadcast image editing unit 121 moves the photographed image in the broadcast image BV, the object OB in the photographed image, Dimensional coordinate, that is, a three-dimensional vector, and the broadcasting voice editing unit 122 edits the broadcasting voice based on the above-described three-dimensional coordinate change. The virtual space (VS) of a three-dimensional space can be defined using x-axis, y-axis, and z-axis, and the maximum value of the coordinate value for each axis is 1, Can be defined. However, the present invention is not limited to this, and the virtual space VS may be defined to have different maximum and minimum values for each of the x-axis, y-axis, and z-axis. The broadcast image editing unit 121 can map the broadcast image BV to the normalized three-dimensional virtual space VS described above.

6B, the broadcast image editing unit 121 of the control unit 120 maps the broadcast image BV to a three-dimensional virtual space VS, Dimensional coordinates to the object OB and the data base DV in the image data DB. Specifically, the broadcast image editing unit 121 compares the three-dimensional coordinates Sx, Sy, Sz of the photographed image in the virtual space VS and the three-dimensional coordinates Ox, Oy, Oz of the object OB in the photographed image, Dimensional coordinates D2x, D2y, and D2z for the first data image DV and the three-dimensional coordinates D2x, D2y, and D2z for the second data image DV. Here, the three-dimensional coordinates of the object (OB) and the data image (DV) in the photographed image, the photographed image can be defined as three-dimensional coordinates of the respective center of gravity, , But is not limited thereto.

6A, when the entire screen of the broadcast image BV corresponds to the entire captured image, the broadcast image editing unit 121 of the control unit 120 displays the three-dimensional coordinates Sx, Sy, Sz of the photographed image, (0, 0, 0) of the origin (O). However, if the photographed image is enlarged to include only a part of the photographed image in the broadcast image BV, or if the photographed image is reduced in the broadcast image BV, The z-coordinate Sz of the three-dimensional coordinates Sx, Sy, Sz can be increased or decreased. In addition, when only a part of the photographed image is included in the broadcast image BV, the x-axis coordinate Sx and the y-axis coordinate Sz of the three-dimensional coordinates Sx, Sy, Sz of the photographed image can also be increased or decreased . In FIG. 6B, the three-dimensional coordinates of the photographed image are omitted for convenience of explanation.

Then, the broadcast image editing unit 121 can define (Ox, Oy, Oz) which is the three-dimensional coordinates OP of the object OB. The broadcast image editing unit 121 can give the three-dimensional coordinates OP of the object OB (Ox, Oy, Oz) in consideration of the position of the object OB in the captured image. That is, in consideration of the left-right position, the up-down position, and the back-and-forth position of the object OB in the photographed image, the broadcast image editing unit 121 obtains the three-dimensional coordinates OP (Ox, Oy, Oz) of the object OB Can be defined. Also, in some embodiments, the broadcast image editing unit 121 may calculate the three-dimensional coordinates (OP) (Ox, Oy, Oz) of the object OB using the image projected on the view port . That is, it is assumed that there is a variable virtual camera that is located outside the three-dimensional virtual space VS and that is projected on the view port to show the viewer a broadcast screen, and based on the relative coordinates with the variable virtual camera Vertical positions and front and back positions of the object OB and determines the three-dimensional coordinates OP of the object OB (Ox, Oy, Oz). However, the present invention is not limited to this, and the three-dimensional coordinates OP (Ox, Oy, Oz) of the object OB may be given in other manners. 6B, it is assumed that the three-dimensional coordinates (OP) (Ox, Oy, Oz) of the object OB are (0.5, 0, 0.5).

Next, the broadcast image editing unit 121 compares the three-dimensional coordinate DP1 (D1x, D1y, D1z) of the first data base DV and the three-dimensional coordinate DP2 of the second data base DV D2x, D2y, D2z) can be defined. The broadcast image editing unit 121 compares the first data image DV with the first data image DV and the second data image DV with the coordinates D2x, D2y and D2z may be substantially the same as the method of defining the three-dimensional coordinates (OP) (Ox, Oy, Oz) of the object OB described above. In FIG. 6B, the three-dimensional coordinates DP1 of the first data image DV are (-0.3, 0.5, 0) and the three-dimensional coordinates of the second data image DV are DP2) and (D2x, D2y, D2z) are assumed to be (-0.3, 0.3, 0).

6B, the control unit 120 includes a plurality of audio output units 132 for outputting broadcast audio and a plurality of audio output devices 32a to 32f corresponding to the plurality of audio output units 132, Can be given. That is, the number of the plurality of audio output units 132 is determined according to the number of the total audio output channels to be used in the virtual studio broadcast editing and transmitting apparatus 100 in which the object position change is converted into a vector and the video and sound signals are synchronized Dimensional coordinates of the audio output unit 132 can be given based on the number of the audio output units 132 determined. 6B, it is assumed that the speaker 32 is used as the sound output apparatuses 32a to 32f corresponding to the plurality of sound output units 132. In the virtual space VS, right, left, upper, lower, And six speakers 32 on the rear side are assumed to be used as sound output apparatuses 32a to 32f corresponding to the sound output unit 132. Each of the six sound output apparatuses 32a to 32f is assumed to be 1, (0, 0, 0), (0, 0, 0), (0, 0, 0) Coordinates. 6B, it is assumed that all of the sound output apparatuses 32a to 32f corresponding to the sound output unit 132 are disposed at the outermost of the virtual space VS. However, the present invention is not limited to this, The number of total sound output channels and the position change of the object may be converted into a vector and arranged at an arbitrary position in the virtual space VS according to the virtual studio broadcast editing and transmitting apparatus 100 synchronized with the video and sound signals. However, in order to more realistically impart a stereoscopic effect of the broadcast audio output, the two audio output units 132 are arranged so as to be symmetrical to the x-axis, the y-axis, the z-axis or the origin O in the virtual space VS can do.

The broadcast audio editing unit 122 of the control unit 120 can adjust the output of each of the plurality of audio output units 132 based on the three-dimensional coordinates of the captured image, the object OB in the captured image, have. That is, the broadcast audio editing unit 122 extracts the right audio output unit 32a (32a) corresponding to the right audio output unit based on the x-axis coordinate of the three-dimensional coordinates of the object (OB) ) And the left audio output device 32b corresponding to the left audio output unit, and adjusts the y-axis coordinate of the three-dimensional coordinates of the object image OB or the data image DV in the photographed image, (32c) corresponding to the upper audio output unit and the lower audio output unit (32d) corresponding to the lower audio output unit are adjusted based on the captured image and the object The front audio output device 32e corresponding to the front audio output unit and the rear audio output device 32f corresponding to the rear audio output unit corresponding to the z-axis coordinate of the three-dimensional coordinates of the object image DV or OB Can adjust the broadcast voice have. That is, the broadcast audio output to the audio output devices 32a to 32f corresponding to the audio output units 132 disposed at positions mutually symmetrical with respect to the x-axis, y-axis, and z-axis can be relatively controlled.

For example, when the three-dimensional coordinates (Ox, Oy, Oz) of the object OB are the origin O (0,0,0), the voice corresponding to the object OB, All of the recorded voice can be output to the six voice output units 132 at the same size. However, when the object OB is disposed on the rightmost side in the virtual space VS and the three-dimensional coordinates OP of the object OB are defined as (1, a, a), which is Ox, Oy, Oz (-1? A? 1). In the right audio output apparatus 32a, the recording voice for the object OB is output as the maximum output. In the left audio output apparatus 32b, Lt; / RTI > Conversely, when the object OB is disposed at the leftmost side in the virtual space VS and the three-dimensional coordinates OP of the object OB are defined as (Ox, Oy, Oz) (-1, a, a) The recording sound for the object OB is outputted as the minimum output in the right audio output device 32a but the recording sound for the object OB is outputted in the left audio output device 32b at the maximum. Here, the minimum output may be a mute or a predetermined value that is larger than 0 but is very small, but hereinafter, the minimum output will be described as mute for convenience of explanation.

In the case where the three-dimensional coordinates (Ox, Oy, Oz) of the object OB are defined as (b, a, a) The output value of the recorded voice for the object OB in the right audio output device 32a and the left audio output device 32b can be determined. For example, if b = 0.5, the distance between the object OB and the rightmost point on the x-axis is 0.5, and the distance between the object OB and the leftmost point is 1.5, 75% of the volume of the voice can be outputted to the right voice output device 32a and 25% can be outputted to the left voice output device 32b. However, if the minimum output is any value other than 0, the percentage of the sound volume of the recording voice with respect to the object OB may be determined in consideration of the difference between the maximum output and the minimum output. The output value of the recorded voice to the object OB in the upper audio output device 32c and the lower audio output device 32d can be relatively determined according to the y-axis coordinate of the object OB with the same principle as described above, The output value of the recorded voice for the object OB in the front side audio output device 32e and the rear side audio output device 32f can be relatively determined in accordance with the z axis coordinate of the object OB. The output values of the plurality of audio output apparatuses 32a to 32f corresponding to the plurality of audio output units 132 according to the three-dimensional coordinates OP (Ox, Oy, Oz) of the object OB are defined as follows .

[Equation 1]

Recorded voice output to object OB of right audio output device 32a = volume of recorded audio * (1 + Ox) / 2

&Quot; (2) "

Recorded voice output for object OB of left audio output device 32b = Volume of recorded voice * (1 - Ox) / 2

&Quot; (3) "

Recorded voice output for object OB of upper audio output device 32c = volume of recorded voice * (1 + Oy) / 2

&Quot; (4) "

Recorded voice output for object OB of lower voice output device 32d = volume of recorded voice * (1 - Oy) / 2

&Quot; (5) "

(1 + Oz) / 2 (sound volume of the recording voice) of the object OB of the front sound output apparatus 32e =

&Quot; (6) "

Recorded voice output for object OB of rear side sound output device 32f = volume of recorded voice * (1 - Oz) / 2

6B, six audio output units 132 are defined in a virtual space VS, and the number of audio output units 132 and the number of audio output units 132 When the positions of the corresponding sound output apparatuses 32a to 32f are changed, the recorded sound output values can be defined differently in consideration of the respective positions and the symmetric relationship. In addition, the above-described example is only one example in the case of using the six audio output units 132, and other methods may be used. For example, the sum of the recording audio output of the object OB of the right audio output device 32a and the recording audio output of the object OB of the left audio output device 32b, The sum of the recording voice output to the object OB and the recording voice output to the object OB of the lower voice output device 132d and the sum of the voice voice output to the object OB of the front voice output device 32e and the voice The sum of the recording voice output to the object OB of the output device 32f is always the same and the sum of the recording voice output to the object OB of all the voice output units 132 is defined as the total volume of the recording voice .

The three-dimensional coordinates (Ox, Oy, Oz) of the object OB are (0.5, 0, 0.5), so that the right voice The output device 32a outputs 75% of the volume of the recording voice to the object OB, the left voice output part 32b to 25% of the volume of the recording voice to the object OB, and the upper voice output device 32c 50% of the volume of the recording voice for the object OB, the lower voice output device 32d is 50% of the volume of the recording voice for the object OB and the front voice output device 32e is for the object OB 75% of the volume of the recording voice and the rear voice output device 32f can output 25% of the volume of the voice to be recorded to the object OB. Also, the audio output for the data images DV1 and DV2 can be adjusted to the same principle.

In some embodiments, broadcast audio output for the data images DV1 and DV2 may be processed independently of the three-dimensional coordinates DP1 and DP2 of the data images DV1 and DV2. For example, in the broadcast image BV shown in FIG. 5, the object OB may be an announcer, and the data images DV1 and DV2 may be a specific image for explaining news delivered by the announcer. In this case, the announcer's message must be transmitted to the viewer. If the audio of the data DV1 or DV2 is output, it may be difficult for the viewer to receive the announcer's message, and it may be helpful to understand the announcer's message It is possible. Accordingly, in the virtual studio broadcast editing and transmitting apparatus 100 that converts the object position change into a vector and synchronizes the video and sound signals according to an embodiment of the present invention, (DV) of the data image (DV) based on the relationship between the recorded voice for the object image (OB) and the voice of the data image (DV) and / or the relationship between the captured image and the data image It may be mutually exclusive regardless of the coordinates DP, or may be output to the plurality of audio output units 132 based on the three-dimensional coordinates DP.

Further, in some embodiments, it is possible to determine whether or not to mute the voice for the data base (DV) based on the z-axis coordinate value among the three-dimensional coordinates of the data base (DV). For example, when the z-axis coordinate value among the three-dimensional coordinates DP of the data image DV is equal to or larger than the threshold value, the intention of the person editing the broadcast image BV transmits the voice of the data image DV to the viewer The voice of the data base DV can be output to the plurality of voice output units 132 based on the three-dimensional coordinates DP of the data base DV. However, when the z-axis coordinate value of the three-dimensional coordinates of the data image DV is less than the threshold value, the data image DV is located at the very rear side in the virtual space VS and the data image DV is very large The intention of the person editing the broadcast video BV may not be to deliver the audio of the data video DV to the viewer. In this case, the voice of the data image DV may be muted.

When the object OB and the material image DV in the photographed image move or the screen is switched during the broadcasting editing process, the object OB and the material image DV in the photographed image, The three-dimensional coordinates imparted to the object can be continuously changed.

Accordingly, the broadcast audio editing unit 122 changes the three-dimensional coordinate indicating the movement of the photographed image, the movement of the object OB in the photographed image, and the movement of the data image DV, that is, Is edited in real time (S230).

The broadcast audio editing unit 122 can display a plurality of audio output (s) when the photographed image, the object OB or the material image DV in the photographed image moves along the x axis in the virtual space VS in the virtual space VS, The output of the sound output unit 132 corresponding to the right and left outputs of the image pickup unit 132 is adjusted in real time and the object OB or the material image DV in the photographed image is converted into y The output of the audio output unit 132 corresponding to the upper and lower outputs of the plurality of audio output units 132 is adjusted in real time and the object OB or the material image DV in the captured image, When the user moves along the z axis in the virtual space VS, the output of the audio output unit 132 corresponding to the front and rear outputs of the plurality of audio output units 132 is adjusted in real time, and the broadcast audio is edited in real time. Hereinafter, with reference to FIG. 6C, it is assumed that, for example, the movement of the object OB is performed, the broadcast sound editing unit 122 of the control unit 120 broadcasts the object OB to each of the plurality of sound output units 132 A process of controlling the output of voice will be described.

)는 점 OP에서 점 OP' 사이의 방향 및 거리를 나타내는 것으로, 점 OP와 점 OP' 사이의 3차원 벡터

)는 점 OP'-점 OP로 (Ox'-Ox, Oy'-Oy, Oz'-Oz), 즉 (-1, 0, 0)이고, 녹음 음성을 녹음하는 마이크(12a)가 객체(OB)에 고정된 상태이므로, 우측 음성 출력 장치(32a)에서의 객체(OB)에 대한 녹음 음성의 출력은 녹음 음성의 음량의 50%가 감소하고, 좌측 음성 출력 장치(32b)에서의 객체(OB)에 대한 녹음 음성의 출력은 녹음 음성의 음량의 50%가 증가할 수 있다. The object OB having (Ox, Oy, Oz) which is the three-dimensional coordinates OP of (0.5, 0, 0.5) as shown in FIG. 6B is obtained as (-0.5, 0, 0.5 (Ox ', Oy', Oz ') of the three-dimensional coordinates (OP') of the audio signals output from the plurality of audio output units 132. Only the x-axis coordinate of the object OB changes from 0.5 to -0.5. In this case, the broadcast audio editing unit 122 of the control unit 120 determines whether or not the audio output unit on the right side The output of the audio output device 32a and the output of the left audio output device 32b can be adjusted. At this time, a three-dimensional vector representing the movement of the object OB

) Represents a direction and a distance between a point OP and a point OP ', and a three-dimensional vector between a point OP and a point OP'

) Is a point OP'- point OP (Ox'-Ox, Oy'-Oy, Oz'-Oz), that is, (-1, 0, 0), and the microphone 12a recording the recording voice is OB The output of the recorded voice to the object OB in the right audio output device 32a is reduced by 50% of the volume of the recorded audio and the output of the object OB in the left audio output device 32b ) May increase the volume of the recorded voice by 50%.

This principle can be applied to the sound output apparatus 32 corresponding to all the sound output units 132, which are summarized as follows.

&Quot; (7) "

The change amount of the recorded voice output to the object OB of the right voice output device 32a = the volume of the recorded voice * (Ox'-Ox) / 2

&Quot; (8) "

The change amount of the recorded voice output to the object OB of the left voice output device 32b = the volume of the recorded voice * (Ox-Ox ') / 2

&Quot; (9) "

The amount of change of the recorded voice output to the object OB of the upper voice output device 32c = the volume of the recorded voice * (Oy'-Oy) / 2

&Quot; (10) "

(Oy-Oy ') / 2 (amount of change in recording voice output to the object OB of the lower sound output apparatus 32d =

&Quot; (11) "

The amount of change of the recorded voice output to the object OB of the front side audio output device 32e = the volume of the recorded voice * (Oz'-Oz) / 2

&Quot; (12) "

(Oz-Oz ') / 2 (amount of change in recording voice output relative to the object OB of the rear side audio output device 32f =

6C, a broadcast audio editing unit (not shown) of the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals according to an embodiment of the present invention, 122 have adjusted the outputs of the plurality of audio output units 132 in accordance with the x-axis movement of the object OB, it is also possible that not only the object OB but also the photographed image or the data image DV is displayed on the x- And the manner in which the outputs of the plurality of audio output units 132 are adjusted when moving along the z axis may be the same as described above.

6C, six audio output units 132 are defined, and the number of the audio output units 132 and the number of the audio output units 32a To 32f are changed, the change amount of the recorded voice output can be defined differently in consideration of the respective positions and the symmetric relationship. In addition, the above-described example is only an example in the case of using the six audio output devices 32a to 32f, and other methods may be used. For example, the sum of the recording audio output of the object OB of the right audio output device 32a and the recording audio output of the object OB of the left audio output device 32b, The recording audio output for the object OB and the sum of the recording audio output for the object OB of the lower audio output device 32d and the sum of the recording audio output for the object OB of the front audio output device 32e and the audio output of the rear audio The sum of the recording voice output to the object OB of the output device 32f is always the same and the sum of the recording voice output to the object OB of all the audio output devices 32a to 32f The sum of the outputs may be defined as the total volume of the recorded voice.

In some embodiments, the broadcast audio editing unit 122 of the control unit 120 calculates the relative coordinates between the variable virtual camera and the object image (OB) in the photographed image, and the data image (DV) as described above , The broadcasting voice for each of the photographed image, the object OB and the data base DV can be edited in real time based on the relative coordinates. For example, relative coordinates (Vx-Ox, Vy-Oy, Vz-Oz) of the three-dimensional coordinates (Vx, Vy, Vz) of the variable virtual camera and the three- The audio output to the plurality of audio output units 132 may be adjusted in real time based on the increase or decrease of the relative coordinates every frame.

In some embodiments, when an object (OB) or a material image (DV) in a photographed image, a photographed image moves so as to deviate from a virtual space (VS), the voice for each of them can be muted. For example, when the object OB in the photographed image moves to one side and deviates from the virtual space VS, the object OB is not seen in the broadcast image BV. In this case, since it is not necessary to reproduce the voice for the object OB, the voice for the object OB can be muted.

In some embodiments, the broadcast voice editor 122 of the control unit 120 may adjust the rate at which the volume of each voice output unit 132 is adjusted to be equal to the moving speed of the object OB. Here, the moving speed of the object OB can be calculated by calculating a three-dimensional vector representing the movement of the object OB every frame, and the volume of the sound output unit 132 is adjusted every frame The speed at which the volume of the sound output unit 132 is adjusted can be adjusted to be equal to the moving speed of the object OB.

Subsequently, the video output unit 131 outputs the broadcast video BV to the video output apparatus, and the plurality of audio output units 132 output the broadcast audio to the plurality of audio output apparatuses 32a through 32f (S300) .

The control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100 that converts the object position change into a vector and synchronizes the video and sound signals according to an embodiment of the present invention receives the user's selection from the input unit 110 The user can manipulate the GUI menu of the main screen 180 to combine the photographed image, the object OB in the photographed image, and the data image DV into the virtual studio, and add various effects. By manipulating the respective menus of the GUI menu described above, various effects such as the subtitle 192, screen switching, background music, and the like can be added to edit the broadcast video BV. Referring to FIG. 3B, it is possible to confirm a broadcast image BV to be broadcasted and actually edited through the transmission screen 190. As shown in the transmission screen 190, it can be confirmed that the subtitle 192, the logo 191, and the like are actually applied. For example, an image such as the logo 191 shown in FIG. 3B may be inserted by manipulating the PIP object control 186 of the main screen 180, and the A / V preset control 185 may be operated, (192) may be inserted.

Then, the edited broadcast image BV and the broadcast sound are transmitted from the control unit 120 to the TV, the monitor, the electric sign board, and the speaker 32 through the video output unit 131 and the plurality of audio output units 132 of the output unit 130 ), Or may be stored in the memory 150. [0050]

In the conventional broadcasting editing and transmission method, a separate editing process is required so that the audio corresponding to the video is output, not the audio being automatically controlled according to each video, and the cost, time, and additional manpower are required . In addition, there is a problem that the reproduction timing of the video and audio is inaccurate in the process of manually adjusting the audio output in accordance with the timing of the image movement or the screen switching.

Accordingly, in the virtual studio broadcast editing and transmitting device 100 in which video and sound signals are synchronized by converting the object position change into a vector according to an embodiment of the present invention, and a broadcast editing and transmitting method using the same, The voice corresponding to the object OB can be adjusted in real time and automatically according to the position or movement of the object OB, thereby reducing manpower, time and cost. Also, in the virtual studio broadcast editing and transmitting device 100 that synchronizes video and sound signals by converting the object position change into a vector according to an embodiment of the present invention, and a broadcasting editing and transmitting method using the same, The broadcast voice editing unit 122 edits the broadcast voice in real time and automatically according to the position or movement of the object OB in the mobile terminal 100. This makes it possible to edit the broadcast voice more accurately when a person manually edits the broadcast voice . That is, in the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting the object position change according to an embodiment of the present invention into a vector, and a broadcast editing and transmission method using the same, The output of the broadcast audio can be automatically adjusted in real time based on the change of the broadcast video BV according to the editing of the broadcast video BV without separately editing or adjusting the broadcast voice according to the broadcast video It is possible to save manpower, time, and cost for editing the broadcast video (BV) and broadcast voice rather than manually editing the voice manually. Accordingly, in the virtual studio broadcast editing and transmitting device 100 that synchronizes video and sound signals by converting object position changes according to an embodiment of the present invention into a vector, and broadcast editing and transmission methods using the same, It is possible to give a high degree of freedom to a broadcasting operator who performs broadcasting voice editing, and a user can easily produce broadcast video (BV) and broadcasting voice more quickly and efficiently.

On the other hand, broadcasts using 3D video and 3D voice are produced so that viewers watching the broadcast video can feel as if they are in the actual scene. In order to output audio in 3D, the audio output to each of the plurality of audio output units must be adjusted in accordance with the precise moving point and moving speed of the object or the image. However, when video and audio are edited in real time to produce a broadcast image, the actual moving position and the moving speed of the image or object are accurately adjusted so that the audio output of each of the plurality of audio output units is individually adjusted, Which is difficult to edit. Particularly, as described above, it is difficult to realize a 3D stereophonic sound accurately in the process of manually adjusting the audio output in accordance with the timing of the movement of the image or the switching of the screen.

Accordingly, in the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting object position changes according to an embodiment of the present invention into a vector, and a broadcast editing and transmission method using the virtual studio broadcast editing and transmitting apparatus 100, Dimensional virtual space VS and assigns three-dimensional coordinates to the object image OB and the material image DV in the photographed image constituting the broadcast image BV, the photographed image, The voice corresponding to the object OB and the voice corresponding to the data image DV can be automatically adjusted in real time. For example, when the broadcast sound editing unit 122 of the control unit 120 adjusts the output of the plurality of sound output units 132 with respect to the sound of the object OB, It is possible to increase the output of the object 132 and lower the output of the audio output unit 132 far from the object OB so that the viewer who watches the broadcast can feel the audiovisual stereoscopic effect. As described above, since the process of adjusting all the broadcast audio outputs is automatically adjusted according to the broadcast video BV, it is possible to automatically output a 3D voice with a lively feeling according to the broadcast video BV, Compared to editing the output, it is possible to transmit the sound sense more accurately and quickly.

On the other hand, when a video and a recorded voice such as a movie are edited in advance and the edited video and audio are simply reproduced to viewers, And the information about the stored voice is transmitted to the speaker so that a three-dimensional voice can be output. However, in the case of editing video and audio in advance before reproduction such as a movie, it is predicted how the object will move on the movie, and only the sound for the object is adjusted according to the predicted path, Sound editing is also easy. However, at the time of real-time broadcast editing and transmission, there is a case where it is not possible to predict in advance which position the object positioned in the photographed image will move to and in which position the object image or the like is moved in a non-scheduled position and direction . Accordingly, when broadcast is transmitted in real time different from a medium such as a movie, it is difficult to edit the broadcast audio corresponding to the broadcast image into 3D sound corresponding to movement of the object or movement of the data image.

Accordingly, in the virtual studio broadcast editing and transmitting device 100 that synchronizes video and sound signals by converting the object position change into a vector according to an embodiment of the present invention, and a broadcasting editing and transmitting method using the same, Directional and moving speeds, that is, a three-dimensional vector, can be detected, and the sound output of each of the plurality of sound output units 132 can be automatically adjusted to output a three-dimensional 3D sound. As described above, the broadcast audio editing unit 122 of the control unit 120 extracts the photographed image based on the three-dimensional vector indicating the movement of the photographed image, the movement of the object OB in the photographed image, and the movement of the data image DV, The output of the plurality of audio output units 132 can be adjusted so that the viewer can feel the vividness even if the object OB and the data base DV in the photographed image move. The control unit 120 automatically detects the moving direction and the moving speed of the object OB even when the moving direction of the object OB and the moving speed of the object OB are not predicted, The output is adjusted so that the viewer can view the broadcast video (BV) and feel the audiovisual stereoscopic effect.

FIG. 7 is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting object position changes into vectors according to another embodiment of the present invention, and in a broadcasting editing and transmitting method using the same, FIG. 1 is a schematic view for explaining a case where a recording voice is recorded from a plurality of voice input devices. In FIG. 7, it is assumed that a plurality of audio input devices 12b among the audio input devices 12 described in FIG. 5 are disposed at fixed positions.

7, the voice input unit of the input unit 110 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and sound signals according to another embodiment of the present invention, In real time, a plurality of recorded voices from a plurality of audio input devices (12b) fixed at a plurality of positions at a photographed position.

The broadcast image editing unit 121 of the control unit 120 gives three-dimensional coordinates in a virtual space VS to the plurality of audio input devices 12b1 to 12b8. For example, in FIG. 7, it is assumed that the microphone 12 is used as the plurality of voice input devices 12b1 to 12b8, and that a plurality of voice input devices are disposed at each vertex of the virtual space VS . Accordingly, the first audio input device 12b1 is (-1, -1, 1), the second audio input device 12b2 is (-1, -1, -1), the third audio input device 12b3 is -1, -1), the fourth audio input device is (1, -1, 1), the fifth audio input device is (-1, 1, 1) 1, -1), the seventh sound input device has (1, 1, -1), and the eighth sound input device has three-dimensional coordinates of (1, 1, 1). The number of the plurality of audio input devices 12b1 to 12b8 and the plurality of positions to which the plurality of audio input devices 12b1 to 12b8 are fixed shown in Fig. 7 are for convenience of description, and the plurality of audio input devices 12b1 to 12b8 ) And the position where they are fixed are not limited thereto.

The broadcast image editing unit 121 can detect the three-dimensional coordinates of the object OB in the virtual space VS based on the real-time change of the plurality of recorded sounds received from the plurality of sound input devices 12b1 to 12b8 . Specifically, the audio input unit of the input unit 110 can receive a plurality of recorded audio corresponding to the object OB in real time from the plurality of audio input devices 12b1 to 12b8 fixed at a plurality of positions at the shooting location. The broadcast image editing unit 121 of the control unit 120 can detect the three-dimensional coordinates of the object OB through the volume of the recorded voice of the object OB received by each of the voice input devices 12b. For example, if the sizes of the plurality of recorded sounds received by the plurality of sound input devices 12b1 to 12b8 are all the same, the distances between the object OB and the plurality of sound input devices 12b1 to 12b8 are all the objects OB) are the same, it is possible to detect that the three-dimensional coordinate of the object OB is the origin O. As described above, the size of the recorded voice of the received object OB is compared with each of the plurality of voice input devices 12b1 to 12b8, and the closer the object OB is, the larger the recorded voice is, The three-dimensional coordinates of the object OB can be detected on the basis of the recorded voice size received by each of the voice input devices 12b by using a point having a small recording voice size. At this time, the controller 120 may directly calculate the three-dimensional position of the object OB based on the sizes of the plurality of recorded voice recorded in real time. The sizes and ratios of the voice of the object OB received in each of the plurality of voice input devices 12b1 to 12b8 according to the specific position of the object OB are stored in advance in the memory 150 in the form of a lookup table, The control unit 120 may detect the three-dimensional coordinates of the object OB by searching the lookup table based on the recorded voice received by the plurality of voice input devices 12b1 to 12b8. However, the present invention is not limited to this, and the three-dimensional coordinates of the object OB may be detected in other manners.

7, the broadcast image editing unit 121 of the control unit 120 edits an object (e.g., a music file) based on the recorded voice of the object OB received by the first audio input device 12b1 to the eighth audio input device 12b8 OB) can be detected. According to the initial position OP of the object OB, the magnitudes of the recorded voice received by the fourth voice input device 12b4 and the eighth voice input device 12b8 are the same, and the plurality of voice input devices 12b1 to 12b8 The size of the received recording voice may be the largest. The sizes of the recording sounds received by the first audio input device 12b1, the third audio input device 12b3, the fifth audio input device 12b5, and the seventh audio input device 12b7 are the same, May be smaller than the size of the recording voice received by the voice input device 12b4 and the eighth voice input device 12b8. The size of the recording voice received by the second audio input device 12b2 and the sixth audio input device 12b6 is the same and the size of the recording voice received by the remaining plurality of audio input devices 12b1 to 12b8 is the same Can be small. At this time, the x-axis coordinates of the object OB are the fifth audio input device 12b5 and the eighth audio input device 12b8, the sixth audio input device 12b6, and the seventh audio input device 12b6 arranged symmetrically on both sides of the x- The ratio of the size of the recording voice received in the input device 12b7, the first audio input device 12b1 and the fourth audio input device 12b4 or the second audio input device 12b2 and the third audio input device 12b3 Axis coordinate of the object OB can be detected from the x-axis coordinate of the object OB. The y-axis coordinate and the z-axis coordinate of the object OB are also calculated in such a manner that the ratio of the size of the recording voice received to the voice input device 12b arranged symmetrically on both sides of each coordinate axis as described above The three-dimensional coordinates OP of the object OB can be detected.

를 검출할 수 있다.On the other hand, when the object OB moves in the virtual space VS, the broadcast image editing unit 121 of the control unit 120 detects a three-dimensional vector indicating a change in three-dimensional coordinates according to the movement of the object OB can do. Referring to FIG. 7, the broadcast image editing unit 121 can detect the moving position of the object OB first. That is, since the object OB has moved in the direction of decreasing the x-axis coordinate in parallel with the x-axis at the initial position, the broadcast image editing unit 121 detects the coordinates OP 'of the moving position of the object OB . At this time, the process of detecting the moving three-dimensional coordinates OP 'of the object OB is substantially the same as the process of detecting the three-dimensional coordinates OP of the object OB at the initial position described above, do. The broadcast image editing unit 121 of the control unit 120 compares the three-dimensional coordinates OP of the object OB with the initial three-dimensional coordinates OP of the object OB to detect the movement of the object OB 3D vector representing

Can be detected.

를 검출하면, 제어부(120)의 방송 음성 편집부(122)는 객체(OB)의 이동에 따른 가상의 공간(VS)에서 객체(OB)의 3차원 좌표의 변화, 즉 3차원 벡터

에 기초하여 녹음 음성을 복수의 음성 출력부(132) 각각으로 출력할 수 있다. 이는 앞서 도 6c를 참조하여 설명한 제어부(120)의 방송 음성 편집부(122)가 객체(OB)의 이동에 따른 복수의 음성 출력부(132) 각각에 대한 방송 음성의 출력을 조절하는 과정을 실질적으로 동일하므로 중복 설명을 생략한다. The broadcast image editing unit 121 of the control unit 120 displays a three-dimensional vector representing the change of the three-dimensional coordinate according to the movement of the object OB in the virtual space VS

The broadcast audio editing unit 122 of the control unit 120 changes the three-dimensional coordinates of the object OB in the virtual space VS according to the movement of the object OB, that is,

And outputs the recorded voice to each of the plurality of voice output units 132. [ This is because the broadcast voice editing unit 122 of the control unit 120 described above with reference to FIG. 6C controls the output of the broadcast voice to each of the plurality of voice output units 132 according to the movement of the object OB, So duplicate description is omitted.

In the virtual studio broadcast editing and transmitting device 100 and the broadcast editing and transmitting method using the virtual studio broadcast editing and transmitting device 100, which synchronizes the video and sound signals by converting the object position change according to another embodiment of the present invention into a vector, The three-dimensional coordinates OP of the object OB and the object ID of the object OB in the virtual space VS are calculated based on the size of the voice of the object OB received in the plurality of voice input devices 12b1 to 12b8 fixed at the plurality of positions. Dimensional coordinates in accordance with the movement of the object OB and to adjust the broadcast speech for each of the plurality of audio output units 132 in real time based on the change of the three-dimensional coordinates OP of the object OB have. Accordingly, in the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting the object position change according to another embodiment of the present invention into a vector and a broadcasting editing and transmitting method using the same, Dimensional coordinates (OP) of the object OB and the object OB using the size of the recorded voice of the object OB received by the plurality of audio input devices 12b1 to 12b8 even in this unpredictable situation Dimensional vector of the object OB can be detected, and a 3D voice with a lively feeling according to the movement of the object OB can be automatically output in real time.

FIG. 8A is a diagram illustrating a virtual studio broadcast editing and transmitting apparatus for synchronizing video and sound signals by converting object position changes into vectors according to another embodiment of the present invention, and a broadcasting editing and transmitting method using the same. It is a scene of broadcast video edited by synthesizing video in virtual studio. FIG. 8B is a diagram showing a mapping of the broadcast image of FIG. 8A into a three-dimensional virtual space. 8C is a view showing a moving image of the data image in the broadcast image of FIG. 8A. FIG. 8D is a diagram showing a state in which the broadcast image of FIG. 8C is mapped into a three-dimensional virtual space.

Referring to FIGS. 8A and 8B, the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes video and audio signals by converting the object position change into a vector according to another embodiment of the present invention, The broadcast audio editing unit 122 edits the broadcast audio data of each of the plurality of audio output units 132 based on the three-dimensional coordinates of the photographed image or the data image DV in a virtual space (VS) Can be adjusted.

Referring to FIG. 8A, a broadcast image BV is composed of an object OB and a data image DV in a photographed image. Referring to FIG. 8B, the broadcast image BV of FIG. The position of the object OB and the data image DV in the photographed image can be expressed in three-dimensional coordinates. 8B, the three-dimensional coordinates OP of the object OB in the photographed image are (Ox, Oy, Oz), the initial three-dimensional coordinates DP of the data image DV are (Dx, Dy, Dz) . Referring to FIG. 8C, the data image DV is moved backward in the broadcast image BV of FIG. 8A. Referring to FIG. 8D, the broadcast image BV of FIG. 8C is transformed into a three- When mapping, the positions of the object OB and the data image DV in the photographed image can be expressed in three-dimensional coordinates. On the other hand, in FIG. 8C, it is assumed that the data image DV moves backward, that is, it moves parallel to the z-axis and moves in the direction of decreasing the z-axis coordinate.

8A to 8D, since the object OB is fixed without moving, the three-dimensional coordinates OP of the object OB can also be the same as (Ox, Oy, Oz) Dimensional coordinates DP and DP 'of the data image DV are also changed from (Dx, Dy, Dz) to (Dx', Dy ', Dz') because they have shifted away from the object OB Can be detected. The broadcast voice editing unit 122 of the control unit 120 may extract the data image DV based on the three-dimensional coordinates DP of the data base DV and the three-dimensional vector in the virtual space VS according to the movement of the data base DV. DV) to each of the plurality of audio output units 132. [0052] FIG. This is because the broadcast voice editing unit 122 of the control unit 120 described above with reference to FIG. 6C controls the output of the broadcast voice to each of the plurality of voice output units 132 according to the movement of the object OB, So duplicate description is omitted.

Meanwhile, as described above, in some embodiments, it is possible to determine whether or not the data image DV is muted based on the z-axis coordinate value among the three-dimensional coordinates DP of the data image DV. If the z axis coordinate value of the three dimensional coordinates (DP) of the data image (DV) is less than the threshold value, the data image (DV) is located at the rear side of the virtual space (VS) , And may be muted so as not to deliver the audio corresponding to the data base DV to the viewer. Therefore, as shown in FIGS. 8A to 8D, the data image DV moves in a direction in which the z-axis coordinate decreases along the z-axis, so that the z-axis coordinate value of the data image DV is smaller than the specific z- The audio corresponding to the data video DV can be muted without being outputted.

In the virtual studio broadcast editing and transmitting device 100, which synchronizes video and sound signals by converting object position changes according to another embodiment of the present invention into a vector, and a broadcast editing and transmitting method using the same, When the data image DV in the image BV moves, the three-dimensional coordinate DP of the data image DV and the change of the three-dimensional coordinate, that is, the three-dimensional vector, are detected in the virtual space VS, The broadcast audio editing unit 122 of the control unit 120 can adjust the broadcast audio for each of the plurality of audio output units 132 in real time based on the change of the three-dimensional coordinates of the data image DV.

As described above, when the control unit 120 not only adjusts the output of the broadcast sound when the data video DV moves in the broadcast video BV, but also when the broadcast video BV moves The broadcast voice can be adjusted in real time. First, the control unit 120 detects changes in three-dimensional coordinates and three-dimensional coordinates of a photographed image in a virtual space VS, and detects a change in three-dimensional coordinates of the photographed image, that is, (132) can be adjusted in real time. Therefore, in the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting object position changes according to another embodiment of the present invention into a vector, and a broadcast editing and transmission method using the same, It is possible to automatically output real-time 3D voices according to the movement of the photographed image and the data image DV.

Figure 9A is a schematic diagram of an example for the local bank selector of Figure 3A. FIG. 9B illustrates a method for selecting and editing a first local bank in a virtual studio broadcast editing and transmitting apparatus and a broadcast editing and transmitting method using the virtual studio broadcast editing and transmitting apparatus in which an image and sound signals are synchronized by converting an object position change into a vector according to another embodiment of the present invention Fig. 8 is a diagram showing a virtual space when an input is received. Fig. FIG. 9C is a diagram illustrating a virtual studio broadcast editing and transmitting apparatus for synchronizing video and audio signals by converting object position changes into vectors according to another embodiment of the present invention, and a broadcasting editing and transmitting method using the same, In which an input for selecting a second local bank is received. FIG. 9D is a diagram illustrating a virtual studio broadcast editing and transmitting device in which video and sound signals are synchronized by converting an object position change into a vector according to another embodiment of the present invention, and a broadcasting editing and transmitting method using the same, In the case where an input for selecting the third local bank is received. In FIG. 9A, only three local banks LB are shown for convenience of explanation. FIG. 9B shows a virtual space VS for the photographed image photographed through the first camera C1 shown in FIG. 5, and FIG. 9C shows the virtual space VS taken by the second camera C2 shown in FIG. And FIG. 9C shows a virtual space VS '' of the photographed image photographed through the third camera C3 shown in FIG. 5. The virtual space VS 'shown in FIG.

Referring to FIG. 9A, the memory 150 of the virtual studio broadcast editing and transmitting apparatus 100, which synchronizes video and sound signals by converting the object position change into a vector according to another embodiment of the present invention, A plurality of local banks LB, which are 3D view settings in the first embodiment, can be stored. Here, the local bank LB is a kind of preset that stores the position of the photographed image and the data image DV to be output as the broadcast image BV in the virtual space VS in the broadcast image BV corresponding to the virtual studio to be. The virtual studio broadcast editing and transmission device 100 synchronizes the video and sound signals by converting the object position change into a vector and outputs the captured image and the material image DV constituting one local bank LB And the memory 150 can store the local bank LB based on the input information.

Thereafter, when an input for selecting a specific local bank LB among a plurality of local banks LB is received, the broadcast audio editing unit 122 switches the broadcast image BV to a specific local bank LB. For example, when the object OB moves or the photographed image or the data image DV moves in the process of switching from the initially set local bank LB to the specific local bank LB, Or the broadcast image BV is switched to the specific local bank LB in such a manner that the data image DV is linearly or rotationally moved or the x-, y- or z-axis of the virtual space VS is rotated .

Thereafter, based on the change of the three-dimensional coordinates of the photographed image and the change of the three-dimensional coordinates of the data image DV generated as the broadcast image BV is switched to the specific local bank LB, ) Can be adjusted in real time.

This is because the broadcast voice editing unit 122 of the control unit 120 described above with reference to FIG. 6C controls the output of the broadcast voice to each of the plurality of voice output units 132 according to the movement of the object OB, So duplicate description is omitted.

9A, when an input for selecting the first local bank LB1 located at the front of the object OB is received, a virtual space VS shown in FIG. 9B is mapped to the broadcast image BV, . The first local bank LB1 is a viewpoint of the object OB from the front and is the same as the first camera C1 photographing the object OB from the front in FIG. In this case, the initial three-dimensional coordinates OP of the object OB can be assumed to be (Ox, Oy, Oz).

Thereafter, when an input for switching the broadcast image BV from the first local bank LB1 to the second local bank LB2 is received, a virtual space VS 'as shown in FIG. 9C is displayed on the broadcast image BV Can be mapped. Since the second local bank LB2 is an image photographed through the second camera C2, the second local bank LB2 is the viewpoint of the camera C2 which shoots the object OB in the direction of 45 degrees of the right diagonal line in Fig. When switching from the first local bank LB1 to the second local bank LB2, for example, it is defined that the z axis and the x axis of the virtual space VS rotate counterclockwise 45 degrees with respect to the y axis You may. Or when the object OB is switched from the first local bank LB1 to the second local bank LB2 by rotating the object OB about the y axis by 45 degrees in the clockwise direction in the virtual space VS.

에 기초하여 복수의 음성 출력부(132) 각각에 대한 방송 음성의 출력이 조절될 수 있다. 이는, 로컬 뱅크(LB)의 전환 속도에 의해 결정될 수도 있고, 실제 객체(OB)의 이동과 프레임 레이트에 의해 결정될 수도 있다.(Ox, Oy, Oz) of the three-dimensional coordinates OP of the object OB in the second local bank LB2 when the z axis and the x axis are rotated in the first local bank LB1, (Ox ', Oy', Oz ') of the three-dimensional coordinates OP' of the object OB. At this time, the three-dimensional coordinate (OP) of the object OB can continuously change while drawing an arc, and the change of the three-dimensional coordinate, that is, the three-dimensional vector, The output of the broadcast sound to each of the sound output units 132 may be adjusted and the output of the three-dimensional vector (OP ') connecting the three-dimensional coordinates (OP') changed in the initial three-

The output of the broadcast voice to each of the plurality of audio output units 132 can be adjusted. This may be determined by the switching speed of the local bank LB or may be determined by the movement of the actual object OB and the frame rate.

On the other hand, when an input for switching the broadcast image BV from the first local bank LB1 to the third local bank LB3 is received, a virtual space VS '' shown in FIG. Can be mapped. Since the third local bank LB3 is an image photographed through the third camera C3, the object OB is viewed from the 45-degree diagonal line of the left diagonal in Fig. 5, This is the viewpoint of the camera C3 which shoots in the 45-degree diagonal direction. When switching from the first local bank LB1 to the third local bank LB3, for example, it is defined that the z axis and x axis of the virtual space VS rotate clockwise by 45 degrees with respect to the y axis It is possible. Or when the object OB is switched from the first local bank LB1 to the third local bank LB3 by rotating the object OB about the y axis by 45 degrees in the counterclockwise direction in the virtual space VS .

를 검출하고, 3차원 벡터

에 기초하여 복수의 음성 출력부(132) 각각에 대한 방송 음성의 출력을 조절하는 과정은 상술한 바와 실질적으로 동일하므로 중복 설명을 생략한다.(Ox, Oy, Oz) of the three-dimensional coordinates OP of the object OB in the third local bank LB3 when the z'-axis and the x'-axis are rotated in the first local bank LB1, , And (Ox '', Oy '', Oz '') of the three-dimensional coordinates OP '' of the object OB. At this time, the change of the three-dimensional coordinates of the object OB, that is, the change of the three-

And detects a three-dimensional vector

The process of adjusting the output of the broadcast audio to each of the plurality of audio output units 132 is substantially the same as that described above, so that the duplicated description will be omitted.

In the virtual studio broadcast editing and transmitting device 100 that synchronizes video and sound signals by converting the object position change according to another embodiment of the present invention into a vector and a broadcast editing and transmitting method using the same, Stores a plurality of local banks LB storing the positions of the photographed images and the data images DV to be output as the broadcast video BV in the space VS and stores the position of the object OB in the selected local bank LB according to the selected local bank LB. The three-dimensional coordinates OP are changed. Thus, the output to each of the plurality of audio output units 132 through which the broadcast audio is output can be automatically adjusted in real time. Accordingly, in the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting the object position change according to another embodiment of the present invention into a vector, and a broadcast editing and transmission method using the same, The local bank LB stores the timing of outputting the broadcast image BV and the broadcast image BV in advance so that the user can select a local bank LB at a time point at which the user desires to output the broadcast image. Can be automatically and in real time edited up to the broadcasting voice, so that a colorful and lively broadcasting image BV can be produced.

Hereinafter, with reference to FIG. 10 and FIG. 11, it will be assumed that a broadcast relay system (hereinafter, referred to as " virtual broadcast " 1000) will be described in detail.

FIG. 10 is a block diagram schematically illustrating the configurations of a real-time broadcast relay system using a virtual studio broadcast editing and transmission device in which video and sound signals are synchronized by converting object position changes into vectors according to various embodiments of the present invention. FIG. 11A is a schematic diagram illustrating packet data transmitted from a virtual studio broadcast editing and transmitting apparatus in which an object position change is converted into a vector according to various embodiments of the present invention and the video and sound signals are synchronized. 11B is a diagram showing corresponding output points of each channel of the packet data.

10, the real-time broadcast relay system 1000 includes a virtual studio broadcast editing and transmission device 100, a server 200, and a broadcast relay terminal CL that synchronize image and sound signals by converting object position changes into vectors, ).

The virtual studio broadcast editing and transmission device 100 synchronizes the video and sound signals by converting the object position change into a vector and edits the shot video and the recorded audio received from the input unit 110 to produce a broadcast video and a broadcast audio. The process of editing the photographed image and the recorded voice into the broadcast image and the broadcast voice is the same as that described above with reference to FIG. 1 to FIG. 9D, so duplicate description will be omitted. The virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and sound signals, transmits the broadcast video and broadcast voice to the server 200. Specifically, a plurality of broadcast voices for one broadcast image are bundled into one packet data (PD) and transmitted to the server.

As described above with reference to FIG. 6B, the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, The broadcast voice can be edited by giving three-dimensional coordinates to each of the plurality of voice output devices 32. [

Referring to Fig. 11A, the packet data PD includes time information TS, a plurality of channel information CH1a to CHna, and a plurality of broadcast voices CH1b to CHnb corresponding to a channel. The packet data PD are sequentially transmitted and a plurality of channel information CH1a to CHna and a plurality of broadcast voice CH1b to CHnb having the same time information TS are transmitted as one packet data PD do. The packet data PD includes the first time information TS1, the channel 1 information CH1a, the broadcast voice CH1b of the channel 1, the channel 2 information CH2a, the broadcast voice CH2b of the channel 2, CH3a), channel 3 information (CHna), and channel n broadcast speech (CHnb) are arranged in this order. That is, it is composed of the time information TS, the channel information CH1a to CHna, and the broadcast voice CH1b to CHnb of the channel. Thereafter, a plurality of channel information corresponding to the second time information TS2 and a broadcast voice of a plurality of channels are sequentially transmitted.

The time information TS indicates which point of the broadcast image the plurality of broadcast speeches CH1b to CHnb coincide with. For example, if the time information TS indicates one minute of the broadcast image, a plurality of broadcast speeches CH1b to CHnb can be simultaneously output at one minute of the broadcast image.

The channel information CH1a to CHna indicates the output point of the broadcast voice CH1b to CHnb corresponding to each channel, that is, the speaker at which point the broadcast voice CH1b to CHnb is outputted. The plurality of broadcast speeches CH1b to CHnb are edited by assigning virtual coordinates to voice output points in a virtual space VS and are edited by the volume of the broadcast sound CH1b to CHnb output from each of the channels CH1a to CHna And the like may be different.

The broadcast voice CH1b to CHnb may correspond to the broadcast channels CH1b to CHnb corresponding to the channels CH1a to CHna and the broadcast voice CH1b to CHnb may be different according to the channels CH1a to CHna. Each broadcast voice CH1b to CHnb is edited on the assumption that it will be outputted at different points, and different broadcast voices CH1b to CHnb are output according to movement of an object in a broadcast image, movement of a data image, or the like.

Referring again to FIG. 10, the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, transmits the packet data PD to the server 200, Transmits packet data again to the broadcast relay terminal CL.

The broadcast relay terminal CL outputs the received packet data PD to the video output device and the audio output device. The broadcast relay terminal CL may be implemented as a separate client device or may be implemented integrally with a PC, a video output device, or a voice output device, but is not limited thereto.

Hereinafter, the output points of the channels CH1a to CHna in the viewing environment where the broadcast relay terminal CL is installed will be described in more detail with reference to FIG. 11B.

Referring to FIG. 11B, a speaker SP, which is a sound output device disposed at an output point corresponding to the broadcast relay terminal CL, each channel CH1a to CHna, and a display 31, which is a video output device, are disposed.

Although the video output apparatus displays the broadcast video and the video output apparatus is the display 31 for the convenience of explanation, the video output apparatus may be a TV, a monitor, a beam projector, and the like. Although a plurality of audio output apparatuses are arranged and the audio output apparatus is shown as a speaker SP for convenience of explanation, the present invention is not limited thereto.

The speaker SP can be arranged around the viewer's viewing point, that is, the audience. For example, the first speaker SP1 is located on the front left side of the viewer, the second speaker SP2 is on the front center of the viewer, the third speaker SP3 is on the front right side of the viewer, the fourth speaker SP4 is on the left side The fifth speaker SP5 is positioned on the right side of the viewer, the sixth speaker SP6 is positioned on the rear left side of the viewer, the seventh speaker SP7 is positioned on the rear center of the viewer, and the eighth speaker SP8 is positioned on the rear right side of the viewer . In addition, the ninth speaker SP9 may be disposed at the center to provide a more vivid broadcast sound.

At this time, the first speaker SP1, the second speaker SP2, the second speaker SP3, and the fourth speaker SP4, CH1, CH2, CH3, The fifth speaker SP5 is connected to the channel 5 (CH5), the sixth speaker SP6 is connected to the channel 6 (CH6), the seventh speaker SP7 is connected to the channel 7 (CH7) The channel 8 (CH8) and the ninth speaker (SP9) are connected to the channel 9 (CH9), so that the broadcast voice of each channel CH can be output. At this time, the number of the channels CH and the arrangement of the speakers SP corresponding to the respective channels CH may be varied, but are not limited thereto.

On the other hand, the number of channels CH may be different from the number of speakers SP arranged in the actual viewing environment. In this case, the broadcast relay terminal CL selectively outputs only the broadcast voice of the channel CH corresponding to the speaker SP arranged in the actual viewing environment in the packet data PD. This will be described in more detail with reference to FIGS. 12A to 12B.

12A is a diagram exemplarily showing a speaker arrangement of a classroom. 12B is a diagram exemplarily showing the speaker arrangement of another classroom. 12C is a diagram exemplifying the speaker arrangement of the auditorium. 12D is a diagram exemplarily showing a speaker arrangement of another auditorium. 12A to 12D, it is assumed that the speaker arrangement of the classroom and the auditorium of the school is assumed, and the present invention is not limited to the place where broadcasting is conducted such as government offices, companies, academies, etc. in addition to the school.

Referring to FIG. 12A, a first broadcast relay terminal CL1, a first speaker SP1, and a third speaker SP3 are disposed in a classroom. The first speaker SP1 is disposed on the front left side of the viewer, that is, the student, and corresponds to the channel 1 (CH1) of the packet data PD. The third speaker SP2 is disposed on the front right side of the student and corresponds to channel 3 (CH3) of the packet data PD.

The first broadcast relay terminal CL1 may receive packet data PD including information of nine channels CH1 to CH9 from the server. However, only the first speaker SP1 and the third speaker SP3 corresponding to the channel 1 (CH1) and the channel 3 (CH3) are disposed in the classroom.

Therefore, the first broadcast relay terminal CL1 does not edit the broadcast voice of each channel CH of the packet data PD again to two channels but also broadcasts the broadcast voice of the channel 1 (CH1) and the channel 3 (CH3) To the first speaker SP1 and the third speaker SP3.

Referring to FIG. 12B, a second broadcast relay terminal CL2, a first speaker SP1, a third speaker SP3, a sixth speaker SP6, and an eighth speaker SP8 are disposed in the classroom. The first speaker SP1 is connected to the first speaker SP3 and the third speaker SP3 is connected to the third speaker CH3 while the sixth speaker SP6 is connected to the channel 6 and the eighth speaker SP8 is connected to the channel 8 ).

The second broadcast relay terminal CL2 only corresponds to the broadcast voice of channel 1 (CH1), channel 3 (CH3), channel 6 (CH6) and channel 8 (CH8) among the nine channels CH in the packet data PD To the speaker SP.

12C, a third broadcast relay terminal CL3, a first speaker SP1, a third speaker SP3, a fourth speaker SP4, a fifth speaker SP5, a sixth speaker SP6 And an eighth speaker SP8 are disposed. The first speaker SP1 is connected to the first speaker SP3 and the third speaker SP3 is connected to the third speaker CH3 while the fourth speaker SP4 is connected to the fourth speaker CH4. The sixth speaker SP6 corresponds to the channel 6 (CH6), and the eighth speaker SP8 corresponds to the channel 8 (CH8).

The third broadcast relay terminal CL3 selects the channel CH1, CH3, CH4, CH5 and CH6 of the nine channels CH of the packet data PD And channel 8 (CH8) to the corresponding speaker SP.

12D, a fourth broadcast relay terminal CL4, a first speaker SP1, a second speaker SP2, a third speaker SP3, a fourth speaker SP4, a fifth speaker SP5 , A sixth speaker (SP6), a seventh speaker (SP7), and an eighth speaker (SP8). The first speaker SP1 is connected to the first speaker SP2 and the second speaker SP2 is connected to the second speaker CH3. The fifth speaker SP5 is the channel 5 (CH5), the sixth speaker SP6 is the channel 6 (CH6), the seventh speaker SP7 is the channel 7 (CH7), the eighth speaker SP8 is the channel 8 (CH8).

The fourth broadcast relay terminal CL4 selects one of the nine channels CH of the packet data PD as the channel 1 (CH1), the channel 2 (CH2), the channel 3 (CH3), the channel 4 (CH4) ), Channel 6 (CH6), channel 7 (CH7), and channel 8 (CH8) to each corresponding speaker SP.

As described above, the arrangement of the speaker SP where the broadcast relay terminal CL is installed may not coincide with the channel CH of the packet data PD. The virtual studio broadcast editing and transmitting device 100 synchronizes the video and sound signals by converting the object position change into a vector and transmits the packet data PD without transmitting different packet data PD according to the viewing environment of various viewers. Only the data suitable for the viewing environment can be selectively output.

If packet data PD according to the viewing environment is transmitted to each of the server 100 and the broadcast relay terminal CL, it takes a lot of time and is complicated and may not be suitable for broadcasting in real time. However, since only one channel CH corresponding to the viewing environment is selectively output from one packet data PD, the packet data PD can be easily produced and transmitted, and can be broadcasted to viewers in various environments in real time.

In the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes video and sound signals by converting the object position change according to various embodiments of the present invention into a vector, and a broadcasting editing and transmitting method using the virtual studio broadcasting editing and transmitting apparatus 100, Accordingly, in the process of automatically editing the broadcasting voice corresponding to the broadcast video BV, various calculations can be applied to the controller 120 to reduce the amount of computation and perform high-speed computation. In particular, in the virtual studio broadcast editing and transmitting device 100, which synchronizes video and sound signals by converting object position changes according to various embodiments of the present invention into vectors, and a broadcasting editing and transmitting method using the same, A multimedia data real-time processing engine for voice high-speed integration processing can be used.

First, the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, applies Real Time Intelligence Format Convergence (RTIFC) technology . When receiving the photographed image from the input unit 110 of the virtual studio broadcast editing and transmission apparatus 100 that converts the object position change to a vector and synchronizes the image and sound signals, the data value for each pixel is an R value, a G value , And can be received in the form of a 32-bit map expressed up to alpha indicative of transparency in B value. However, since the load of the graphics card 140 increases rapidly even when only one bit is used, the amount of computation can be greatly increased when the data for all the pixels is calculated with 32 bits. Accordingly, the control unit 120 of the virtual studio broadcast editing and transmission apparatus 100 that converts the object position change into a vector and synchronizes the video and audio signals converts the data value of the pixel into 12 bits of the YUV422 format, . The YUV422 format is a format that handles image data with a ratio of luminance to color signal of 4: 2: 2. The YUV422 format is a format that handles image data with a ratio of luminance to color signal of 4: 2: 2, which can reduce the amount of data rather than the RGB format, but does not show the transparency value.

Since no transparency value is required in a general image, no particular problem occurs even if the YUV422 format is used as described above. However, since the transparency value is required to perform the chroma key processing as described above, there is a problem in that chroma key processing can not be performed when data is converted into the format of YUV422. In order to solve this problem, the control unit 120 of the virtual studio broadcast editing and transmission apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, converts the image that does not require chroma key processing into 12 bits of YUV 422 And the image requiring chroma key processing uses 32-bit format including transparency, so the amount of computation can be reduced compared with the conventional method.

In addition, when the control unit 120 changes the resolution of an image, semi-interpolation and interpolation may be appropriately mixed so that the image is displayed smoothly without being distorted. Conventionally, an interpolation method is used to resize an original image when resolution of an original image is different from resolution of an actual broadcast image BV. For example, if an original image having a resolution of 1920 * 1080 is divided into 30 * In order to reduce the resolution to 20, a method of extracting pixels every 70th and reducing it to 30 * 20 is used. However, such an interpolation method shrinks sharply by considering the difference between pixels linearly, so that the image is likely to be distorted. In order to solve such a problem, the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and sound signals, first converts the image of 1920 * Using a semi-interpolation that resizes to a size larger than 1/10, which is a larger size, interpolates an image of 192 * 108 and shrinks it to 30 * 20. That is, the control unit 120 of the virtual studio broadcast editing and transmission apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, does not directly resize the original image to the size of the final image at the time of resizing, It is possible to display the image more naturally by using both semi-interpolation and interpolation without rapidly changing the resolution of the image by interpolating the size of the final image after semi-interpolation with the size of the image.

Meanwhile, the control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100, which converts the object position change into a vector and synchronizes the video and audio signals, uses a multithreading technique and a process load balancing technique Real-time processing throughput can be adjusted. The control unit 120 of the virtual studio broadcast editing and transmitting apparatus 100 that synchronizes the video and sound signals by converting the object position change into a vector, processes the video and audio, processes the final processed broadcast Video (BV) and broadcast voice. Thus, a thread for processing each process may include, for example, a video delivery thread, an image editing thread, a voice editing thread, and the like. In addition, since the virtual studio broadcast editing and transmitting apparatus 100 that converts the object position change to a vector and synchronizes the video and sound signals also performs the chroma key processing, it also includes the chroma key thread for the chroma key processing .

The control unit 120 of the virtual studio broadcast editing and transmission apparatus 100 that converts the object position change into a vector and synchronizes the video and sound signals can measure the load of the control unit 120 in real time. For example, we can measure the load on various threads, such as video delivery threads, video editing threads, voice editing threads, and so on. At this time, the controller 120 measures the jitter of each thread in real time. Here, the jitter is a value indicating the time displacement of the load of each thread, and is a value indicating how fast or late the data amount processed by each thread is than the reference data amount. At this time, when the jitter in the voice editing thread, which is a thread for voice editing, is larger than a predetermined threshold value, the control unit 120 can allocate a specific thread as a dedicated thread for voice editing, The number of video transmission threads, voice transmission threads, and the like can be reduced. If the jitter in the video editing thread, which is a video editing thread, is greater than a predetermined threshold value, the control unit 120 can allocate a specific thread as a dedicated thread for image editing, The number of video transmission threads, voice transmission threads, and the like can be reduced.

The virtual studio broadcast editing and transmitting device 100, which synchronizes video and sound signals by converting object position changes according to various embodiments of the present invention into a vector, and a broadcasting editing and transmitting method using the same, The multi-channel 3D image and voice high-speed integration processing can be performed using the multimedia data real-time processing engine.

Although the embodiments of the present invention have been described in detail with reference to the accompanying drawings, it is to be understood that the present invention is not limited to those embodiments and various changes and modifications may be made without departing from the scope of the present invention. . Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Device
110: input unit
12bn: the n-th audio input device
11: Camera
12: microphone
13: Webcam
14: USB
15: Computer
120:
121: Broadcast video editor
122: Broadcast voice editor
130:
131: Video output unit
132: Audio output unit
32a to 32f: audio output device
31: Display
32: Speaker
140: Graphics card
150: Memory
160: front part
161: Power switch
162: Reset switch
163: USB terminal
164: CD-ROM
170:
171: Audio output terminal
172: Video input terminal
173: Video output terminal
174: USB terminal
175: Expansion module pack terminal
180: Main screen
181: Source Viewer
181a: External Source Viewer
181b: Internal Source Viewer
182a: A / V Player Control
182b: A / V mixer volume control
183a: preview
183b: Program view
184a: Transition bar
184b: Transition effect control
185: A / V preset control
186: PIP Object Control
187a: Local bank selector
187b: Global Bank Selector
188: A / V recorder
189: CD player
190: Output screen
191: Logo
192: Subtitle
200: Server
1000: Broadcast relay system
DV: Data Video
DP: 3D coordinates of the data image
BV: Broadcast video
VS: virtual space
LB: Local Bank
BA: Chroma background
OB: object
OP: 3D coordinate of the object
C1: First camera
C2: Second camera
C3: Third camera
O: origin
CL: broadcast relay terminal
SP: Speaker
PD: packet data
TS: Time information
CHna: Channel information
CHnb: Broadcast voice

Claims (14)

An input unit including an image input unit for receiving a photographed image in real time and a voice input unit for receiving a recorded voice corresponding to the photographed image in real time;
A memory for storing the photographed image and the recorded voice received through the input unit and storing one or more data images to be used for editing;
A broadcast image editor for receiving the photographed image from the input unit or the memory and receiving the data image from the memory to edit the broadcast image in real time, and a broadcast voice editor for editing the broadcast voice corresponding to the broadcast image in real time ; And
And an output unit including a video output unit for outputting the broadcast video and a plurality of audio output units for outputting the broadcast audio,
Wherein the broadcast image editing unit edits the shot image, the object in the shot image, and the data image in three-dimensional coordinates in the broadcast image,
Wherein the broadcast voice editing unit uses the voice corresponding to the recording voice and the data base based on the change of the three-dimensional coordinate indicating the movement of the photographed image, the movement of the object in the photographed image, and the movement of the data base And editing the broadcast audio in real time by synchronizing video and sound signals by converting object position changes into vectors. The method according to claim 1,
Wherein the broadcast image editing unit maps the broadcast image to a three-dimensional virtual space, gives three-dimensional coordinates to the photographed image, the object in the photographed image, and the data image in the virtual space,
Wherein the broadcast voice editing unit is configured to edit the broadcast data for each of the plurality of audio output units based on a three-dimensional vector indicating the movement of the photographed image in the virtual space, the movement of the object in the captured image, A virtual studio broadcast editing and transmission device that synchronizes video and sound signals by converting the object position change into a vector, which edits the audio output in real time. 3. The method of claim 2,
Wherein the broadcast voice editor comprises:
Wherein the output of the audio output unit corresponding to the left and right outputs of the plurality of audio output units is edited in real time when the captured image, the object in the captured image, or the data base moves along the x axis in the virtual space,
And outputting the output of the audio output unit corresponding to the upper and lower outputs of the plurality of audio output units in real time when the captured image, the object in the captured image, or the data image moves along the y axis in the virtual space,
And outputting the output of the audio output unit corresponding to the forward and backward outputs of the plurality of audio output units in real time when the captured image, the object in the captured image, or the data base moves along the z axis in the virtual space. A virtual studio broadcast editing and transmission device that synchronizes video and audio signals by converting the change into a vector. The method of claim 3,
Wherein the broadcast voice editor comprises:
When an object in the photographed image moves in the virtual space,
Wherein the broadcast sound for each of the plurality of audio output units is edited in real time based on a change in three-dimensional coordinates of the object in the virtual space, Studio broadcast editing and transmission equipment. 5. The method of claim 4,
Wherein the voice input unit receives the recorded voice in real time from a voice input device moving with an object in the captured image,
Wherein the broadcast voice editing unit converts the object position change into a vector and outputs the recorded voice to each of the plurality of voice output units based on a change in the position of the voice input device in the virtual space, A virtual studio broadcast editing and transmission device. 5. The method of claim 4,
Wherein the voice input unit receives a plurality of voice recordings in real time from a plurality of voice input devices fixed at a plurality of positions at a capturing place where the captured video is captured,
Wherein the broadcast image editing unit assigns three-dimensional coordinates to the fixed plurality of audio input apparatuses in the virtual space and transmits the three-dimensional coordinates to the plurality of audio input apparatuses from the fixed plurality of audio input apparatuses, Detecting movement of the object in space,
Wherein the broadcast voice editing unit outputs the recorded voice to each of the plurality of voice output units based on a change of the three-dimensional coordinates of the object in the virtual space according to the detected movement of the object, To synchronize video and audio signals. 3. The method of claim 2,
Wherein the broadcast voice editor comprises:
When the photographed image or the data image moves in the virtual space,
Wherein the broadcast sound for each of the plurality of audio output units is edited in real time based on a change in three-dimensional coordinates of the taken image or the data image in the virtual space, Virtual studio broadcast editing and transmission equipment with synchronized signals. 8. The method of claim 7,
Wherein the broadcast voice editor comprises:
When the photographed image or the data image moves out of the virtual space,
And mute the sound corresponding to the recorded voice or the data image corresponding to the photographed image, and synchronizing the video and sound signals by converting the object position change into a vector. The method according to claim 1,
Wherein the memory stores a plurality of local banks storing positions of the captured image and the data image to be outputted as the broadcast image in the virtual space,
When an input for selecting a particular local bank among the plurality of local banks is received,
Wherein the broadcast voice editing unit is configured to edit the broadcast image on the basis of the change of the three-dimensional coordinates of the photographed image and the change of the three-dimensional coordinates of the data image generated by switching the broadcast image to the specific local bank, A virtual studio broadcast editing and transmission device for synchronizing video and sound signals by converting object position changes into vectors, 10. The method of claim 9,
Wherein the broadcast voice editor comprises:
When an input is selected to select switching from a first bank to a second bank of the plurality of local banks,
Dimensional coordinate of the photographed image represented by the first bank and the three-dimensional coordinate of the data image, the three-dimensional coordinate of the photographed image represented by the second bank and the three-dimensional coordinate of the data image,
Dimensional vector for each of the captured image and the data image, and edits the broadcast speech for each of the plurality of audio output units in real time based on the three-dimensional vector, converts the object position change into a vector Virtual studio broadcasting editing and transmission device synchronizing video and sound signals. In a virtual studio broadcasting editing and transmission method of synchronizing video and sound signals by converting object position changes implemented by a virtual studio broadcast editing and transmission device into a vector by synchronizing video and sound signals by converting object position changes into vectors ,
The method comprising the steps of: receiving, in an input unit, a photographed image of an object and a recorded voice corresponding to the photographed image in real time;
Editing the broadcast video and the broadcast audio corresponding to the broadcast video in real time based on the taken image, the recorded voice, and the data image stored in the memory in the control unit; And
Outputting the broadcast image at a video output unit and outputting the broadcast audio at a plurality of audio output units,
Wherein the editing of the broadcast video and the broadcast voice in real time by the controller comprises:
Mapping the broadcast image to a three-dimensional virtual space in a broadcast image editor of the controller;
Displaying the photographed image, the object in the photographed image, and the data image in three-dimensional coordinates in the virtual space in the broadcast image editing unit; And
Editing the broadcast speech in real time based on a change in three-dimensional coordinates indicating movement of the shot image, movement of the object in the shot image, and movement of the data image in the broadcast voice editor of the controller, A method for editing and transmitting virtual studio broadcasting by synchronizing video and sound signals by converting object position changes into vectors. 12. The method of claim 11,
Wherein editing the broadcast voice in real time based on a change in the three-dimensional coordinates of the photographed image, the object in the photographed image, or the data image in the broadcast voice editor,
When the moving picture editing unit moves the photographed image, the object in the photographed image, or the data image in the virtual space along the x axis in the virtual space, a voice corresponding to the left and right output of the plurality of voice output units The output of the output unit is edited in real time,
And outputting the output of the audio output unit corresponding to the upper and lower outputs of the plurality of audio output units in real time when the captured image, the object in the captured image, or the data image moves along the y axis in the virtual space,
The output of the audio output unit corresponding to the forward and backward outputs among the plurality of audio output units is edited in real time when the shot image, the object in the shot image, or the data image moves along the z axis in the virtual space, A method for editing and transmitting a virtual studio broadcast by synchronizing an image and a sound signal by converting an object position change into a vector. A virtual studio broadcast editing and transmission device that synchronizes image and sound signals by converting a change of an object position for editing packet data and a broadcast image including a plurality of broadcast audio in real time into a vector;
A server for receiving the packet data and the broadcast image from a virtual studio broadcast editing and transmitting device that converts the object position change into a vector and synchronizes video and sound signals; And
And a broadcast relay terminal for receiving the packet data and the broadcast image from the server and outputting the packet data and the broadcast image to a video output apparatus and a plurality of audio output apparatuses,
Wherein the packet data includes time information, a plurality of channel information, and a plurality of broadcast voices corresponding to the plurality of channels,
Wherein the broadcast relay terminal selectively outputs only a channel corresponding to the plurality of audio output apparatuses out of the plurality of channels of the packet data. 14. The method of claim 13,
The virtual studio broadcast editing and transmitting device, which converts the object position change into a vector and synchronizes video and audio signals,
A broadcasting image editing unit for displaying the photographed image in the broadcast image, the object in the photographed image, and the data image in three-dimensional coordinates; And
And a voice corresponding to the recording voice and the data base based on the change of the three-dimensional coordinate indicating the movement of the photographed image, the movement of the object in the photographed image, and the movement of the data image in the broadcast voice, And a broadcast voice editing unit for editing the broadcast voice by channel.

Images