KR102024437B1 - System and method for transmitting a plurality of video image - Google Patents

Abstract

The present invention relates to a system for transmitting a plurality of captured images and a control method thereof. A control method of a plurality of captured image transmission systems according to the present invention includes the steps of: receiving respective captured image signals from a plurality of cameras; Detecting a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras; Synchronizing each of the captured image signals based on a time point at which a specific audio signal is detected from each of the captured image signals; Generating one integrated video signal by multiplexing the synchronized captured video signals; And transmitting the integrated video signal generated in the step to the receiving terminal.

Description

SYSTEM AND METHOD FOR TRANSMITTING PICTURE IMAGES AND CONTROL METHOD THEREOF {SYSTEM AND METHOD FOR TRANSMITTING A PLURALITY OF VIDEO IMAGE}

The present invention relates to a system for transmitting an image and a control method thereof, and more particularly, to a system for transmitting images captured by a plurality of cameras and a control method thereof.

Recently, various types of broadcast contents services are provided, not limited to analog broadcasting.

However, in the related art, as a broadcast station transmits only an image photographed by one camera, a viewer cannot simultaneously view images photographed at various angles or at various positions.

In particular, in the case of a sports in which multiple players participate, such a service is not provided in the related art even though there are demands of viewers to check and view the appearance of players at various positions.

For example, in a baseball game, some viewers would like to see the pitcher pitching and the first or third runner at the same time. Only the pitcher and the runner can simultaneously show the pitcher and the runner at the same time. In this case, the camera angle or the angle of view satisfies the above conditions. There is a problem, which is still insufficient to meet the needs of viewers.

Of course, although the broadcast station itself may edit and transmit images captured by a plurality of cameras into a single image, in this case, there is a problem in that it cannot be controlled separately, such as enlarging some images included in the edited image on the viewing terminal.

In other words, the PIP technology for dividing the TV screen to display a plurality of broadcast channels in each divided area is used, but as described above, a service for simultaneously displaying various types of images separately displayed in one channel is provided. Since it is not provided, there is a need for improvement.

Patent Registration No. 10-0974638

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and an object thereof is to provide a system and a method of controlling the same so that a user can simultaneously watch various positions in a stadium when watching a sporting event.

In order to achieve the above object, a plurality of captured image transmission system according to the present invention, the image receiving unit for receiving each of the captured image signal from the plurality of cameras; A detector which detects a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras; A synchronization processor for synchronizing each of the captured image signals based on a time point at which a specific audio signal is detected from the captured image signal by the detector; An integrated image generator for multiplexing each captured image signal synchronized by the synchronization processor to generate one integrated image signal; And a transmitter for transmitting an integrated video signal generated by the integrated video generator to a receiving terminal.

In addition, to achieve the above object, a control method of a plurality of captured image transmission system according to the present invention comprises the steps of: receiving respective captured image signals from a plurality of cameras; Detecting a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras; Synchronizing each of the captured image signals based on a time point at which a specific audio signal is detected from each of the captured image signals; Generating one integrated video signal by multiplexing the synchronized captured video signals; And transmitting the integrated video signal generated in the above step to the receiving terminal.

As described above, according to the present invention, the sports competition broadcast viewer can simultaneously watch the athletes of various positions or the game situation of various positions in one broadcasting channel.

In particular, since the captured image transmission system transmits an integrated image signal including the synchronized information, the viewer terminal using the same may allow time synchronization between the images simultaneously displayed, so that the viewers may not be able to view the images due to the time difference between the images. Discomfort can be minimized.

In particular, since the photographed image transmission system uses a specific audio signal included in each photographed image to perform synchronization on photographed images received from a plurality of cameras, there is no need for a complicated device for synchronization processing, and a specific audio signal is detected. Only algorithms can handle accurate time synchronization.

1 is a schematic structural diagram of an entire communication system including a captured image transmission system according to an embodiment of the present invention;
FIG. 2 is a functional block diagram of FIG. 1;
3 is a control flowchart of an entire communication system including a captured image transmission system according to an exemplary embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, each embodiment according to the present invention is only one example to help understanding of the present invention, and the present invention is not limited to these embodiments. In particular, the present invention may be composed of a combination of at least one or more of individual configurations, individual functions, or individual steps included in each embodiment.

In particular, for convenience, some claims in the claims include alphabets such as '(a)', but these alphabets do not prescribe the order of each step.

In addition, each signal referred to in each embodiment according to the present invention may refer to one signal transmitted by one connection or the like, but may also mean a series of signal groups transmitted for the purpose of performing a specific function described below. . That is, in each embodiment, a plurality of signals transmitted at predetermined time intervals or transmitted after receiving a response signal from the counterpart device may be represented by one signal name for convenience.

A schematic configuration of an entire communication system including a captured image transmission system 100 for transmitting a plurality of captured images to a viewer terminal 400 according to an embodiment of the present invention is illustrated in FIG. 1.

In the drawing, each of the cameras 210, 220, 230 and 240 is an image captured and transmitted. In the present embodiment, the cameras 210, 220, 230 and 240 may be installed in the same place (for example, a sports stadium) and photographed.

In this embodiment, each camera 210, 220, 230. 240 is installed in various places of the baseball stadium for the relay of the baseball game as an example.

Each of the cameras 210, 220, 230. 240 transmits a captured image signal to the captured image transmission system 100. As such, the captured image signal transmitted by each camera 210, 220, 230. 240 may be an analog signal or a digital signal. Signals and sound signals may be included.

On the other hand, the audio signal generator 300 emits a specific sound, for example, may emit a predetermined inaudible audio signal. Of course, the audio signal generator 300 may not only emit an inaudible audio signal, but may also emit an audible audio signal.

Since the audio signal generator 300 is necessary for synchronizing the captured images of the cameras 210, 220, 230 and 240 as described below, if the cameras 210, 220, 230 and 240 are gathered near each other but only the photographing angles are different, one audio signal generator ( Only 300 may be provided.

On the other hand, if the cameras 210,220,230.240 are far from each other, the time that the sound emitted from the audio signal generator 300 reaches the cameras 210,220,230.240 may be different, so in this case the same from each camera 210,220,230.240 Each audio signal generator 300 may be installed at a distance, and the audio signal generator 300 needs to be controlled so that the time at which the specific sound is emitted is the same.

Of course, when the camera 210, 220, 230. 240, where a certain type of sound naturally occurs periodically or intermittently in the location where the camera is photographed, such a separate audio signal generator 300 is not necessarily required.

On the other hand, the viewer terminal 400 is to receive and display the image content from the captured image transmission system 100, as shown in Figure 1 may be composed of a set-top box 410 and a television 420, or smart It may correspond to a portable wireless communication terminal such as a phone.

As such, the viewer terminal 400 may correspond to any type of terminal including a function of receiving, playing, and displaying video content transmitted by the captured image transmission system 100, that is, an integrated video signal, which will be described later.

The photographed image transmitting system 100 receives a photographed image signal from each of the cameras 210, 220, 230 and 240 described above, performs a synchronization process for each of the photographed image signals, and then integrates and transmits the captured image signal to the viewer terminal 400. do.

An example of a specific functional block of the captured image transmission system 100 is shown in FIG. 2.

As shown in the figure, the captured image transmission system 100 may include an image receiver 110, a detector 120, a synchronization processor 130, an integrated image generator 140, and a transmitter 150. Can be.

The image receiving unit 110 performs a function of receiving each captured image signal from the plurality of cameras 210, 220, 230.

That is, referring to FIG. 1, the image receiver 110 receives the captured image signal from the first camera 210, the second camera 220, the third camera 230, and the fourth camera 240, respectively.

The detector 120 detects a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras 210, 220, 230, and 240.

For example, when the audio signal generator 300 emits an audible audio signal, when the captured video signal transmitted by the cameras 210, 220, 230. 240 includes an audible audio signal of the corresponding audio signal generator 300, the detector 120 may detect the audio signal generator 300. It detects this.

Of course, when the audio signal generator 300 emits an inaudible audio signal, when the captured video signal transmitted by the cameras 210, 220, 230. 240 includes the inaudible audio signal of the corresponding audio signal generator 300, the detector 120 This may be detected.

Which audio signal the detection unit 120 detects may be set in advance in response to the audio signal emitted by the audio signal generator 300.

That is, the detector 120 may be configured to detect an audio signal of a specific audible frequency band (ie, an audible audio signal) and / or an audio signal of an inaudible frequency band (ie, an ultrasonic signal as an inaudible audio signal, for example). Can be.

The detecting unit 120 may detect a specific audio signal from the captured image signal by detecting a specific pattern while analyzing a sound signal of the captured image signal in a time domain or a frequency domain.

The synchronization processor 130 performs a function of synchronizing each of the captured image signals based on a time point when a specific audio signal is detected from each of the captured image signals by the detector 120.

For example, the synchronization processor 130 may generate each synchronization video signal in which synchronization time information based on a time point when a specific audio signal is detected from each captured video signal is matched with each corresponding captured video signal. have.

Here, the synchronized video signal may be an analog signal, but when processed as a digital signal, for example, the synchronized video signal may be an MPEG transport stream (ie, an MPEG transport stream, or TS for short) format (hereinafter referred to as a TS file). . The MPEG transport stream uses a packet including a header and a payload as its basic data units. Since the structure itself corresponds to a known technology, a detailed description thereof will be omitted.

In this case, the synchronization processor 130 generates a TS file corresponding to each captured video signal, and at a time when a specific audio signal is detected in each captured video signal in the TS file (preferably for each packet of the TS file). The corresponding time information, that is, the above-described synchronization time information is included.

That is, the process of including the synchronization time information corresponding to the point in time at which the specific audio signal is detected in each captured image signal in the TS file corresponding to each captured image signal may correspond to the synchronization process mentioned in the present embodiment. .

For example, the first specific audio signal is detected in the fourth frame among the frames of the captured video signal received from the first camera 210, and among the frames of the captured video signal received from the second camera 220. When the first specific audio signal is detected in the sixth frame, six of the frames of the TS packet including the fourth frame of the captured video signal of the first camera 210 and the captured video signal of the second camera 220. Each of the TS packets including the first frame may include synchronization time information of '1', and as another example, the second specific audio signal in the sixth frame of the frame of the captured video signal received by the first camera 210 may be included. Is detected, and when a second specific audio signal is detected in the eighth frame among the frames of the captured video signal received by the camera 220 A synchronization of '2' to a TS packet including a sixth frame of frames of the captured video signal of the first camera 210 and a TS packet including an eighth frame of the frames of the captured video signal of the second camera 220 Time information can be included.

In addition, the synchronization processor 130 may remove a specific audio signal included in each captured video signal when generating a synchronized video signal corresponding to each captured video signal.

That is, when a specific audio signal is detected in each captured video signal received from the plurality of cameras 210, 220, 230. 240, the synchronization processor 130 removes the specific audio signal included in each captured video signal and then removes the specific audio signal. It is possible to generate a synchronization image signal including both the removed image signal and the above-described synchronization time information.

Accordingly, the specific audio signal may be used only to determine synchronization time information, and may not be output when the viewer terminal 400 watches the corresponding image.

In particular, in removing the specific audio signal, the synchronization processor 130 may consider the frequency band of the detected specific audio signal.

For example, when a specific audible audio signal is detected by the detector 120, the synchronization processor 130 performs a process of removing the specific audible audio signal from each captured image signal as described above. If a specific inaudible audio signal is detected at 120, the process of removing the specific inaudible audio signal from each captured image signal may not be performed.

Accordingly, the system processing load can be minimized by eliminating the viewer from listening to a specific audible audio signal inserted for time synchronization, and eliminating the removal process for the specific audible audio signal.

Meanwhile, the integrated image generator 140 performs a function of multiplexing each captured image signal synchronized by the synchronization processor 130 to generate one integrated image signal.

In this case, the integrated video signal may be an analog signal. However, in the present embodiment, the integrated video signal is video data subjected to digital processing.

For example, when the synchronization processor 130 generates the respective synchronization image signals by matching the synchronization time information based on the time point at which the specific audio signal is detected from each of the captured image signals with the corresponding respective captured image signals, The integrated image generator 140 multiplexes each of the synchronized image signals and generates one integrated image signal.

Here, one integrated video signal generated by the integrated image generator 140 may correspond to a video signal that can be viewed on one viewing channel.

That is, as mentioned above, if the synchronized video signal is a file of a TS format, the integrated video signal may be an integrated file composed of a combination of each of these TS files. It can be transmitted in a streaming manner.

Here, the term “file” does not necessarily include all the images from the beginning to the end of the image, but may be a file for each section generated in real time by the captured image signal received from each camera 210, 220, 230.

The transmitter 150 transmits the integrated image signal generated by the integrated image generator 140 to the receiving terminal.

For example, the transmitter 150 may transmit an integrated video signal after connecting a communication channel capable of mutual communication with the viewer terminal 400, or may not connect the communication channel with the viewer terminal 400. The integrated video signal may be transmitted by transmitting by radio waves, such as a direction broadcasting station.

As described above, the image is generated by the integrated image generating unit 140 and then transmitted in succession of packet units including the integrated TS file header and the payload, which are transmitted by the transmitting unit 150 to the viewer terminal 400. Specific information may be included in an initial header for indicating that the viewer terminal 400 is a stream for transmitting a plurality of TS files.

For example, if PID (packet ID) is 0 in the header for the session connection, the viewer terminal 400 includes a value that specifies how many TS files (ie, how many programs) to send in the subsequent payload. It can be seen.

For example, if the header and payload of the first integrated TS file stream for a session connection specify that three programs are included in the integrated TS file, the captured image transmission system 100 may determine that subsequent packets (headers 3 TS files (programs) are successively sent to + payload), and this process can be repeated.

That is, the captured image transmission system 100 may transmit the TS packet, the TS packet, the TS packet, and the TS packet after the TS packet, the TS packet, and the TS packet.

In addition, the integrated image generation unit 140 may generate an integrated TS file by including information on how many TS files are included in the thin packet (header and payload), and thereafter, respectively. The process may then sequentially include attaching individual TS files.

Hereinafter, an overall control flow including a captured image transmission system 100 according to an exemplary embodiment will be described with reference to FIG. 3.

First, each camera 210, 220, 230. 240 captures an image and transmits a captured image signal corresponding thereto to the captured image transmission system 100.

Accordingly, the captured image transmission system 100 determines whether a predetermined specific audio signal is detected from the captured image signals received from each of the cameras 210, 220, and 230.

As a result of the determination, when a predetermined audio signal is detected from each captured video signal, the captured image transmission system 100 performs a synchronization process for each captured image signal based on the detected specific audio signal.

Here, the synchronization process may be, for example, matching the same synchronization time information with each frame corresponding to the point in time at which the same specific audio signal is detected.

Subsequently, the captured image transmission system 100 generates a file (for example, a TS file) in a separate format for transmission by using the captured image signal received from each of the cameras 210, 220, 230, and 240, and matches the time information for synchronization as described above. In this case, the matched synchronization time information may be included in each TS file.

Of course, when a specific audio signal is not detected from each captured image signal as a result of the determination in step S5, the captured image transmission system 100 may generate each TS file not including the above-described synchronization time information.

Thereafter, the captured image transmission system 100 generates an integrated TS file obtained by multiplexing each TS file.

Here, the TS file as well as the integrated TS file may correspond to a content stream generated in real time.

Subsequently, the captured image transmission system 100 may transmit the generated integrated TS file to the viewer terminal 400 through one broadcast channel.

For example, when the viewer terminal 400 selects broadcast channel number 5 to watch a baseball relay game, the captured image transmission system 100 transmits to the plurality of cameras 210, 220, 230. 240 through the broadcast channel 5. It is possible to transmit a plurality of TS files each including time information for synchronization as well as a captured image photographed by the integrated form (integrated TS file).

That is, the captured image transmission system 100 transmits the above-described integrated TS file to the viewer terminal 400 through the same IP: port.

Accordingly, the viewer terminal 400 may extract and display individual TS files included in the integrated TS file transmitted by the captured image transmission system 100.

For example, the viewer terminal 400 may display an image pitched by a pitcher on one screen as a main image, and simultaneously display an image of a first baseman, an image of an outfielder, an image of a director, and the like.

In particular, since each individual TS file includes time information for synchronization, the viewer terminal 400 may synchronize images simultaneously displayed at the same time.

For example, the viewer terminal 400 corresponds to a TS file corresponding to the first camera 210, a TS file corresponding to the second camera 220, and a third camera 230 from the captured image transmission system 100. When receiving an integrated TS file including a TS file in a streaming manner and simultaneously displaying images taken by the respective cameras 210, 220, 230. 240 on the screen, synchronization included in the TS file extracted from the received integrated TS file Using the time information, images captured at the same time may be displayed at the same time.

That is, the time for transmitting the captured image may be different for each of the cameras 210, 220, 230. 240, or some of the captured image signals transmitted by the cameras 210, 220, 230. 240 may be delayed due to network conditions. The system 100 includes synchronization time information based on the timing point at which a predetermined audio signal is detected from each captured video signal in each TS file, so that the viewer terminal 400 is configured to display each TS file included in each integrated TS file. By using the time information for synchronization it is possible to display images taken at the same time at the same time.

On the other hand, the process of performing each of the above-described embodiments can be performed by a program or an application stored in a predetermined recording medium (for example, computer readable). Here, the recording medium includes both an electronic recording medium such as a random access memory (RAM), a magnetic recording medium such as a hard disk, an optical recording medium such as a compact disk (CD), and the like.

In this case, the program stored in the recording medium may be executed on hardware such as a computer or a smartphone to perform the above-described embodiments. In particular, at least one of the functional blocks of the photographed image transmission system according to the present invention may be implemented by such a program or application.

In addition, this invention is not limited to the above-mentioned specific Example, It can implement in a various deformation | transformation and correction within the range which does not deviate from the summary of this invention. It will be apparent that such variations and modifications are included in the present invention as long as they belong to the appended claims.

100: recording video transmission system 200: camera
300: audio signal generator 400: viewer terminal
110: image receiver 120: detector
130: synchronization processing unit 140: integrated image generation unit
150: transmission unit

Claims (14)

(a) receiving respective captured image signals from a plurality of cameras;
(b) detecting a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras;
(c) synchronizing the respective captured image signals based on a time point at which a specific audio signal is detected in the captured image signals;
(d) multiplexing each of the synchronized photographed video signals to generate one integrated video signal;
Transmitting the integrated video signal generated in step (d) to a receiving terminal;
In the step (c), generating the respective synchronized video signals including the synchronization time information based on the time point at which the specific audio signal is detected from the respective captured video signals with the corresponding respective captured video signals. Steps;
In the step (d), the control method for a plurality of captured image transmission systems, characterized in that to generate a single integrated video signal by multiplexing each of the synchronization video signal. delete The method of claim 1,
In the step (c), after the detected specific audio signal is removed from each of the captured video signals, the respective synchronized video signals are generated. The method of claim 1,
Each captured image signal received from the plurality of cameras includes a specific audio signal emitted from an audio signal generator located near each camera.
In the step (b), the specific audio signal emitted from the audio signal generator is detected from each of the captured image signals received from the plurality of cameras. The method of claim 4, wherein
And a specific audio signal detected in each captured video signal received from the plurality of cameras in step (b) is a preset inaudible audio signal. The method of claim 1,
The integrated video signal generated in step (d) is a video signal that can be viewed on one viewing channel. A computer-readable recording medium having recorded thereon a program for executing the method of claim 1. An application program stored in a computer readable recording medium in combination with hardware for executing the method of any one of claims 1 to 3. An image receiver configured to receive respective captured image signals from a plurality of cameras;
A detector which detects a predetermined specific audio signal from each of the captured image signals received from the plurality of cameras;
A synchronization processor for synchronizing each of the captured image signals based on a time point at which a specific audio signal is detected from the captured image signal by the detector;
An integrated image generator for multiplexing each captured image signal synchronized by the synchronization processor to generate one integrated image signal;
A transmitter for transmitting an integrated video signal generated by the integrated video generator to a receiving terminal;
The synchronization processor generates each synchronization video signal including the synchronization time information based on a point in time at which a specific audio signal is detected from each of the captured video signals and matching the corresponding captured video signal, and including the same.
And the integrated image generating unit multiplexes each of the synchronized image signals to generate one integrated image signal. delete The method of claim 9,
The synchronization processor is further configured to generate the respective synchronized video signals after removing the detected specific audio signal from each of the captured video signals. The method of claim 9,
Each captured image signal received from the plurality of cameras includes a specific audio signal emitted from an audio signal generator located near each camera.
And the detection unit detects a specific audio signal emitted from the audio signal generator in each captured image signal received from the plurality of cameras. The method of claim 12,
And a specific audio signal detected from each captured image signal received from the plurality of cameras is a preset inaudible audio signal. The method of claim 9,
And one integrated video signal generated by the integrated video generator is a video signal that can be viewed on one viewing channel.

Images