JP2008118602A - Television broadcast video image constitution system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a novel video scan system for attaining a wide visual field of 360 degrees in television broadcasting, also including an audio processing system suited thereto. <P>SOLUTION: The present invention captures a visual field as a cylindrical coordinate system, converts the entire visual field into time-sequential signals while scanning the coordinate system in a longitudinal (y) direction and moving a scan position in a circumferential direction (θ), and reconstitutes a video image through the procedures, so that the problem can be solved by such a scan system. At such a time, arbitrary a portion of the visual field can be extracted and displayed by designating four points on arbitrary scan coordinates as a video display area. Furthermore, it is regarded important to match directions of video information and a sound source and audio is multiplexed on a video signal in a visual field direction matched with its direction of arrival. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to the field of television broadcasting technology, and more particularly, to a scanning method for forming an image.

  The present invention is characterized in that the video corresponds to the entire viewing angle of the viewer, that is, 360 degrees, and by cutting out all or any part of the video and visualizing it, the viewer can be given any line of sight. It can be directed in the direction.

  A TV image has a viewing angle defined by the field of view of the camera, and the image is scanned in the vertical (y) direction while scanning in the horizontal (x) direction, thereby converting a two-dimensional image into a time-series video signal. In the display, the video was reconstructed by observing this procedure.

  In this method, the displayed image is limited to the field of view and direction of the camera, and the viewer's request for a wide field of view and an arbitrary line-of-sight direction cannot be satisfied. Therefore, since the field of view is narrow, there is a problem that information of a part that does not enter the field of view is lost, and as a result, incorrect information may be transmitted.

  The present invention relates to a technique for reconstructing an image by scanning a wide visual field seamlessly.

The problem you are trying to solve

  In conventional television broadcasting, the size of an image depends on the field of view of the photographing apparatus, and the direction of photographing, that is, the direction of the camera depends on the judgment of the camera operator. Therefore, there is a problem that the field of view and direction desired by each viewer are not satisfied, and the field of view is narrow, so that information that does not enter the field of view is lost.

  If the lack of information is done intentionally, there is no possibility of manipulating information, and the social trust of the information system may be significantly impaired. In an upcoming information society, information systems must be highly trusted, and at least maintaining technical reliability of information is important as security for the maintenance and development of an information society. It is.

  On the other hand, even if it is intended to widen the field of view of an existing photographing apparatus, a conventional scanning line is a method of moving in the vertical direction while scanning in the horizontal direction, so that a frame for defining the scanning range is required. Therefore, in order to acquire and reconstruct a wide field image, a plurality of images are connected to form the entire field of view. For image processing of a video that spans multiple frames, it is indispensable to repeat complex coordinate transformations. Therefore, the conventional scanning method for forming an image is a method that is not suitable for high-speed image processing.

Further, even if a wide-field video can be acquired by a commercially available panoramic camera or the like, the video is greatly distorted because the conventional video transmission method and receiver are not compatible with such a video method.

The present invention is a method for converting a wide-field image into a time-series signal without distortion by moving in the circumferential direction of the field of view while scanning in the vertical direction, and extracting it at an arbitrary position and size in the display. In order to meet the above requirements, a system for freely displaying from a wide field of view to an extracted partial video is provided.

  In addition, it is necessary to obtain sound from various directions for a wide field of view video, but if the direction of the sound source is not aligned with the direction of the visual field of the image, the sound information becomes uncomfortable. The present invention relates to solving such problems in wide-angle viewing television broadcasting.

Issues to solve

  It is an object of the present invention to provide a new video composition method for realizing a wide field of view of 360 degrees in television broadcasting and an audio processing method that can be adapted thereto.

Means to solve

  In order to achieve the aforementioned object, the present invention acquires video and audio having the same amount of information as the viewer is on the spot, and transmits the video and audio in a state where the direction of the visual field and the direction of the audio source are aligned. After that, the gist is that it is possible to display information equivalent to being present on the spot.

  The video composition method of the present invention assumes a cylindrical coordinate video space meaning a total viewing angle of 360 degrees, scans in the longitudinal (y) direction of the cylindrical coordinates, and scans while moving it in the circumferential (θ) direction. It depends on the scanning method of repeating. Therefore, this system can create a seamless 360-degree video without requiring a scanning space called a screen frame, which was necessary in the conventional system.

  Audio is multiplexed into a video signal and processed as a unit. At this time, multiplexing is performed so that the direction of the sound source coincides with the direction of the visual field, and video transmission and recording are performed.

The viewer who has received the signal uses the cylindrical coordinates (y, θ) as the spatial position of the video signal, and {(y ₁ , θ ₁ ), (y ₂ , θ ₁ ), (y ₁ , θ ₂ ), By specifying (y ₂ , θ ₂ )} and selecting and extracting the reproduced video area, it is possible to display it on an arbitrary screen size. In addition, it is possible to automatically extract the voice that matches the part.

  The present invention relates to a patent application Japanese Patent Application No. 2004-118991 and relates to a specific scanning / video composition system and a video / audio composite system.

  Hereinafter, an embodiment of the present invention will be described with reference to FIG. The field space to be scanned is a two-dimensional cylindrical coordinate system represented by height y and circumferential angle θ. The height y means the vertical width of the line of sight, and θ means the viewing angle. The scanning lines have different vertical directions, but scanning may be performed only in the upward direction, only in the downward direction, or alternately in the vertical direction. Audio information from a sound source in the same direction as the visual field direction is multiplexed on the video signal.

FIG. 2 shows an example of a screen being reproduced. A part of the video space represented by any four designated points is extracted and displayed. In this case as well, the video is reproduced and configured according to the scanning procedure when the video is acquired.

At this time, the audio information is also synchronized with the reproduction of the video, and the audio environment that matches the video can be reproduced.

In the above coordinate system, when θ = 360 degrees, the circuit goes around once and returns to the original state. However, since time has passed, the rotational scanning is seamless (seamless) as shown in FIG. Become. Hereinafter, it will be referred to as a “spiral scanning method”.

  An embodiment of the present invention is shown in FIGS. FIG. 2 illustrates a screen 4 on which four points are designated and an image is cut out and displayed from the scanning space, using scanning lines 3. The solid line portion of the scanning line indicates the scanning line at the time of display. At the time of photographing, the scanning area extends to the portion indicated by the broken line but is not displayed.

FIG. 4 shows the entire broadcasting system from shooting to display.
When the image pickup device 11 such as a CCD or CMOS is controlled to be scanned in the vertical direction by the control signal 13 from the control device 12, the time-series video signal 14 is directed to the control circuit 12. Is output.

At this time, if an image exists in the direction of the designated visual field, a video signal is transmitted, and if no image exists, that is, if the photographing apparatus itself does not have a visual field in that direction, NULL is generated.

    The control device 12 is connected to the microphone array 15. The sound source direction is determined from the sound waveform vector included in the sound signal 16 from the microphone array 15, and the sound is detected by the time division method, the frequency division method, the phase division method, or the like on the video scanning line of the visual field having the same direction. The signal is multiplexed into the video signal.

After the video signal 17 and the multiplexed signal 18 are confirmed on the monitor 19, the video signal 17 and the multiplexed signal 18 are stored as video recordings on the recording device 20, while being transmitted to the television video distribution line 23 by the television broadcast signal transmission device 21.

  When the recorded video image and the edited image are reproduced, the reproduced video signal 22 is transferred to the signal transmission device 21. The television broadcast signal multiplexed with the audio is transmitted to the viewer through the distribution line 23.

The viewer receives at the television broadcast receiver 24. Since video and audio are multiplexed in the television broadcast signal, the control device 25 separates them and the control device 25 extracts the designated display area.

The extracted video portion is displayed as a video by the video display device 26. The video is reconstructed by scanning in the same procedure as the video signal is acquired. At this time, as shown in FIG. 2, an arbitrary portion is extracted and displayed by a method in which only the portion inside the screen area designated by the scanning line is sent out and displayed on the screen and the outside portion is not shown as a sending mark.

  The sound is generated by a speaker array arranged at a position corresponding to each direction in which the visual field direction and the sound direction are aligned.

A cylindrical coordinate system, which is an image acquisition space according to the present invention, and an upward scanning line and a downward scanning line are shown. 2 shows a display screen of a receiver according to the invention. The screen is extracted by designating arbitrary four points from the transmitted video signal. At this time, if the captured video information exceeds the screen area, the video information of the scanning line portion outside the designated area is not displayed. In the cylindrical coordinate system, the scanning is two-dimensional in the y direction and the θ direction. However, when θ is scanned from 0 to 360, the time has elapsed, It becomes a scanning locus. This is called a spiral scanning method, and this figure expresses this situation. 1 illustrates a conceptual configuration of a video system for implementing a video configuration method according to the present invention.

Explanation of symbols

1 Cylindrical coordinate display of the scanned video area (the vertical axis y is the scanning range above and below the line of sight, and the horizontal axis is the angular direction θ of the cylindrical coordinates)
2 Scanning line during video shooting (upper and lower zigzag scanning is shown)
3 Receiver scanning line (scanning line on the receiver surface. The dotted line is the scanning line at the time of photographing, and the solid line is the scanning line at the time of display)
4 Receiver surface (Video clipped at 4 coordinate points is displayed)
11 Imaging device (CCD or CMOS device, etc.)
12 Control device (controls operations such as scanning direction and scanning speed of the image sensor)
13 Control signal (signal that controls the operation of the image sensor)
14 Video signal (Time series signal of cylindrical coordinate 2D image scanned by specified form)
15 Microphone array (A combination of a number of microphones, and the sound source position is determined by the volume ratio, phase difference, or waveform difference from each microphone.)
16 Audio signal (Audio signal from microphone array)
17 Video signal (time-series signal obtained from the camera)
18 A multiplexed signal of a video signal and an audio signal (a signal obtained by multiplexing the above-described video signal 7 and audio signal 6 and multiplexed so that the sound source direction of the audio and the viewing angle direction of the video match)
19 Monitor device (can check video and audio matching etc.)
20 Video recording device (video recording device that saves acquired video and audio signals)
21 TV broadcast signal transmission device (device that sends video signal multiplexed with audio to TV broadcast distribution line)
22 Playback video signal (multiplexed signal from video recorder)
23 TV broadcast distribution line (line that distributes TV broadcast signals to recipients, either a net line or a dedicated line)
24 TV broadcast receiver 25 Control device (separates video and audio components from a TV broadcast signal and extracts a designated display area)
26 Video display device 27 Speaker array (Sound components are pronounced by speakers arranged at positions matching the direction of the sound source.)

Claims (6)

(Overall method) A television broadcast video scanning method, in which a cylindrical space corresponding to a viewing angle of 360 degrees is scanned in the vertical (y) direction while rotating in the angular direction (θ) to perform virtual three-dimensional scanning. A television broadcasting system characterized in that a video is formed by using a space as a cylindrical two-dimensional (y-θ) video space. (Camera and signal format) The entire field of view can be obtained regardless of the range of the field of view actually captured by the method of rotating and scanning in the angle method of the cylindrical coordinate corresponding to the field of view while scanning in the vertical direction as described in claim 1. A television broadcast photographing apparatus characterized by acquiring 360-degree cylindrical two-dimensional video space information as a time-series signal. (Viewing angle dependent multiplexing of video and audio) The viewing angle dependent audio having a sound source in a direction that coincides with the viewing direction (θ) is multiplexed with the time series video signal described in claim 2 on the video signal. A viewing angle-dependent audio multiplex television signal system. (Video recording) A video recording system and apparatus characterized in that a signal multiplexed by the audio and video multiplexing system described in claim 3 is recorded as a video signal. (Receiver and adapter) Based on the video signal described in claim 2 or the multiplexed signal described in claim 3, the entire video or any part of the video is selectively extracted, reconstructed and displayed. A television receiver characterized by having a function, and a video processing apparatus having a function of converting to another video composition system. (Broadcast equipment, transmission equipment) A broadcast signal processing equipment characterized by conforming to the video composition system described in claim 1, and the video signal transmission equipment of the system.