JP2012213070A - Video signal generation device, video signal generation method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a stereoscopic video signal which enables acquisition of stereoscopic video further enhancing presence in a video signal generation device applied to a stereoscopic video reproduction device such as a digital television broadcast reception device.SOLUTION: A depth information change unit of a video signal generation device for generating a video signal for stereoscopic video based on depth information changes the depth information based on audio information. Thereby, a video signal generation unit generates a video signal based on the depth information changed by the depth information change unit.

Description

  Embodiments described herein relate generally to a video signal generation device, a video signal generation method, and a control program.

  Currently, development of a technique for allowing a viewer to recognize a stereoscopic image using a flat image display screen is underway. This technology prepares two types of video images having parallax corresponding to the distance between both human eyes, allowing the right-eye video to be viewed by the viewer's right eye, and the left-eye video to be viewed by the viewer's left eye. It provides a stereoscopic view to the user. Specifically, the right-eye video and the left-eye video are alternately displayed on the same video display screen, and the left-eye shutter is displayed when the right-eye video is displayed on the stereoscopic glasses worn by the viewer. Is closed, and when the left-eye video is displayed, control is performed so that the right-eye shutter is closed (see, for example, Patent Document 1).

JP 2010-268433 A

By the way, such a stereoscopic video playback technique has been put into practical use at present, but in the future, a stereoscopic video playback technique with a higher sense of reality is desired.
Accordingly, an object of the present invention is to provide a video signal generation device, a video signal generation method, and a control program for generating a stereoscopic video signal capable of obtaining a stereoscopic video image with a further improved sense of reality.

The video signal generation device of the embodiment generates a video signal for stereoscopic video based on depth information.
Then, the depth information changing unit of the video signal generating device changes the depth information based on the audio information.
Thus, the video signal generation unit generates a video signal based on the depth information changed by the depth information change unit.

FIG. 1 is a block diagram illustrating a signal processing system of a digital television broadcast receiver according to an embodiment. FIG. 2 is a functional block diagram of the signal processing unit in the first embodiment. FIG. 3 is a functional block diagram of a signal processing unit in the second embodiment. FIG. 4 is a functional block diagram of a signal processing unit in the third embodiment. FIG. 5 is a conceptual explanatory diagram of assignment of depth information based on audio information for an object (object) in an image.

Next, a preferred embodiment will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram illustrating a signal processing system of a digital television broadcast receiver according to an embodiment.
A digital television broadcast receiving apparatus (hereinafter referred to as digital television) 11 shown in FIG. 1 not only performs video display based on a video signal for normal planar view (two-dimensional) display but also stereoscopic view (three-dimensional). ) Video display based on the video signal for display can also be performed.

  As shown in FIG. 1, a digital television 11 selects a broadcast signal of a desired channel by supplying a digital television broadcast signal received by an antenna 12 to a tuner unit 14 via an input terminal 13. It is possible.

  The digital television 11 supplies the broadcast signal selected by the tuner unit 14 to the demodulation / decoding unit 15 and restores it to a digital video signal, a digital audio signal, etc., and then outputs it to the signal processing unit 16. The signal processing unit 16 performs predetermined digital signal processing on the digital video signal and audio signal supplied from the demodulation / decoding unit 15.

  Here, the predetermined digital signal processing performed by the signal processing unit 16 includes processing for converting a normal planar (two-dimensional) display video signal into a stereoscopic (three-dimensional) display video signal and audio signal, At this time, a video signal for stereoscopic display is generated based on a digital video signal and a digital audio signal input from the demodulation / decoding unit 15 or an OSD signal such as caption input from the OSD signal generation unit 19 described later. Processing for generating depth information, processing for converting a video signal for stereoscopic display into a video signal for planar display, and the like are also included.

Further, the signal processing unit 16 outputs a digital video signal to the synthesis processing unit 17 and outputs a digital audio signal to the audio processing unit 18.
The synthesizing processing unit 17 converts the digital video signal supplied from the signal processing unit 16 into video signals for superimposition such as subtitles, GUI (Graphical User Interface), and OSD generated by an OSD (on screen display) signal generation unit 19. An OSD signal is superimposed and output. In this case, if the video signal supplied from the signal processing unit 16 is a video signal for normal planar view display, the synthesis processing unit 17 uses the OSD signal supplied from the OSD signal generation unit 19 as it is. It is superimposed and output.

  In addition, when the video signal supplied from the signal processing unit 16 is a video signal for stereoscopic display, the synthesis processing unit 17 receives the input stereoscopic video from the OSD signal supplied from the OSD signal generation unit 19. After performing the stereoscopic display signal processing corresponding to the display video signal, the OSD signal is superimposed on the input video signal and output.

  The digital television 11 supplies the digital video signal output from the synthesis processing unit 17 to the video processing unit 20. The video processing unit 20 converts the input digital video signal into an analog video signal in a format that can be displayed on a flat-type video display unit 21 having, for example, a liquid crystal display panel. The digital television 11 supplies the analog video signal output from the video processing unit 20 to the video display unit 21 for video display.

  The audio processing unit 18 converts the input digital audio signal into an analog audio signal in a format that can be reproduced by the speaker 22 at the subsequent stage. The analog audio signal output from the audio processing unit 18 is supplied to the speaker 22 for audio reproduction.

  Here, the digital television 11 comprehensively controls all the operations including the above-described various reception operations by the control unit 23. The control unit 23 incorporates a CPU (central processing nit) 23a, receives operation information from the operation unit 24 installed in the main body of the digital television 11, or is sent from the remote controller 25 and received by the reception unit. In response to the operation information received at 26, each unit is controlled so that the operation content is reflected.

  The control unit 23 uses the memory unit 23b. The memory unit 23b mainly includes a ROM (Read Only Memory) storing a control program executed by the CPU 23a, a RAM (Random Access Memory) for providing a work area to the CPU 23a, various setting information, control information, and the like. And a non-volatile memory to be stored. Further, a disk drive unit 27 is connected to the control unit 23. The disk drive unit 27 is a unit that allows an optical disk 28 such as a DVD (Digital Versatile Disk) to be detachable, and has a function of recording / reproducing digital data on the mounted optical disk 28.

  The control unit 23 encrypts the digital video signal and audio signal obtained from the demodulation / decoding unit 15 by the recording / playback processing unit 29 based on the operation of the operation unit 24 or the remote controller 25 by the viewer, and puts it into a predetermined recording format. After the conversion, it can be controlled to be supplied to the disk drive unit 27 and recorded on the optical disk 28.

  The control unit 23 causes the disc drive unit 27 to read out digital video signals and audio signals from the optical disc 28 based on the operation of the operation unit 24 and the remote controller 25 by the viewer, and the recording / playback processing unit 29 After decoding, the signal is supplied to the signal processing unit 16 so that it can be controlled to be used for the video display and the audio reproduction described above.

  An HDD (Hard Disk Drive) 30 is connected to the control unit 23. The control unit 23 encrypts the digital video signal and audio signal obtained from the demodulation / decoding unit 15 by the recording / playback processing unit 29 based on the operation of the operation unit 24 or the remote controller 25 by the viewer, and puts it into a predetermined recording format. After the conversion, it can be controlled to be supplied to the HDD 30 and recorded on the hard disk 30a.

  Further, the control unit 23 causes the HDD 30 to read out digital video signals and audio signals from the hard disk 30 a based on the operation of the operation unit 24 and the remote controller 25 by the viewer, and decodes them by the recording / playback processing unit 29. By supplying the signal to the signal processing unit 16, it can be controlled to be used for the video display and the audio reproduction described above.

  Furthermore, an input terminal 31 is connected to the digital television 11. The input terminal 31 is for directly inputting digital video signals and audio signals from the outside of the digital television 11. The digital video signal and audio signal input through the input terminal 31 are supplied to the signal processing unit 16 through the recording / playback processing unit 29 based on the control of the control unit 23, and thereafter the above-described processing. For video display and audio playback.

  Also, the digital video signal and digital audio signal input through the input terminal 31 are sent to the optical disk 28 by the disk drive unit 27 after passing through the recording / playback processing unit 29 based on the control of the control unit 23. It is used for recording / reproduction and recording / reproduction for the hard disk 30 a by the HDD 30.

  The control unit 23 receives the digital video signal and the digital audio signal recorded on the optical disk 28 between the disk drive unit 27 and the HDD 30 based on the operation of the operation unit 24 and the remote controller 25 by the viewer. It also controls recording on the optical disk 28 and recording of the digital video signal and digital audio signal recorded on the hard disk 30a.

  A network interface 32 is connected to the control unit 23. The network interface 32 is connected to an external network 34 via an input / output terminal 33. The network 34 is connected to a plurality (two in the illustrated case) of network servers 35 and 36 for providing various services using the communication function via the network 34. Therefore, the control unit 23 accesses the desired network servers 35 and 36 via the network interface 32, the input / output terminal 33 and the network 34 to perform information communication, thereby using the service provided there. Be able to.

FIG. 2 is a functional block diagram of the signal processing unit in the first embodiment.
In the first embodiment, the signal processing unit 16 functions as a first depth information generation unit 16A, a second depth information generation unit 16B, a depth information synthesis unit 16C, and a multi-parallax image generation unit 16D.
The first depth information generation unit 16A generates depth information based on audio information for generating a multi-parallax image based on an audio level corresponding to the digital audio signal input from the demodulation / decoding unit 15, and performs depth information synthesis. To the unit 16C.
For example, the depth information based on the audio information is generated based on the magnitude relationship between the audio level corresponding to the digital audio signal and a predetermined threshold value (one or more). More specifically, when an audio level that exceeds a threshold is detected, depth information is generated in which the pop-out amount is larger than when the audio level is not exceeded. On the other hand, when the audio level is lowered beyond the threshold, depth information is generated with a smaller pop-out amount than before the audio level is exceeded.

Further, the second depth information generation unit 16B generates depth information based on the image information in association with the composition, motion, color, etc. of the video corresponding to the digital video signal input from the demodulation / decoding unit 15 to obtain the depth information. The data is output to the combining unit 16C. Specifically, the depth information that generates a greater amount of popping out the one that is placed in front, or the amount of popping out increases for items that move from the back to the front or those that move faster. Depth information is generated, or depth information in which the amount of popping out becomes larger as a brighter one is generated.
As a result, the depth information synthesis unit 16C generates final depth information used for multi-parallax image generation by adding depth information based on audio information to depth information based on image information, and generates multi-parallax image generation To the unit 16D.

The multi-parallax image generation unit 16D generates a multi-parallax image based on the depth information input from the depth information synthesis unit 16C and the digital video signal input from the demodulation decoding unit 15, and outputs the multi-parallax image to the synthesis processing unit 17. Become.
As a result, the stereoscopic video displayed on the video display unit 21 via the synthesis processing unit 17 and the video processing unit 20 has a sense of presence having a protruding amount (or a retracting amount) in consideration of the depth corresponding to the audio information. The stereoscopic video is even more overflowing.

The above description is applicable regardless of whether the digital video signal input from the demodulation / decoding unit 15 is a stereoscopic video signal or a planar video signal.
As described above, according to the first embodiment, it is possible to obtain a stereoscopic video image that further enhances the sense of reality in consideration of not only a digital video signal but also a digital audio signal.

[Second Embodiment]
Next, a second embodiment will be described.
Also in the second embodiment, the signal processing system of the digital television broadcast receiving apparatus is the same as that of the first embodiment, and the detailed description thereof is incorporated.

FIG. 3 is a functional block diagram of a signal processing unit in the second embodiment.
In the second embodiment, the signal processing unit 16 functions as a first depth information generation unit 16A, a second depth information generation unit 16B, a depth information synthesis unit 16C, a multi-parallax image generation unit 16D, and a caption area detection unit 16E. ing.
Here, the description of the first embodiment is used for the first depth information generation unit 16A and the second depth information generation unit 16B.
In the second embodiment, the signal processing unit 16 detects a subtitle region as a specific image region in which a subtitle is displayed based on the OSD signal output from the OSD signal generation unit 19 and combines the subtitle region detection information with the depth information synthesis. It also functions as a caption area detection unit 16E that outputs to the unit 16C.

  As a result, the depth information synthesis unit 16C allocates depth information to the corresponding subtitle area based on the audio information based on the subtitle area detection information output from the subtitle area detection unit 16E, and based on the image information. The final depth information used for generating the multi-parallax image is generated by adding the depth information based on the audio information to the depth information, and is output to the multi-parallax image generation unit 16D.

The multi-parallax image generation unit 16D generates a multi-parallax image based on the depth information input from the depth information synthesis unit 16C and the digital video signal input from the demodulation decoding unit 15, and outputs the multi-parallax image to the synthesis processing unit 17. Become.
As a result, the stereoscopic video displayed on the video display unit 21 via the synthesis processing unit 17 and the video processing unit 20 has a protruding amount (or a retracting amount) in consideration of the depth corresponding to the audio information. In both cases, when the audio level is high in the area where the subtitles are displayed, an effect such as increasing the pop-out amount can be given, resulting in a stereoscopic video with a greater sense of realism. In particular, pictures that were hard of hearing can also be visually perceived as a change in audio level, and can provide a video experience with a higher sense of reality.

The above description can also be applied to the second embodiment regardless of whether the digital video signal input from the demodulation / decoding unit 15 is a stereoscopic video signal or a planar video signal.
As described above, according to the second embodiment, it is possible to obtain a stereoscopic video image that further enhances the sense of reality considering not only the digital video signal but also the digital audio signal. In particular, the subtitles can be made into a video full of realism that expresses a dynamic feeling.

[Third Embodiment]
Next, a third embodiment will be described.
Also in the third embodiment, the signal processing system of the digital television broadcast receiving apparatus is the same as that of the first embodiment, and the detailed description thereof is incorporated.

FIG. 4 is a functional block diagram of a signal processing unit in the third embodiment.
In the third embodiment, the signal processing unit 16 includes a first depth information generation unit 16A, a second depth information generation unit 16B, a depth information synthesis unit 16C, a multi-parallax image generation unit 16D, and an object (object) detection unit 16F. It is functioning.
Here, the description of the first embodiment is used for the first depth information generation unit 16A and the second depth information generation unit 16B.
In the third embodiment, the signal processing unit 16 also functions as an object detection unit 16F that detects a region as a specific image region in which an object that is an image component of an image corresponding to a digital video signal is displayed.

  As a result, the depth information synthesis unit 16C performs depth information on the basis of the audio information for the region in which the corresponding (object) is displayed based on the object (object) detection information output from the object detection unit 16F. And the depth information based on the audio information is added to the depth information based on the image information to generate final depth information to be used for generating the multi-parallax image and output to the multi-parallax image generation unit 16D.

The multi-parallax image generation unit 16D generates a multi-parallax image based on the depth information input from the depth information synthesis unit 16C and the digital video signal input from the demodulation decoding unit 15, and outputs the multi-parallax image to the synthesis processing unit 17. Become.
As a result, the stereoscopic video displayed on the video display unit 21 via the synthesis processing unit 17 and the video processing unit 20 has a protruding amount (or a retracting amount) in consideration of the depth corresponding to the audio information. In both cases, when the sound level is high for the area where the object (object) is displayed, an effect such as increasing the pop-out amount can be given, and the stereoscopic video image is more realistic. It becomes.

FIG. 5 is a conceptual explanatory diagram of assignment of depth information based on audio information for an object (object) in an image.
FIG. 5A is an explanatory diagram illustrating an example of a display image of the frames FR1 to FR5 of the planar video signal input from the demodulation / decoding unit 15 to the signal processing unit 16, and an object that is a sphere in the screen is viewed from the left. When moving to the right.

FIG. 5B illustrates a case where depth information is assigned based on the motion of an object in the video signal when a planar video signal corresponding to the image shown in FIG. 5A is input. In the figure, the upper number in the frame is depth information assigned to the object that is a sphere, and the lower number in the frame is depth information assigned to the background. Here, the assigned depth information has 256 levels of 0 to 255, and the smaller the numerical value, the larger the pop-out amount (depth amount).
That is, the image is such that a spherical object protrudes from the background.

  FIG. 5C is a diagram showing temporal changes in the level of the audio signal, and the audio signal level is equal to or lower than a predetermined threshold value TH during the period of the frames FR1 to FR4.

Therefore, in the period of the frames FR1 to FR4, the depth information assigned to the spherical object does not include the audio signal level.
On the other hand, in the period of the frame FR5, the level of the audio signal exceeds the predetermined threshold value TH. That is, since the sound is louder, as shown in FIG. 5 (d), in the frame FR5, the depth information assigned to the spherical object is set to 96, which corresponds to the case where the pop-out amount is larger, and is further on the near side. It will be a video that feels popping out.
As a result, compared to the video shown in FIG. 5B, the video shown in FIG. 5D becomes a stereoscopic video with a greater sense of realism.

  The above description can be applied to the third embodiment even if the digital video signal input from the demodulation / decoding unit 15 is either a stereoscopic video signal or a planar video signal.

  As described above, according to the third embodiment, it is possible to obtain a stereoscopic video image that further enhances the sense of reality considering not only the digital video signal but also the digital audio signal. In particular, it is possible to make a video full of realism that expresses a dynamic feeling for each object in the screen.

[Effect of the embodiment]
As described above, according to each embodiment, stereoscopic information (so-called 3D) video with a more realistic sensation is obtained by using depth information generated based on audio information in addition to depth information generated from video information. Can be generated.

  In addition, by separately applying the depth information generated based on the audio information to the subtitles or the objects constituting the image, it is possible to generate a stereoscopic video with a more dynamic feeling. In particular, when applied to subtitles, it becomes possible for a hearing-impaired person to feel a change in sound level.

[Modification of Embodiment]
In the above description, the normal voice (monaural or two-channel stereo or the like) is used. However, the same method can be applied to surround sound such as 2.1 channel or 5.1 channel.
More specifically, for example, the audio levels of the front L channel, the front R channel, and the center channel are detected, respectively, and when the audio signal level of the front L channel exceeds a predetermined threshold value TH, it exists on the left side of the screen. For an object (object) having a large pop-out amount, the pop-out can be further increased with respect to depth information generated by motion information or the like.

  Furthermore, if the sound (sound image localization) above a certain level moves from the front channel side to the rear channel side, the pop-out amount increases, and if it moves from the rear channel to the front channel, the pop-out amount decreases. Etc. are also conceivable.

  In the above description, the case where the video signal generation device is a digital television broadcast reception device has been described. However, as the video signal generation device of the present embodiment, a control device such as a CPU, a ROM (Read Only Memory), It is equipped with a storage device such as a RAM, an external storage device such as an HDD and a CD drive device, a display device such as a display device, and an input device such as a keyboard and a mouse, and has a hardware configuration using a normal computer. It is also possible.

The control program executed by the video signal generation device of the present embodiment is a file in an installable format or an executable format, such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), or the like. The program is provided by being recorded on a computer-readable recording medium.
In addition, the control program executed by the video signal generation apparatus of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the control program executed by the video signal generation apparatus of the present embodiment may be provided or distributed via a network such as the Internet.

In addition, the control program executed by the video signal generation apparatus of the present embodiment may be provided by being incorporated in advance in a ROM or the like.
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 11 Digital television broadcast receiver 15 Demodulation decoding part 16 Signal processing part 16A 1st depth information generation part 16B 2nd depth information generation part 16C Depth information synthetic | combination part 16D Multi parallax image generation part 16E Subtitle area | region detection part 16F Object detection part 17 Synthesis processing unit 18 Audio processing unit 19 OSD signal generation unit 20 Video processing unit 21 Video display unit 22 Speaker 23 Control unit TH threshold

Claims (9)

A video signal generation device that generates a video signal for stereoscopic video based on depth information,
A depth information changing unit that changes the depth information based on audio information;
A video signal generation unit that generates a video signal based on the depth information changed by the depth information change unit;
A video signal generation apparatus comprising: The video signal generation unit extracts a specific image area from an image corresponding to the input video signal, assigns the changed depth information to an image of the specific image area, and outputs the video signal for the stereoscopic video Generate
The video signal generation device according to claim 1. The specific image area is a caption area;
The video signal generation unit allocates the changed depth information to the image of the subtitle area, and generates a video signal for the stereoscopic video.
The video signal generation device according to claim 2. The specific image area is an area in which an object that is an image component is displayed;
The video signal generation unit generates the video signal for the stereoscopic video by allocating the changed depth information to an image corresponding to the object;
The video signal generation device according to claim 2. The audio signal is a surround audio signal;
The depth information changing unit changes the depth information in accordance with a change in sound image localization of surround sound corresponding to the surround sound signal.
The video signal generation device according to any one of claims 1 to 3. The depth information changing unit changes the depth information in accordance with a change in sound image localization of surround sound of a predetermined level or more among the surround sound,
The video signal generation device according to any one of claims 1 to 3.   The video signal generation device according to claim 1, further comprising a display unit that displays video based on the generated video signal. A video signal generation method executed by a video signal generation device that generates a stereoscopic video based on depth information,
A depth information changing process for changing the depth information based on audio information;
A video signal generation process for generating a video signal based on the modified depth information;
A video signal generation method comprising: A control program for controlling a video signal generation device that generates a stereoscopic video based on depth information by a computer,
The computer,
Depth information changing means for changing the depth information based on audio information;
Video signal generating means for generating a video signal based on the changed depth information;
Control program to function.

Images