JP2013055629A - Video output device and video output method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video output device and a video output method capable of suitably outputting a stereoscopic video.SOLUTION: The video output device comprises: input means, conversion means, and output means. The input means inputs a video having information for generating a depth of a stereoscopic video. The conversion means converts a depth so as to expand a depth gradation concerning the video input by the input means. The output means outputs the video whose depth is changed by the conversion means.

Description

  Embodiments described herein relate generally to a video output apparatus and a video output method.

There is a proposal related to an imaging apparatus including a plurality of optical systems having parallax. For example, a parallax distribution (histogram) is obtained from first and second images by feature point extraction and corresponding point detection, and moving object detection is performed based on the parallax distribution (histogram). Another object is to enable the stereoscopic imaging apparatus to electronically adjust the convergence angle according to the subject in focus (see, for example, Patent Document 1). As an outline, when the total number of pairs of feature points and corresponding points detected from the first image data and the second image data in the in-focus state exceeds a predetermined threshold th1, a parallax amount histogram is generated. The representative parallax amount is controlled to be determined from the bin of the histogram of the parallax amount.

However, on the other hand, there is a demand for deviating parallax exclusion processing as a problem, but means for realizing such demand is not known.

JP 2011-29905 A (FIG. 5)

An object of the embodiments of the present invention is to provide a video output device and a video output method capable of suitably outputting a stereoscopic video.

In order to solve the above-described problem, the video output apparatus according to the embodiment includes an input unit, a conversion unit, and an output unit. The input means inputs a video having information for generating the depth of the stereoscopic video. The conversion means converts the depth of the video input by the input means so as to extend the depth gradation. The output means outputs the video whose depth has been changed by the converting means.

The block block diagram which shows the signal processing system of the digital television broadcast receiver which shows one Embodiment of this invention. The figure which shows an example of the parallax / depth conversion graph in connection with embodiment. The figure shown in order to demonstrate an example of the conventional parallax / depth conversion in connection with embodiment. The figure which shows an example of the novel parallax / depth conversion in connection with embodiment. The figure which shows the processing image of the novel stereo 3D in connection with embodiment. The flowchart regarding the stereo 3D process in connection with embodiment. The figure which shows the image of the deviating parallax exclusion process in connection with embodiment. The figure which shows the processing image of the conventional stereo 3D in connection with embodiment.

  Hereinafter, an embodiment of the present invention will be described.

(First embodiment)
A first embodiment will be described with reference to FIGS.

FIG. 1 is a block diagram illustrating a signal processing system of a digital television broadcast receiver according to an embodiment.

A digital television broadcast receiving apparatus (hereinafter referred to as digital television) 11 shown in FIG. 1 not only performs video display based on a video signal for normal planar view (two-dimensional) display but also stereoscopic view (three-dimensional). ) Video display based on the video signal for display can also be performed.

As shown in FIG. 1, a digital television 11 selects a broadcast signal of a desired channel by supplying a digital television broadcast signal received by an antenna 12 to a tuner unit 14 via an input terminal 13. It is possible. The digital television 11 functions as a video output device.

The digital television 11 supplies the broadcast signal selected by the tuner unit 14 to the demodulation / decoding unit 15 and restores it to a digital video signal, a digital audio signal, etc., and then outputs it to the signal processing unit 16. The signal processing unit 16 performs predetermined digital signal processing on the digital video signal and audio signal supplied from the demodulation / decoding unit 15.

Here, the predetermined digital signal processing performed by the signal processing unit 16 includes processing for converting a normal planar (two-dimensional) display video signal into a stereoscopic (three-dimensional) display video signal and audio signal, At this time, a video signal for stereoscopic display is generated based on a digital video signal and a digital audio signal input from the demodulation / decoding unit 15 or an OSD signal such as caption input from the OSD signal generation unit 19 described later. Processing for generating depth information, processing for converting a video signal for stereoscopic display into a video signal for planar display, and the like are also included.

Further, the signal processing unit 16 outputs a digital video signal to the synthesis processing unit 17 and outputs a digital audio signal to the audio processing unit 18.

The composition processing unit 17 applies the OSD (on
screen display) Subtitles generated by the signal generator 19 and GUI (Graphical User Interface)
e) An OSD signal that is a video signal for superimposition such as OSD is superimposed and output. In this case, if the video signal supplied from the signal processing unit 16 is a video signal for normal planar view display, the synthesis processing unit 17 uses the OSD signal supplied from the OSD signal generation unit 19 as it is. It is superimposed and output.

In addition, when the video signal supplied from the signal processing unit 16 is a video signal for stereoscopic display, the synthesis processing unit 17 receives the input stereoscopic video from the OSD signal supplied from the OSD signal generation unit 19. After performing the stereoscopic display signal processing corresponding to the display video signal, the OSD signal is superimposed on the input video signal and output.

The digital television 11 supplies the digital video signal output from the synthesis processing unit 17 to the video processing unit 20. The video processing unit 20 converts the input digital video signal into an analog video signal in a format that can be displayed on a flat-type video display unit 21 having, for example, a liquid crystal display panel. The digital television 11 includes a video processing unit 20
The analog video signal output from is supplied to the video display unit 21 for video display (video is output).

The audio processing unit 18 converts the input digital audio signal into an analog audio signal in a format that can be reproduced by the speaker 22 at the subsequent stage. The analog audio signal output from the audio processing unit 18 is supplied to the speaker 22 for audio reproduction.

Here, the digital television 11 comprehensively controls all the operations including the above-described various reception operations by the control unit 23. The control unit 23 is a CPU (central pr
ocessing nit) 23a is built in and receives operation information from the operation unit 24 installed in the main body of the digital television 11 or receives operation information transmitted from the remote controller 25 and received by the reception unit 26. Each unit is controlled so that the operation content is reflected.

The control unit 23 uses the memory unit 23b. The memory unit 23b is mainly a CPU.
A ROM (Read Only Memory) storing a control program executed by the CPU 23a;
a has a random access memory (RAM) for providing a work area, and a non-volatile memory in which various setting information, control information, and the like are stored. Further, a disk drive unit 27 is connected to the control unit 23. The disk drive unit 27 is, for example, a DVD (Digi
tal Versatile Disk) or the like, which has a function of recording and reproducing digital data with respect to the optical disk 28.

The control unit 23 records the digital video signal and the audio signal obtained from the demodulation / decoding unit 15 based on the operation of the operation unit 24 and the remote controller 25 by the viewer.
After being encrypted by 9 and converted into a predetermined recording format, it can be supplied to the disk drive unit 27 and recorded on the optical disk 28.

The control unit 23 causes the disc drive unit 27 to read out digital video signals and audio signals from the optical disc 28 based on the operation of the operation unit 24 and the remote controller 25 by the viewer, and the recording / playback processing unit 29 After decoding, the signal is supplied to the signal processing unit 16 so that it can be controlled to be used for the video display and the audio reproduction described above.

An HDD (Hard Disk Drive) 30 is connected to the control unit 23. The control unit 23
Based on the operation of the operation unit 24 and the remote controller 25 by the viewer, the demodulation / decoding unit 15
The digital video signal and audio signal obtained from the above can be encrypted and converted into a predetermined recording format by the recording / playback processing unit 29, and then supplied to the HDD 30 to be recorded on the hard disk 30a.

Further, the control unit 23 causes the HDD 30 to read out digital video signals and audio signals from the hard disk 30 a based on the operation of the operation unit 24 and the remote controller 25 by the viewer, and decodes them by the recording / playback processing unit 29. By supplying the signal to the signal processing unit 16, it can be controlled to be used for the video display and the audio reproduction described above.

Furthermore, an input terminal 31 is connected to the digital television 11. The input terminal 31 is for directly inputting digital video signals and audio signals from the outside of the digital television 11. The digital video signal and audio signal input through the input terminal 31 are supplied to the signal processing unit 16 through the recording / playback processing unit 29 based on the control of the control unit 23, and thereafter the above-described processing. For video display and audio playback.

Further, the digital video signal and the digital audio signal input through the input terminal 31 are passed through the recording / playback processing unit 29 based on the control of the control unit 23 and then the disk drive unit 2.
7 and recording / reproduction with respect to the optical disk 28, and a hard disk 30a by the HDD
It is used for recording and playback.

The control unit 23 receives the digital video signal and the digital audio signal recorded on the optical disk 28 between the disk drive unit 27 and the HDD 30 based on the operation of the operation unit 24 and the remote controller 25 by the viewer. It also controls recording on the optical disk 28 and recording of the digital video signal and digital audio signal recorded on the hard disk 30a.

A network interface 32 is connected to the control unit 23. The network interface 32 is connected to an external network 34 via an input / output terminal 33. The network 34 is connected to a plurality (two in the illustrated case) of network servers 35 and 36 for providing various services using the communication function via the network 34. Therefore, the control unit 23 accesses the desired network servers 35 and 36 via the network interface 32, the input / output terminal 33 and the network 34 to perform information communication, thereby using the service provided there. Be able to.

Now, FIG. 5A is a diagram showing a stereo 3D processing image when the parallax amount conversion of video is not performed. Stereo 3D refers to an algorithm for detecting disparity information from left and right images prepared for stereoscopic viewing and acquiring the depth information. The generated depth information can be used to generate a multi-parallax image based on the depth information.

In the present embodiment, the signal processing unit 16 includes a histogram acquisition unit (parallax amount detection unit) and a map processing unit (parallax conversion unit / depth conversion unit) (not shown).

The histogram acquisition unit (parallax amount detection unit) has a function of detecting (acquiring) parallax information from the left-eye image and the right-eye image, and creating (acquiring) a parallax histogram based on the information. .

The map processing unit (disparity conversion unit / depth conversion unit) has a function of converting the depth of the object based on the histogram created by the histogram acquisition unit and converting the parallax of the video.

The map processing unit includes an outlier parallax detection unit (not shown) that detects (acquires) outlier parallax. The deviating parallax detection unit has a function of detecting deviating parallax based on the histogram.
Further, the map processing unit has an excluding unit (not shown) having a function of excluding the detected out-of-parallax.
have. This out parallax and the exclusion of out parallax will be described later.

The signal processing unit 16 includes a multi-parallax image creation unit (not shown). The multi-parallax image creation unit has a function of creating left and right parallax images based on the parallax information of the video output from the map processing unit.

Note that the above-described histogram acquisition unit (parallax amount detection unit), map processing unit (parallax conversion unit / depth conversion unit), out-of-parallax detection unit, exclusion unit, and multi-parallax image creation unit may be configured with HW. You may be comprised by SW.

  The stereo 3D process is performed in the following flow.

First, the histogram acquisition unit (parallax detection unit) detects parallax information from the left and right images and records it as a parallax map (histogram). As an example of the parallax information detection method, there is a method using block matching based on the luminance information of the left and right images. The histogram acquisition unit (parallax detection unit) compares the left and right images based on block matching, and detects the parallax (of each object) in the video.

Next, the parallax map generated by the map processing unit is converted into a depth map. There are various formulas for converting the parallax into the depth. As shown in FIG. 2, for example, conversion in which the relationship between the parallax and the depth is linear is given. Here, -64 to 63 are associated with the parallax value, and -128 to 127 are associated with the depth value.

Finally, the multi-parallax image is generated based on the depth map generated by the multi-parallax image creating unit that creates the multi-parallax image, and the stereoscopic image is displayed on the panel.

FIG. 3 is a diagram for explaining an example of general parallax / depth conversion according to the embodiment. An example at the time of producing | generating a depth map by the above flows is shown.

When handling parallax map information as a histogram,
As shown in FIG. 3, the parallax information may concentrate locally. At this time, when the parallax is linearly converted to the depth, the generated depth is also locally concentrated similarly to the parallax, and as a result, the depth range R is narrow and the depth gradation is lowered.

The gradation of depth is related to the sense of popping out and the sense of depth when a stereoscopic image is actually viewed on the panel, and the higher this is, the more stereoscopic the image can be experienced. That is, when it is desired to feel a three-dimensional effect, it can be said that it is desirable that the depth gradation is high.

FIG. 4 is a diagram illustrating an example of parallax / depth conversion in the processing of the embodiment. This is a diagram illustrating a method of extending the depth gradation when disparity information is locally concentrated as shown in FIG. The procedure is shown below.

First, the histogram acquisition unit (parallax detection unit) generates a parallax histogram from the parallax information, searches for the maximum and minimum bins containing values, and handles them as “max_dv” and “min_dv”.

Next, the following conversion formula is applied to the parallax “dv”. “Bin_num” indicates the number of bins in the parallax histogram, and this value is arbitrary.

The above conversion equation is a conversion equation that can increase the difference between the maximum parallax amount and the minimum parallax amount in the image (increase the difference between the upper limit and the lower limit of the existing parallax amount).

By applying Equation 1 to the parallax, for example, when dv is max_dv and min_dv, the values are converted into the upper and lower limits of the parallax (63 and -64 in the case of FIG. 4).

In addition, other parallaxes are also converted, and as shown in FIG. 4, while maintaining the shape of the histogram,
Overall, the range of parallax is extended. In this way, by performing depth conversion after parallax expansion, it is possible to improve the gradation of depth as the depth range R ′. In other words, in a conventional system, an image with poor stereoscopic effect can be experienced (viewed) as an image with a stereoscopic effect by applying this system.

In the above, the maximum parallax amount and the minimum parallax amount are converted to be the upper limit and the lower limit of the parallax, respectively, but the first parallax amount and the second parallax amount smaller than the first parallax amount. If it converts so that the difference with may become large, it is not limited to this.

FIG. 5 is a diagram illustrating a stereo 3D processing image according to the embodiment. In particular, FIG.
) Shows a stereo 3D processing image when depth conversion (parallax amount conversion) is performed, and FIG. 6 is a flowchart of the processing in the signal processing unit 16 or the control unit 23. As described above, the difference from the general system is that the parallax is histogrammed and the parallax is processed based on the information, and as shown in FIG. ) An image with a greater stereoscopic effect can be output.

  Step S61: First, the histogram acquisition unit (parallax amount detection unit) performs parallax detection.

Step S62: Next, the histogram acquisition unit (parallax amount detection unit) performs parallax histogram acquisition.

Step S63: Next, the map processing unit (parallax conversion unit / depth conversion unit) performs step S6.
Various parameters are calculated based on the histogram obtained in step 2. For example, the MAX value and MIN value of Equation 1 and the parallax histogram are obtained.

Step S64: Further, the map processing unit (parallax conversion unit / depth conversion unit) performs parallax map processing.

Step S65: Finally, the map processing unit (parallax conversion unit / depth conversion unit) performs parallax / depth conversion.

  Details of these processes are also described above.

Based on the input video, a histogram acquisition unit (parallax amount detection unit) detects the parallax amount of each object in the video and creates (acquires) a histogram regarding the parallax amount. The map processing unit (parallax conversion unit / depth conversion unit) performs depth conversion and parallax amount conversion so that the depth gradation in the video is increased based on the obtained histogram. When converting the amount of parallax,
The multi-parallax image creation unit creates a video based on the converted parallax information, and the output unit outputs the video.

By the method of the above embodiment, the stereoscopic effect when performing stereo 3D processing can be generally increased.

  In the following, further improvements will be described.

First, FIG. 8 shows the stereo 3D processing described above. Here, FIG. 8 shows an example of the case where parallax conversion is performed using the maximum and minimum bins containing values after generating a parallax histogram. If the value in the bin is 1 or more, regardless of the value, “min_dv” and “max_dv” are determined, and therefore, using only a small number of parallax information as shown in FIG. There may be cases where conversion takes place. As a result, according to the above processing procedure, the minimum and maximum amount of parallax popping out (displaced) is affected as in the conversion from FIG. 8A to FIG. Will not be fully expanded. In the first place, there is a high possibility that parallax having only a small number is erroneously detected parallax (hereinafter referred to as out-of-parallax), and it is not desirable to correct the total parallax based on such parallax information.

Therefore, as shown in FIG. 7, a method of determining “min_dv” and “max_dv” after excluding out-of-parallax is employed. For example, simply skip the values in the out-of-bins that represent 1 percent of the entire histogram.

By excluding out-of-parallax when converting the amount of parallax / depth, the gradation of parallax is enhanced as shown in FIG. 7, and the user can enjoy a more stereoscopic image.

Here, after the histogram acquisition unit (parallax amount detection unit) acquires the histogram, the above-described disparity detection unit detects the disparity, and the map processing unit (parallax conversion unit / depth conversion unit) detects the parallax amount / depth conversion. This outlier parallax is excluded before. The map processing unit (parallax conversion unit / depth conversion unit) performs the parallax amount / depth conversion based on the histogram from which the outlier parallax is excluded.

Note that the method for obtaining out parallax is a method of excluding an arbitrary number of data from both ends of the histogram based on an empirical rule that the vector value is extremely large is out of parallax, and stereo images are mainly in the x direction (when viewing). Since there is parallax in the left and right direction and almost no parallax in the y direction (up and down direction during viewing), there is a method of determining that when there is a certain parallax or more in the y direction, it is determined as parallax. By the method of the present embodiment, it is possible to enhance the stereoscopic effect when stereo 3D processing is performed.

(Second Embodiment)
A second embodiment of the present invention will be described with reference to FIGS. Description of the parts common to the first embodiment is omitted.

The formula (1) shown in the first embodiment may be modified such that each parameter is weighted as in the following formula (2).

For example, α is weighted from 0 to 1. In addition, β can deal with the bias of the histogram or can be set to express the user's preference.

The method for extending the width of the depth in the algorithm for detecting the parallax information from the left and right images for stereoscopic viewing and estimating the depth information has been described above.

As an overview, this is a method of performing more accurate processing by removing erroneous information when acquiring a parameter from a parallax histogram, and more accurately excluding erroneous information when acquiring a parameter from a parallax histogram. This is a method for performing a correct process.

In addition, this invention is not limited to the said embodiment, In the range which does not deviate from the summary, it can implement in various modifications.

Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements according to different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 11 Digital television broadcast receiver 15 Demodulation decoding part 16 Signal processing part 17 Synthesis | combination processing part 18 Audio | voice processing part 19 OSD signal generation part 20 Video processing part 21 Video display part 22 Speaker 23 Control part

Claims (7)

Input means for inputting a video having information for generating a depth of a stereoscopic video;
Conversion means for converting the depth so as to extend the depth gradation for the video input by the input means;
Output means for outputting video whose depth has been changed by the converting means;
A video output device comprising: Further comprising creating means for creating a parallax image based on the depth converted by the converting means;
The video output apparatus according to claim 1, wherein the output unit outputs the parallax video created by the creation unit. It further comprises detection means for detecting parallax from the video input by the input means,
The video output apparatus according to claim 1, wherein the conversion unit performs the depth conversion based on the detected parallax. A histogram acquisition means for acquiring a histogram of parallax detected by the detection means;
The video output apparatus according to claim 3, wherein the conversion unit performs the depth conversion based on a parallax histogram acquired by the histogram acquisition unit. Out-of-focus parallax detection means for detecting out-of-focus parallax in the input video;
And further comprising an excluding means for excluding outlier parallax detected by the outlier parallax,
The video output apparatus according to claim 1, wherein the conversion unit performs the depth conversion on the parallax from which the out-of-run parallax is excluded by the exclusion unit. The video output apparatus according to claim 2, further comprising a display unit that displays the parallax video output by the output unit. Input video with information to generate depth of stereoscopic video,
Convert the depth of the input video to expand the depth gradation,
An image output method for outputting an image in which the depth is changed.

Images