JP2013031056A - Stereoscopic image control apparatus, stereoscopic image control method, and program for stereoscopic image control - Google Patents

Abstract

PROBLEM TO BE SOLVED: To form a stereoscopic image on an observer within a predetermined parallax range without regard to the parallax of the stereoscopic image, a screen size, and an observation distance, when the stereoscopic image is reproduced.SOLUTION: An observation environment registration part 220 of a stereoscopic image control apparatus 120 registers observation environment information containing the screen size and the observation distance that is a distance between an observer and a screen. An image data acquisition part 210 acquires two pieces of image data having a binocular parallax and parallax information on the parallax. A parallax correction portion 222 determines, on the basis of the observation environment information registered by the observation environment registration part and the parallax information acquired by the image data acquisition part, whether the parallax of image data remains within a predetermined parallax range, and if determining that the parallax of image data does not remain within the predetermined parallax range, reduces the image based on the two pieces of image data at least horizontally so that the parallax of image data may remain within the predetermined parallax range.

Description

  The present invention relates to a stereoscopic video control apparatus, a stereoscopic video control method, and a stereoscopic video control program for controlling video data for perceiving a stereoscopic video.

  In recent years, stereoscopic video technology has been attracting attention as it presents two images with binocular parallax (horizontal parallax) on a display and makes the viewer perceive that the subject image exists stereoscopically. . The two images used in such a technique are images captured at different optical axes (viewpoints) or CG (Computer Graphics) images corresponding to images captured at different optical axes. The imaging position in the perspective direction of the subject image included in the two images is determined by the size of the binocular parallax (hereinafter sometimes simply referred to as “parallax”) of the subject image between the two images.

  If the parallax of the subject image between the two images is adjusted in this way, the imaging position in the perspective direction of the subject image can be changed. However, the parallax is too large in the near direction, and the subject image jumps out excessively. If the image is formed or the parallax is excessively increased in the far direction and the image is formed too deeply, a burden may be imposed on the eyes of the observer. Therefore, when the parallax between two images increases, the width (dynamic range) of the depth amount expressed in the projection direction (near direction) and the depth direction (far direction) of the three-dimensional image is determined according to the parallax. There is known a technique for correcting by shifting the relative position in the horizontal direction (for example, Patent Document 1).

JP 2010-45584 A

  In general, the parallax range between images that allows an observer to comfortably view a stereoscopic image without causing eye strain is said to be 1.7 ° or less in terms of angle of view, and the pupil is the distance between the pupils of both eyes. It is said that it is desirable that the distance be 5 cm or less. Therefore, it is desirable to suppress the parallax between two images within a predetermined (appropriate) parallax range. However, it is difficult to perform imaging while always paying attention so that the parallax of all subjects included in the imaging range is within the predetermined parallax range, and the parallax between the captured images exceeds the predetermined parallax range. There are often things.

  In addition, even if it is set within a predetermined parallax range at the imaging stage, depending on the screen size of the stereoscopic video display device and the observation distance that is the distance between the observer and the stereoscopic video display device at the reproduction stage, There was a risk that parallax would increase and cause eye strain.

  Therefore, in view of such problems, the present invention makes it possible to form an image of a stereoscopic image on a viewer within a predetermined parallax range, regardless of the parallax of the stereoscopic video, the screen size, and the observation distance when reproducing the stereoscopic video. An object of the present invention is to provide a stereoscopic video control device, a stereoscopic video control method, and a stereoscopic video control program.

  In order to solve the above problems, a stereoscopic image control apparatus according to the present invention includes an observation environment registration unit that registers observation environment information including a screen size and an observation distance that is a distance between the observer and the screen, and binocular The parallax of the video data based on the two video data having parallax and the parallax information related to the parallax information, and the parallax information acquired by the video data obtaining unit and the observation environment information registered in the observation environment registration unit At least a video based on two video data so that the parallax of the video data falls within the predetermined parallax range when it is determined that the video data does not fall within the predetermined parallax range. And a parallax correction unit that reduces in a horizontal direction.

  The video data acquisition unit acquires a near maximum value that is the maximum value in the projection direction of the parallax of two video data in a predetermined video unit and a far maximum value that is the maximum value in the direction in which the video data is deepened. The video may be reduced at least in the horizontal direction so that both the maximum value and the far maximum value fall within a predetermined parallax range.

  The video data acquisition unit further derives the largest value among the near maximum values over the entire time of the two video data as the total near maximum value, and the largest among the far maximum values over the entire time of the two video data. The parallax correction unit derives the value as the total far maximum value, and the parallax correction unit displays the video at least in the horizontal direction over the entire time according to a reduction ratio of 1 in which both the total near maximum value and the total far maximum value fall within a predetermined parallax range. It may be reduced.

  When the excess parallax exceeding the predetermined parallax range is less than the first threshold, the parallax correction unit reduces the image only in the horizontal direction, the excess parallax is equal to or greater than the first threshold, and is greater than the first threshold. If it is less, the video may be reduced in the horizontal direction and the vertical direction. If the excess parallax is equal to or greater than the second threshold, the video may be reduced in the horizontal direction and the vertical direction while maintaining a predetermined aspect ratio.

  The parallax correction unit may reduce the video based on the two video data after relatively shifting the video based on the two video data in the horizontal direction.

  In order to solve the above-described problem, the stereoscopic video control method of the present invention registers two pieces of observation environment information including a screen size and an observation distance that is a distance between the observer and the screen and has binocular parallax. The video data and the parallax information related to the parallax are acquired, and based on the registered observation environment information and the acquired parallax information, it is determined whether or not the parallax between the two video data falls within a predetermined parallax range, and the predetermined parallax The video based on the two video data is reduced at least in the horizontal direction so that the parallax of the video data falls within the predetermined parallax range when it is determined that the video data does not fall within the range.

  In order to solve the above-described problem, a stereoscopic image control program according to the present invention includes an observation environment registration unit that registers observation environment information including a screen size and an observation distance that is a distance between an observer and a screen. And two video data having binocular parallax and a video data acquisition unit that acquires parallax information related to the parallax, observation environment information registered in the observation environment registration unit, and parallax information acquired by the video data acquisition unit It is determined whether or not the parallax of the video data falls within the predetermined parallax range, and when it is determined that the parallax of the video data does not fall within the predetermined parallax range, It is characterized by functioning as a parallax correction unit that reduces the image based on at least the horizontal direction.

  According to the present invention, at the time of reproducing a stereoscopic image, it is possible to form an image of the stereoscopic image on the observer within a predetermined parallax range regardless of the parallax of the stereoscopic image, the screen size of the stereoscopic image display device, and the observation distance. Become.

It is the functional block diagram which showed the schematic relationship of the three-dimensional video system. It is the functional block diagram which showed the schematic function of the three-dimensional video imaging device. It is the perspective view which showed the external appearance of the three-dimensional video imaging device. It is explanatory drawing for demonstrating the derivation | leading-out process of the parallax by pattern matching. It is the functional block diagram which showed the schematic function of the three-dimensional video control apparatus. It is explanatory drawing for demonstrating the predetermined parallax range based on observation environment information. It is explanatory drawing for demonstrating the size adjustment process by a parallax correction part. It is a flowchart which shows the flow of a process of a three-dimensional video imaging method. It is a flowchart which shows the flow of a process of a three-dimensional video control method. It is explanatory drawing which showed the display mode of the stereoscopic video display apparatus. It is explanatory drawing which illustrated the parallax distribution in 1 frame.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(3D image system 100)
FIG. 1 is a functional block diagram showing a schematic relationship of the stereoscopic video system 100. The stereoscopic video system 100 generates two video data having binocular parallax (hereinafter, when it is necessary to distinguish between left and right, they are respectively expressed as left-eye video data and right-eye video data). 3D image pickup device 110, a 3D image control device 120 that controls reproduction of the generated image data, and a 3D image display device 130 that displays the display data subjected to the reproduction control. In the stereoscopic video system 100, for example, the photographer 10 generates two video data using the stereoscopic video imaging device 110, and the observer 12 uses the polarized glasses and the video provided through the stereoscopic video display device 130. Observe the data. Here, in order to facilitate understanding, a person who generates video data is an imager 10 and a person who observes the video data is an observer 12, but this embodiment is a user such as an imager or an observer. Needless to say, it is not limited by the form of use such as generation or observation of video data.

  The imager 10 can change the imaging position in the perspective direction of the subject image when the captured image is displayed by adjusting the distance between the stereoscopic video imaging device 110 and the subject and the convergence angle. However, if an attempt is made to capture a stereoscopic video without any reference, the parallax of the subject becomes too large, and the viewer's 12 eyes may be burdened during reproduction. In the present embodiment, even if the parallax is not within a predetermined parallax range in the target video data, the object is to form an image of a stereoscopic video within the predetermined parallax range by the viewer 12. Hereinafter, the stereoscopic video imaging device 110 and the stereoscopic video control device 120 capable of realizing such an object will be described in detail.

(Stereoscopic imaging device 110)
FIG. 2 is a functional block diagram illustrating a schematic function of the stereoscopic video imaging apparatus 110, and FIG. 3 is a perspective view illustrating an appearance of the stereoscopic video imaging apparatus 110. In FIG. 2, the solid line indicates the flow of data, and the broken line indicates the flow of the control signal. As shown in FIGS. 2 and 3, the stereoscopic video imaging apparatus 110 includes two imaging units 150, a data processing unit 152, a synthesis unit 154, a display unit 156, an operation unit 158, and a video holding unit 160. The central control unit 162 is included.

  Two imaging units 150 are provided in the stereoscopic video imaging apparatus 110, each of which includes an imaging lens 150a, a focus lens 150b used for focus adjustment, an aperture (iris) 150c used for exposure adjustment, and an imaging lens 150a. An image sensor 150d that photoelectrically converts the light beam incident through the image sensor into electrical data (video data), and a drive unit 150e that drives the focus lens 150b, the diaphragm 150c, and the image sensor 150d, respectively, in accordance with control signals from an image capture controller 170 described later. The left and right two video data on each of the two optical axes that are arranged in parallel or intersect in the imaging direction are generated. Such video data may be a moving image or a still image.

  The data processing unit 152 performs video signal processing such as R (Red), G (Green), and B (Blue) processing (γ correction and color correction), enhancement processing, and noise reduction processing on the video data received from the imaging unit 150. The processed video data is output to the synthesis unit 154. The synthesizer 154 synthesizes the left and right video data processed by the data processor 152 to generate display data. The display unit 156 includes a liquid crystal display, an organic EL (Electro Luminescence) display, and the like, and displays the display data synthesized by the synthesis unit 154. The operation unit 158 includes an operation key, a cross key, a joystick, a touch panel superimposed on the display surface of the display unit 156, and the like, and receives an operation input from the photographer 10.

  The video holding unit 160 includes an HDD, a flash memory, a nonvolatile RAM, and the like, and holds video data transmitted from the central control unit 162 in accordance with a control signal from the storage control unit 172 described later. The video holding unit 160 may be configured as a device that holds video data in a removable storage medium such as an optical disc medium such as a CD (Compact Disc), DVD, or BD (Blu-ray Disc), or a portable memory card. Good.

  The central control unit 162 manages and controls the entire three-dimensional image pickup device 110 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like. In the present embodiment, the central control unit 162 also functions as the imaging control unit 170, the storage control unit 172, and the parallax derivation unit 174.

  The imaging control unit 170 causes the driving unit 150e to drive the focus lens 150b, the diaphragm 150c, and the imaging element 150d so that appropriate video data can be obtained according to the operation input of the photographer 10. The storage control unit 172 stores the video data transmitted from the data processing unit 152 in the video holding unit 160 while the imaging unit 150 is generating video data.

  First, the parallax deriving unit 174 performs pattern matching on the two pieces of video data transmitted from the data processing unit 152, and is each frame (still image data arranged in time series constituting one video, The parallax is derived for the left and right still image data in the video). Specifically, the parallax deriving unit 174 compares two pieces of video data (left-eye video data and right-eye video data) for imaging the stereoscopic video transmitted from the data processing unit 152, and both video data A parallax vector (hereinafter simply referred to as parallax) indicating the direction and magnitude of parallax of the object (subject) inside is derived. Here, the object in the video is not specified, and the parallax is derived for a plurality of blocks obtained by dividing each of the left-eye video data and the right-eye video data in a grid (specify the object by deriving the spatial frequency etc. Then, the parallax may be derived for each object.) However, since the parallax occurs only in the horizontal direction in the stereoscopic video, the direction of the parallax can be simply indicated by a positive / negative sign (+ −). Here, the direction in which the image is formed on the viewer 12 side (protruding side) from the display position of the stereoscopic image display device 130 is normal, and the image is formed on the side opposite to the observer (the back side) from the display position of the stereoscopic image display device 130. The direction to be made negative. The parallax deriving unit 174 derives the parallax using a pattern matching algorithm.

  FIG. 4 is an explanatory diagram for explaining a parallax derivation process by pattern matching. As shown in FIG. 4, the parallax deriving unit 174 blocks one of the two pieces of video data (here, left-eye video data) with a predetermined size (for example, 240 pixels × 135 pixels). (Area) 180 (64 divisions here), and for any block 180a selected from the left-eye video data, a block 180b having the same size and high correlation value is extracted from the right-eye video data. The parallax 182 is detected from the positional relationship between the two. As such pattern matching, for example, an SAD (Sum of Absolute Difference) that takes a difference in luminance value, an SSD (Sum of Squared intensity Difference) that uses the difference squared, or an average value from the luminance value of each pixel. Since various existing methods such as NCC (Normalized Cross Correlation) that takes the similarity of the subtracted variance values can be adopted, detailed description thereof is omitted here.

  Subsequently, the parallax deriving unit 174 adds up the derived parallaxes of the plurality of blocks 180 in units of frames. For example, the parallax deriving unit 174 extracts a near-maximum value that is the maximum value in the projection direction of each of the plurality of blocks 180 in one frame and a far-maximum value that is the maximum value in the depth direction. However, based on the perspective direction, the near maximum value is represented by a positive value, and the far maximum value is represented by a negative value. Therefore, although the far maximum value is the maximum value in the direction in which the absolute value becomes deeper, the value itself corresponds to the minimum value.

  In addition, the parallax deriving unit 174 extracts, in a predetermined number of frames, the one that maximizes the parallax in the pop-out direction of the block 180 for each frame, compares the extracted maximum values between the frames, and extracts the maximum The largest maximum value among the values can also be specified as the near maximum value in the predetermined number of frames. The same applies to the far maximum value.

  Further, the disparity deriving unit 174 calculates the maximum value of the near maximum value in the frame over the entire time (all frames) (hereinafter referred to as the total near maximum value) and the entire time of all the maximum values in the frame (all frames). ) (The maximum value in absolute value) (hereinafter referred to as the total far maximum value). Therefore, the total near maximum value and the total far maximum value are the maximum value in the projection direction of the parallax and the maximum value in the depth direction over the entire time of the video data. However, like the near maximum value and the far maximum value, the total near maximum value is represented by a positive value, and the total far maximum value is represented by a negative value. The total maximum value and total maximum value derived in this way are held together with the video data as parallax information, and are referred to in the stereoscopic video control device 120.

(3D image control device 120)
FIG. 5 is a functional block diagram illustrating schematic functions of the stereoscopic video control apparatus 120. As shown in FIG. 5, the stereoscopic video control apparatus 120 includes a video data acquisition unit 210, a synthesis unit 212, an operation unit 214, and a central control unit 216. The central control unit 216 functions as an observation environment registration unit 220 and a parallax correction unit 222. The synthesizing unit 212, the operation unit 214, and the central control unit 216 have substantially the same functions as the synthesizing unit 154, the operation unit 158, and the central control unit 162 that have already been described as components in the stereoscopic video imaging apparatus 110, and thus redundant description is omitted. Here, the video data acquisition unit 210, the observation environment registration unit 220, and the parallax correction unit 222 having different configurations will be mainly described.

  The video data acquisition unit 210 converts left and right two-eye parallaxes having binocular parallax obtained by converting a video image perceived as a three-dimensional video image and a CG video image corresponding to a video image captured at different optical axes. One video data is acquired. Here, the left and right video data may be acquired individually or may be acquired as integral data. In the present embodiment, it is assumed that the video data acquisition unit 210 acquires video data generated by the stereoscopic video imaging device 110, and the video data is associated with disparity information including the near maximum value and the maximum far value. Suppose that

  The observation environment registration unit 220 registers observation environment information of the stereoscopic video display device 130 in accordance with an input by the observer 12 through the operation unit 214. Here, the observation environment information is information including the screen size of the stereoscopic video display device 130 and the observation distance that is the distance between the observer 12 and the stereoscopic video display device 130.

  FIG. 6 is an explanatory diagram for explaining a predetermined parallax range based on observation environment information. As shown in FIG. 6A, when the object 232a is arranged in the left-eye video data 230a and the object 232b is arranged in the right-eye video data 230b (the object 232a of the left-eye video data 230a is for the right eye When present on the right side of the object 232b of the video data 230b: parallax> 0), a straight line connecting the left eye 234a and the object 232a of the video data 230a for the left eye, and an object 232b of the video data 230b for the right eye 234b and the right eye And the straight line connecting the two intersect each other in front of the screen of the stereoscopic image display device 130, and the image of the object 232 is imaged at the intersecting point, and the observer 12 sees the object 232 popping out.

For example, an angle formed by a connection line between the imaging position of the object 232 and the left eye 234a and the right eye 234b is θ _f, and a connection line connecting the fixation point 236 on the screen of the stereoscopic image display device 130 with the left eye 234a and the right eye 234b. When the corner is referred to as theta _c formed by the predetermined parallax range is θ _f -θ _c <1.7 °. Therefore, when the interpupillary distance I is fixed at 5 cm to 6.25 cm, the predetermined parallax range H is uniquely derived from the screen size S and the observation distance L of the stereoscopic image display device 130.

Further, the predetermined parallax range is not limited to the case where the object 232 is imaged on the near side of the stereoscopic video display device 130, but is also applied to the case where the object 232 is imaged on the far side. As shown in FIG. 6B, when the object 232a is arranged in the left-eye video data 230a and the object 232b is arranged in the right-eye video data 230b (the object 232a of the left-eye video data 230a is for the right eye When present on the left side of the object 232b of the video data 230b: parallax <0), a straight line connecting the left eye 234a and the object 232a of the video data 230a for the left eye, and an object 232b of the video data 230b for the right eye 234b and the right eye And the line connecting the two intersect with each other at the rear of the screen of the stereoscopic image display device 130, and the image of the object 232 is imaged at the intersecting point, and the observer 12 sees the object 232 in the back. At this time, if the angle formed by the connection between the imaging position of the object 232 and the left eye 234a and the right eye 234b is θ _b , the predetermined parallax range is θ _c −θ _b <1.7 °, and FIG. a) Similarly, the predetermined parallax range H is uniquely derived from the screen size S and the observation distance L of the stereoscopic image display device 130.

  Therefore, the observation environment registration unit 220 can derive a predetermined parallax range according to the screen size of the stereoscopic video display device 130 and the observation distance.

(Shift adjustment)
The parallax correction unit 222 compares the total near maximum value and the total far maximum value associated with the two video data acquired by the video data acquisition unit 210 with the predetermined parallax range derived by the observation environment registration unit 220, The video data is processed according to the comparison result, and the parallax is corrected.

For example, a parallax range satisfying the above-described θ _f −θ _c <1.7 ° and θ _c −θ _b <1.7 ° is set as a lower limit P _D <parallax p <upper limit P _U. If the total near-maximum value at this time parallax p it is grasped that exceeds the upper limit P _U, the frame there are blocks corresponding to the total near maximum, it can be seen that the disparity is not within a predetermined parallax range. However, even if the parallax p exceeds the upper limit P _U in this way, if the parallax p of another block at that time is equal to or higher than the lower limit P _D , the parallax is shifted in the negative direction as a whole. by, it is possible to avoid that the parallax p exceeds the upper limit P _U.

Specifically, the parallax correction unit 222, the total near or maximum value is staggering limit P _U, the absolute of the absolute value of acceleration P _D of the total distance the maximum value is below the lower limit P _D (total far maximum value and come to value), and the difference between the total near-maximum value and the total distance the maximum value be less than the difference between the upper limit P _U and the lower limit P _D, relatively shifting the two image data in the horizontal direction Thus, the total near maximum value and the total far maximum value are both processed so as to fall within the upper limit P _U and the lower limit P _D. However, since the size of the video data is usually fixed, a part of the video data that can be visually recognized is lost in the horizontal direction by shifting one of the video data in the horizontal direction. Therefore, the aspect ratio of the visually recognizable image cannot be maintained in the horizontal direction, and a display (band-like black region) indicating that there is no image is made in that portion. With this configuration, it is possible to avoid a situation in which a burden is placed on the eyes of the observer 12, although the original video data is different from the intended perspective.

(Size adjustment)
On the other hand, there is a limit to the shift adjustment. That is, when a near object and a distant object are simultaneously captured in the imaging unit 150, and the difference between the total near maximum value and the total far maximum value is larger than the difference between the upper limit P _U and the lower limit P _D , also two of the video data as the relative shift in either direction, either the total near-maximum value and the total distance the maximum value exceeds the range of always limit P _U and the lower limit P _D.

  Therefore, the parallax correction unit 222 determines whether the total near-maximum value and the total far-maximum value are within the predetermined parallax range, and when it is determined that they are not within the predetermined parallax range, The video is reduced in the horizontal direction so that the far maximum value falls within a predetermined parallax range. In stereoscopic video, the larger the horizontal parallax, the more the object will pop out and the deeper it will be. Here, the parallax correction unit 222 reduces the absolute amount of parallax by reducing the image itself in the horizontal direction.

FIG. 7 is an explanatory diagram for explaining the size adjustment processing by the parallax correction unit 222. Here, the excess parallax is taken on the horizontal axis, and the maximum excess parallax (limit value) is set to 1, and the relative value is shown. Here, the excess parallax refers to the parallax in excess when the parallax exceeds the difference between the upper limit P _U and the lower limit P _D that are a predetermined parallax range. Further, the vertical axis represents the reduction ratio of the screen, and a non-reduced state is represented by 1 as a relative value. When the difference between the total near maximum value and the total far maximum value is larger than the difference between the upper limit P _U and the lower limit P _D , the parallax correction unit 222 fixes the above-described shift adjustment with a predetermined adjustment value, and performs size adjustment. Do. The parallax correction unit 222 reduces the video only in the horizontal direction while maintaining the vertical reduction ratio of the video at 1 while the excess parallax is small (see FIG. 7 (1)).

  When the excess parallax is reduced in the horizontal direction, the ratio between the horizontal direction and the vertical direction (hereinafter simply referred to as aspect ratio) is greater than or equal to a first threshold value (here, 1/16) that is assumed to be uncomfortable. (For example, when the aspect ratio before reduction is 16: 9 but becomes 15: 9), the horizontal direction of the video is reduced along with the reduction in the horizontal direction so as not to increase the sense of strangeness of the aspect ratio. Is also executed (see FIG. 7B). However, the absolute value of the vertical reduction ratio (reduction ratio / excess parallax) with respect to the excess parallax is larger than the absolute value of the horizontal reduction ratio. This is to make the aspect ratio equal to the aspect ratio before reduction when the excess parallax is greater than or equal to the first threshold. Here, the start point of reduction in the vertical direction is set to an excess parallax value corresponding to an aspect ratio of 15: 9. However, the value is not limited to this value, and can be arbitrarily set by the observer 12.

  In FIG. 7, when the excess parallax reaches the second threshold (here, 0.5), the aspect ratio returns to the aspect ratio before reduction. Therefore, when the excess parallax is equal to or larger than the second threshold, the reduction is performed in the horizontal direction and the vertical direction with the same reduction ratio (maintaining the aspect ratio) with respect to the excess parallax (see FIG. 7 (3)). By determining the reduction ratio of the video based on the trajectory shown in FIG. 7, the viewer 12 can take as large a video as possible to view without feeling uncomfortable due to the aspect ratio.

  Here, the aspect ratio before reduction and in the state of FIG. 7 (3) is preferably 16: 9 or 4: 3. However, the effective display area is 16: 9 or 4: 3 by the shift adjustment described above. May not be maintained. In this case, the video reduction ratio is determined based on the final aspect ratio of the shift adjustment.

  Hereinafter, a specific processing flow of the above-described stereoscopic video imaging device 110 and the stereoscopic video control device 120 will be described with reference to flowcharts of FIGS. 8 and 9.

(Stereoscopic imaging method)
According to FIG. 8, when the stereoscopic video imaging device 110 is turned on, the initialization process is activated, and the variables (total near maximum value and total far maximum value) in the stereoscopic video imaging device 110 are reset to zero. (S300). When the photographer 10 attempts to capture a stereoscopic video through the stereoscopic video imaging device 110 (YES in S302), the imaging control unit 170 of the stereoscopic video imaging device 110 determines an appropriate video according to the operation input of the imaging person 10. The focus lens 150b, the diaphragm 150c, and the image sensor 150d are driven by the drive unit 150e so that data can be obtained (S304). Further, the storage control unit 172 stores the video data transmitted from the data processing unit 152 in the video holding unit 160 while the imaging unit 150 is generating video data (S306).

  Then, the parallax deriving unit 174 derives the parallax of one or more blocks of the two stored video data through pattern matching (S308), and extracts the near maximum value and the far maximum value (S310). Subsequently, the parallax deriving unit 174 compares the near maximum value with the total near maximum value (S312), and if the near maximum value exceeds the total near maximum value (YES in S312), the disparity maximum value is set to the near maximum value. To generate a new total maximum value (S314). Similarly, the parallax deriving unit 174 compares the far maximum value and the total far maximum value (S316), and if the far maximum value falls below the total far maximum value (YES in S316), the distant maximum value is converted into the far maximum value. To generate a new maximum total far value (S318).

  Next, it is determined whether or not the photographer 10 has performed a stereoscopic video imaging stop operation (S320). If the stop operation has not been performed (NO in S320), the imaging unit driving process in step S304 is repeated and stopped. If the operation has been performed (YES in S320), the total near-maximum value and the total far-maximum value are held together with the video data as parallax information (S322), and the process returns to the operation waiting process for the photographer 10 in step S302.

(3D image control method)
According to FIG. 9, when the observation environment registration unit 220 of the stereoscopic video control device 120 receives an input from the observer 12 through the operation unit 214 (YES in S350), the stereoscopic video display is performed according to the input by the observer 12 The observation environment information (screen size and observation distance of the stereoscopic video display device 130) of the device 130 is registered (S352). Then, when the viewer 12 tries to reproduce the stereoscopic video through the stereoscopic video control device 120 (YES in S354), the video data acquisition unit 210 starts acquiring two video data (S356) and registers the observation environment. The unit 220 derives a predetermined parallax range through the observation environment and the parallax information held together with the video data (S358).

Here, when the parallax range satisfying the above θ _f −θ _c <1.7 ° and θ _c −θ _b <1.7 ° is set as the lower limit P _D <parallax p <upper limit P _U , It is determined whether the value exceeds the upper limit P _U or the total disparity maximum value of the parallax is lower than the lower limit P _D (S360). If neither condition is satisfied (NO in S360), the process proceeds to the video data transmission process in step S376. If any one of the conditions is satisfied (YES in S360), it is determined whether or not the difference between the total near maximum value and the total far maximum value is smaller than the difference between the upper limit P _U and the lower limit P _D (S362). If _U and lower _P smaller than the difference between _D (YES in S362), the parallax correction unit 222, one of the two image data is shifted in the horizontal direction (S364). In this way, as shown in FIG. 10A, the horizontal end of the video displayed on the stereoscopic video display device 130 is missing, but the observer 12 can observe the stereoscopic video within a predetermined parallax range.

Further, when the difference between the total near-maximum value and the total distance the maximum value is equal to or greater than the difference between the upper P _U and the lower limit P _D (NO in S362), the upper limit P _U and the difference or excess disparity between the lower limit P _D It is determined whether or not it is less than the first threshold (here, 1/16) (S366), and if it is less than the first threshold (YES in S366), the parallax correction unit 222 refers to FIG. As indicated by the white arrow in b), the image is reduced only in the horizontal direction at a reduction ratio corresponding to the excess parallax (S368). If the excess parallax is greater than or equal to the first threshold (NO in S366), it is determined whether or not the excess parallax is less than the second threshold (here, 0.5) (S370), and if it is less than the second threshold ( In step S370, the parallax correction unit 222 refers to FIG. 7, and reduces the horizontal and vertical directions of the video at a reduction ratio corresponding to the excess parallax, as indicated by the white arrow in FIG. 10C (S372). ). However, the desired aspect ratio has not been reached at this point. If the excess parallax is equal to or greater than the second threshold (NO in S370), the parallax correction unit 222 refers to FIG. 7, and the video is reduced at a reduction ratio corresponding to the excess parallax, as indicated by the white arrow in FIG. The horizontal and vertical directions are reduced (S374). However, here, the video is reduced in the horizontal direction and the vertical direction at the same reduction ratio with respect to the excess parallax (maintaining a desired aspect ratio). Then, the video data acquired at the reduction ratio determined in this way is synthesized and transmitted to the stereoscopic video display device 130 (S376).

  As described above, when the stereoscopic video imaging apparatus 110 and the stereoscopic video control apparatus 120 described above reproduce a stereoscopic video, the stereoscopic video is displayed in a predetermined parallax range regardless of the parallax of the stereoscopic video, the screen size of the stereoscopic video display apparatus, and the observation distance. This makes it possible to form an image on the observer.

  Furthermore, a stereoscopic video imaging program and a stereoscopic video control program that cause the computer to function as the stereoscopic video imaging device 110 and the stereoscopic video control device 120, and the stereoscopic video imaging program and the stereoscopic video control program recorded thereon are read by the computer. Recording media such as a flexible disk, a magneto-optical disk, a ROM, an EPROM, an EEPROM, a CD, a DVD, and a BD are also provided. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the observation environment registration unit 220 in the stereoscopic video control device 120 obtains the observation environment and derives a predetermined parallax range by combining with the parallax information. When the observation environment can be grasped in advance, for example, the stereoscopic image display device 130 can be specified, the predetermined parallax range can be derived on the stereoscopic image capturing device 110 side. That is, the stereoscopic video imaging apparatus 110 can have the function of the observation environment registration unit 220.

  In the above-described embodiment, the observation environment registration unit 220 in the stereoscopic video control device 120 obtains a predetermined parallax range using the total near maximum value or the total far maximum value generated by the stereoscopic video imaging device 110. However, when the acquired video data does not include such a near maximum value or a total far maximum value, after acquiring the video data on the stereoscopic video control device 120 side, check a plurality of frames. Alternatively, the near maximum value and the far maximum value of the parallax for each frame may be extracted to determine the total near maximum value or the total far maximum value. In other words, the stereoscopic video control device 120 may have a part of the function of the parallax deriving unit 174.

  In the above-described embodiment, the parallax correction unit 222 in the stereoscopic video control device 120 determines one correction amount (shift adjustment and size adjustment) related to the parallax for the acquired video data, and the determination result is displayed as video. Although an example in which the data is fixedly applied to the entire time has been described, the present invention is not limited thereto, and shift adjustment and size adjustment may be performed every moment according to a change in parallax in video data. In this case, it is assumed that the parallax information includes the near maximum value and the far maximum value of the parallax for each frame, and the parallax correction unit 222 calculates the near maximum value, the far maximum value, and a predetermined parallax range each time. In comparison, it is assumed that the horizontal shift amount and the horizontal and vertical reduction ratios are determined in real time. At this time, for example, one or a plurality of low-pass filters (LPF) may be applied to the calculation result to prevent the size of the image from changing complicatedly. By doing so, the size of the screen is not limited by the shift adjustment or the size adjustment even for an image with no large parallax, so that a stereoscopic image can be viewed on a large screen.

  Further, in the stereoscopic image capturing apparatus 110, the configuration in which the parallax is derived with respect to one or a plurality of blocks and the near maximum value and the far maximum value are extracted has been described, but the method of reflecting the parallax in the shift adjustment and the size adjustment is described. The present invention is not limited to this case, and various other statistical methods using a disparity distribution (disparity distribution) in one or a plurality of blocks can be adopted.

  FIG. 11 is an explanatory diagram illustrating the parallax distribution in one frame. In the disparity distribution as shown in FIG. 11, when the near maximum value and the far maximum value are extracted as in the above-described embodiment, for example, the disparity of the block 190 becomes the far maximum value. However, since the parallax of the block 190 has a small number of blocks, it can be determined that the influence of the block 190 itself on the entire video data is small or noise. In this way, not only simple calculations such as the near maximum value and the far maximum value but also the disparity distribution can be used to consider the influence of the disparity of each block on the whole. For example, an average value of the parallax distribution as shown in FIG. 11 is derived, and from the average value, only a predetermined parallax range (for example, ± 40%) is targeted, and other parallax ranges are ignored, or the parallax is 0 From this point, other parallax ranges may be ignored only for a predetermined parallax range (for example, ± 40%). In addition, disparity having a number of blocks equal to or less than a predetermined number (for example, 2) may be ignored.

  With such a configuration, the degree of influence of the parallax of each block on the video data can be reflected more specifically, so that the observer 12 can avoid the burden on the eyes and the stereoscopic video with a more optimal screen size. Can be visually recognized.

  Note that each step in the stereoscopic video imaging method and the stereoscopic video control method of the present specification does not necessarily have to be processed in time series in the order described as the flowchart, and may include processing in parallel or by a subroutine.

  The present invention can be used for a stereoscopic video control apparatus, a stereoscopic video control method, and a stereoscopic video control program for reproducing video data for perceiving stereoscopic video.

DESCRIPTION OF SYMBOLS 100 ... 3D image system 110 ... 3D image imaging device 120 ... 3D image control apparatus 174 ... Parallax derivation | leading-out part 220 ... Observation environment registration part 222 ... Parallax correction part

Claims (7)

An observation environment registration unit for registering observation environment information including a screen size and an observation distance that is a distance between the observer and the screen;
Two video data having binocular parallax and a video data acquisition unit for acquiring parallax information related to the parallax;
Based on the observation environment information registered in the observation environment registration unit and the parallax information acquired by the video data acquisition unit, it is determined whether or not the parallax of the video data falls within a predetermined parallax range. A parallax correction unit that reduces the video based on the two video data at least in the horizontal direction so that the parallax of the video data falls within the predetermined parallax range when it is determined that the video data does not fall within the range;
A stereoscopic video control apparatus comprising: The video data acquisition unit acquires a near-maximum value that is a maximum value in a projection direction of parallax in a predetermined video unit of the two video data and a far-maximum value that is a maximum value in a direction of deepening,
The stereoscopic video control apparatus according to claim 1, wherein the parallax correction unit reduces the video at least in a horizontal direction so that both the near maximum value and the far maximum value fall within a predetermined parallax range. . The video data acquisition unit further derives a maximum value of near maximum values over the entire time of the two video data as a total near maximum value, and a far maximum value over the entire time of the two video data. The largest value is derived as the maximum value
The parallax correction unit reduces the video at least in the horizontal direction over the entire time by a reduction ratio of 1 in which both the total near-maximum value and the total far-maximum value are within a predetermined parallax range. The stereoscopic video control apparatus according to claim 2. The parallax correction unit includes:
If the excess parallax exceeding the predetermined parallax range is less than the first threshold, the video is reduced only in the horizontal direction,
If the excess parallax is greater than or equal to the first threshold and less than a second threshold greater than the first threshold, the video is reduced in the horizontal and vertical directions;
4. The method according to claim 1, wherein when the excess parallax is equal to or greater than the second threshold, the video is reduced in the horizontal direction and the vertical direction while maintaining a predetermined aspect ratio. 5. Stereoscopic image control device.   5. The parallax correction unit according to claim 1, wherein after the video based on the two video data is relatively shifted in the horizontal direction, the video based on the two video data is reduced. The stereoscopic video control apparatus according to item. Register observation environment information including the screen size and the observation distance that is the distance between the observer and the screen.
Obtaining two video data having binocular parallax and parallax information about the parallax;
When it is determined whether the parallax of the two video data is within a predetermined parallax range based on the registered observation environment information and the acquired parallax information, and when it is determined that the parallax is not within the predetermined parallax range A stereoscopic video control method, wherein a video based on the two video data is reduced at least in a horizontal direction so that the parallax of the video data is within the predetermined parallax range. Computer
An observation environment registration unit for registering observation environment information including a screen size and an observation distance that is a distance between the observer and the screen;
Two video data having binocular parallax and a video data acquisition unit for acquiring parallax information related to the parallax;
Based on the observation environment information registered in the observation environment registration unit and the parallax information acquired by the video data acquisition unit, it is determined whether or not the parallax of the video data falls within a predetermined parallax range. A parallax correction unit that reduces the video based on the two video data at least in the horizontal direction so that the parallax of the video data falls within the predetermined parallax range when it is determined that the video data does not fall within the range;
3D video control program characterized in that it is made to function.

Images