WO2012086120A1 - Image processing apparatus, image pickup apparatus, image processing method, and program - Google Patents

Image processing apparatus, image pickup apparatus, image processing method, and program

Info

Publication number
WO2012086120A1
WO2012086120A1 PCT/JP2011/006380 JP2011006380W WO2012086120A1 WO 2012086120 A1 WO2012086120 A1 WO 2012086120A1 JP 2011006380 W JP2011006380 W JP 2011006380W WO 2012086120 A1 WO2012086120 A1 WO 2012086120A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
image
parallax
data
processing
pixel
Prior art date
Application number
PCT/JP2011/006380
Other languages
French (fr)
Japanese (ja)
Inventor
智生 木村
中村 剛
郁雄 渕上
大橋 政宏
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00Stereoscopic photography
    • G03B35/08Stereoscopic photography by simultaneous recording
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/97Determining parameters from multiple pictures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/122Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • G06T2207/10021Stereoscopic video; Stereoscopic image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20021Dividing image into blocks, subimages or windows
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Abstract

A parallax detection unit (103) detects a parallax between a left-eye image and a right-eye image inputted from image data input terminals (101, 102). An improper-parallax detection unit (104) compares the parallax detected by the parallax detection unit (103) and a prescribed threshold value, and evaluates whether the detected parallax is an improper parallax or not on the basis of the result of the comparison. Blurred image generating units (105, 106) execute blurring to pixel areas having improper parallaxes.

Description

The image processing apparatus, an imaging apparatus, an image processing method, program

The present invention relates to the technical field of image blurring technique, the image processing apparatus and mounted on a 3D camera, an image processing program to be used as a 3D camera accessories, an improvement of the image processing method.

Image blurring technology is to focus only on a part of the image of the subject is that of intentionally blurring an image processing other foreground and background. In such image processing causes complement the main object, it is possible to produce a stereoscopic effect.

The 3D camera, set of left-eye lens and the left eye optical system, have provided a set of right-eye lens and the right-eye optical system, the left-eye image data, and can simultaneously capturing the right-eye image data it is a camera. These left-eye image data, and the right-eye image data is reproduced by the stereoscopic dedicated display, these left-eye image data through the liquid crystal shutter glasses, and, by view right-eye image data, the user is stereoscopically You can enjoy the video.

For a publication invention for image blurring techniques are those described in Patent Document 1, the document known invention for 3D cameras, is disclosed in Patent Document 2.

Patent Document 1, the focal length f, F value of the lens, the distance difference by selecting a matrix depending on D, and discloses an image processing apparatus for adding blur by using the selected matrix.

Patent Document 2 calculates an optimum convergence angle to the main object, by performing the shooting, to clear the subject is in focus, discloses a 3D camera to blur the rest. The 3D camera described in this document 2, an image pickup device for the right eye and the left eye is moved by a predetermined amount, the amount of movement of the image of the main object to be photographed in the right eye and the image sensor for the left eye in the back and forth movement by calculating, it acquires the distance information.

Then, from a digital camera to be acquired can be calculated data the distance L to the main subject, repeated movement and the imaging device, and an acquisition of the distance information. Here, the base line connecting the lens and the lens drive unit for the left eye, the distance between the base line and the parallel line through the main object, and the distance L. After obtaining the distance information is completed, from the correlation of the right eye and the movement amount of the image pickup element for the left eye and the amount of movement of the image of the main object, using the principle of triangulation, the distance from the 3D camera to the main object L It is calculated.

Finally, as for optimum convergence angle to the main subject at the position of the calculated distance L, the optical system driving mechanism and the optical system drive mechanism for the right eye left eye, left eye imaging optical system and the right-eye imaging optical system the drives to correct the convergence angle. Here, the convergence angle, during stereoscopic playback, the line of sight of the right eye pupil, and that the angle between the line of sight of the left eye pupil, the optimal convergence angle of the distance L, the optical axis of the left eye imaging optical system intersection of the optical axis of the right-eye image pickup optical system includes a left-eye lens, a convergence angle such that the distance L from the midpoint of the base length of the right-eye lens. Stereoscopic effect, the convergence angle during stereoscopic playback, the ones corresponding to the difference between the convergence angle at the plan view reproduction (Non-Patent Document 3).

Hei 9-181966 Publication JP 2010-114577 JP

"Digital technology Research Report on movie production and display that make full use of" (goods) Digital Content Association, http: //www.dcaj.org/report/2009/data/dc_09_02.pdf, March 2010, 40 page Learning Depth from Single Monocular Images, AshutoshSaxena , etl., Computer Science Department Stanford University, http://ai.stanford.edu/~asaxena/learningdepth/NIPS_LearningDepth.pdf, 2005 "3D camera for people-friendly 3D spread safety Guidelines ", 3D Consortium, revised April 20, 2010

3D camera described in Patent Document 2, the left-eye image and the right-eye image by calculating the amount of movement of the image of the main object, according to the distance information based on the moving amount, the image pickup device and the right-left eye because it assumes feedback control of driving the imaging element of the use, for the main subject, although the convergence angle is present assurance that is appropriate, subject other than the main subject (e.g., the main subject is the person in some cases, what kind of convergence angle arises from the main subject of the sub-subject yelling-through to the foreground side and the background side), is not guaranteed. Here the conventional 3D camera, a left-eye image pickup optical system and the right-eye image pickup optical system to some extent in the 3D camera, because it is entirely separate optical systems present in the spaced position, the imaging optical system and for these eye left-eye image captured by the right imaging optical system, from a comparison image for the right eye, the position of the sub-subject yelling-through to the foreground and background is, the left-eye image, which may differ significantly between the right-eye image. Nevertheless even are having convergence angle with the main subject, thus, the position of the sub-subject yelling-through to the foreground and background and the left eye image, the very different images in the right-eye image, perform the stereoscopic playback If you try to be, there is that the convergence angle of the sub-subject yelling Ballmer to take the foreground and background is extremely large. The convergence angle of the sub-subject yelling-through to the foreground and background, the image pickup device and the image pickup device feedback control or the like for the right eye left eye, the adjustment is because not been Nanito. Thus, the left eye image of the sub-subject yelling-through to the foreground and background, the greater the amount of movement of the right eye image, since the stereoscopic effect for the sub-subject yelling-through to the foreground and background is extremely large, which leads to 3D sickness of the user.

Above, the left-eye image taken by the 3D camera described in Patent Document 2 has described the right-eye image, when the left-eye image and right eye image is pan-focus image, the 3D motion sickness problem more pronounced occur. The pan-focus image, or monofocal lenses, small lenses diameter, that is, the image captured by the depth of field is deep lenses, such as those in focus to all persons and objects, etc. in the image the say. Power focusing function of the digital camera (the power focusing function, instead of manually feeding the lens, the movement of the subject in the viewfinder over senses automatically, based on the information of the movement, feeding the lens motor, a lens characteristics is that the function to be interpolated.) also captured using the often deep-focus image is obtained. Left eye image is pan-focus image, the right-eye image, the main object appearing in an image, all of the sub-subject, because in focus, convergence angle for all of these objects must be appropriate. However, in the technique described in Patent Document 2, nearly virtually impossible to adjust the optical mechanism to adjust the convergence angle for all subjects, it is illegally larger one of the convergence angle of the plurality of objects become, fall into the situation that 3D sickness becomes excessive.

Here, the left eye image blur technique described in document 1, by applying the right-eye image, the left eye image, it is conceivable to blur based on the desired blur characteristic image for a right eye, the aforementioned as such, the left-eye image obtained by the 3D camera, a set of right-eye images, because they are obtained by a completely separate optical systems, the left eye image, even the right-eye image as a blurred based on the individual blur characteristics a left-eye image can not be blurred significantly only movement amount is large sub-subject in the right-eye image. Therefore, the blurring technique described in Patent Document 1, there is a problem that can not be eliminated and 3D motion sickness.

Above, the left-eye image obtained by the 3D camera has been described right-eye image, the left eye image, the set of the right eye image, rather than the image obtained by the 3D camera, as described in Non-Patent Document 2 as there also exist those generated from the plan view image. Left-eye image generated from the plan view image, a set of right-eye images, because not obtained from shooting against realistic things, whether the amount of movement suitability of the object is no guarantee exists. Therefore, in the technology described in Patent Documents 1 and 2, the stereoscopic image generated from the plan view image, it is impossible to rid the 3D motion sickness. The prior art is not affected by the type of image to be processed, in uniform, for the purpose of obtaining a satisfactory stereoscopic image can not be used.

An object of the present invention, the position of the sub-subject yelling-through to the foreground and background is, the left eye image, even when largely different between the right eye image, both the left eye image, the type of the right eye image it is possible to realize a proper stereoscopic effect uniformly without, is to provide a processing apparatus.

To achieve the above object, an image processing apparatus according to the present invention includes a receiving means for receiving an input of a plurality of image data, for each image data, and disparity detection unit for detecting the disparity with the other image data, performing among a plurality of pixel regions in the pair of images subjected to parallax detection, detection means for detecting what parallax becomes invalid as illegal disparity regions, blurring processing for each pixel regions constituting illegal parallax region characterized in that it comprises a processing means.

Whether the convergence angle is on how well the pixel area in the image corresponding to one of the viewpoint, or the difference between the pixel region in the image corresponding to a different perspective is much, that is, a plurality of images with parallax relationship between areas in the data. This disparity detects the pixel area to be incorrect, since subjected to blurring, can be blurred strongly region unauthorized disparity exists. As a result, it is possible to reduce the visual fatigue, discomfort caused by the illegal parallax.

In the present invention, by detecting the pixel region parallax may be illegal, the left-eye image and the right-eye image even pan focus image, allowing good stereoscopic playback because subjected to blurring. Therefore, when stereoscopic photography, no longer requires a feedback control that adjusts the convergence angle by driving the imaging element of the imaging device and the right eye left eye as described in US Pat. Accordingly, it is possible to develop a 3D camera that can shoot a stereoscopic image at a low price. If each of the left eye image and the right-eye image is a pan-focus image at the time of stereoscopic playback, all persons appearing in the image, buildings, trees, will be perceived depth is given to the road, great stereoscopic effect it is possible to obtain.

In the present invention, not limited to the captured image and video of the 3D camera, the prior art described in Non-Patent Documents 1 and 2, it processes the image and video data obtained by converting the 2D image and video object it is possible to, if the extension of converting 2D images and video to the stereoscopic image and video are present in 2D camera, that the apparatus according to the present invention is mounted to the 2D camera, the 2D camera it is possible to increase the added value.

Further in the present invention are either the image captured by the 3D camera, either a pan-focus image, without being influenced by the type of image such or those that have been converted from the plan view image, uniformly, 3D sickness occurs only in the pixel region as may be subjected to blurring.

Is a block diagram showing an example of an image processing apparatus 100 of the configuration according to the first embodiment of the present invention. It is a diagram for explaining detection of a parallax image for the left eye, the right eye image. It is a diagram for explaining the addition of blur against illegal parallax area. Distance and is a diagram for explaining the relationship between the parallax. Is a diagram showing the relationship between the blur strength and distance blurring process by the image processing apparatus 100. Is a flowchart showing the operation of the image processing apparatus 100. An example of a configuration of an image processing apparatus 700 according to the second embodiment of the present invention is a block diagram showing. One eye is a diagram for explaining the visible pixel area. Is a flowchart showing the operation of the image processing apparatus 700. An example of a configuration of an image processing apparatus 1000 according to the third embodiment of the present invention is a block diagram showing. Is a diagram showing the relationship between the blur strength and distance blurring process by the image processing apparatus 1000. An example of a configuration of an image processing apparatus 1200 according to the fourth embodiment of the present invention is a block diagram showing. It is a diagram for explaining the detection of the subject region. Is a diagram showing the relationship between the blur strength and distance blurring process by the image processing apparatus 1200. An example of a configuration of an image processing apparatus 1500 according to the fifth embodiment of the present invention is a block diagram showing. Is a block diagram showing an example of a configuration of an image processing apparatus 1600 according to the sixth embodiment of the present invention Is a flowchart showing the operation of the image processing apparatus 1600. An example of a configuration of an image processing apparatus 1800 according to the seventh embodiment of the present invention is a block diagram showing. Is a flowchart showing the operation of the image processing apparatus 1800. An example of a configuration of an image processing apparatus 2000 according to the eighth embodiment of the present invention is a block diagram showing. Is a diagram showing an example of application of the imaging apparatus according to a ninth embodiment of the present invention. Is a diagram showing an example of application of the imaging apparatus according to a ninth embodiment of the present invention. Is a diagram showing an example of application of the imaging apparatus according to a ninth embodiment of the present invention. An example of a configuration of an imaging apparatus 2400 according to a ninth embodiment of the present invention is a block diagram showing. Is a diagram illustrating an example of a user operation screen of the image pickup device 2400. It illustrates an example of a safe level of change screen Distance is a diagram showing an example of a change screen blurring processing based on the information. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Of changing the blurring processing based on the distance information, it is a diagram showing an example of a display screen. Is a diagram illustrating a blur strength curve is altered by a user operation. Is a diagram showing an example embodying the image processing apparatus according to the present invention in LSI.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
«Embodiment 1 >>
<1.1 Overview>
The image processing apparatus according to the first embodiment, a parallax between the left-eye image, the right-eye image used for display of stereoscopic and compared with a predetermined threshold value, based on the comparison result, determines whether an illegal parallax . Then performing blurring processing on a pixel area having a bad parallax. Thus it specifies a pixel area having a parallax in the viewer can cause visual fatigue, discomfort, stereoscopic sickness etc., by adding a blur to the pixel region, visual fatigue, discomfort caused by incorrect parallax it can be reduced.
<1.2 Configuration>
First, description will be given of a configuration of an image processing apparatus 100 according to the first embodiment. Figure 1 is a block diagram showing an example of a configuration of an image processing apparatus 100. As shown in FIG. 1, the image processing apparatus 100 includes an image data input terminal 101 and 102, the disparity detection unit 103, fraud parallax detection unit 104, the blurred image generating unit 105, 106, an image data output terminal 107, 108 in constructed.
<1.2.1 image data input terminals 101 and 102>
Image data input terminals 101 and 102, left-eye image, the right-eye image used for display of stereoscopic each is input. Here, the left eye image, the right-eye image is an image obtained by imaging the object field from a different perspective, it may be image data captured by e.g. a stereo camera. Not limited to the photographed image may be different from the virtual viewpoint CG created by assuming a (Computer Graphics) and the like. Further, even in a still image may be a moving image including a plurality of still images successive in time.
<1.2.2 parallax detection unit 103>
Parallax detection unit 103 detects the parallax between the input left-eye image, the right-eye image to the image data input terminals 101 and 102. Detection of the parallax is carried out in pixel block units of 16 × 16 pixels, and detects the number of horizontal pixels constituting between corresponding pixel block of the left-eye image, the right-eye image as parallax. Will be specifically described below detects the parallax between the left-eye image, the right-eye image with reference to FIG.

Figure 2 is a diagram showing the detection of a parallax image for the left eye, the right eye image. First, with reference to FIG. 2 (a), the detection of a corresponding pixel block will be described. Figure 2 (a) shows a left-eye image, the right-eye image that is pixel block divided. The detection of parallax is performed in pixel block units of 16 × 16 pixels are drawn as being divided into blocks in fact larger pixel blocks for description. A'pixel blocks A, B and B', C and C'shows corresponding pixel blocks. Corresponding pixel block is determined by a pixel block in which the pixel belongs, corresponding to one pixel belonging to the pixel block of the left-eye image, the pixel block corresponding to the pixel block of the left-eye image. Searching for the corresponding pixel is performed using, for example, block matching method. Here, the block matching method, and performs search using the pixel density pattern of surrounding pixels for performing corresponding point search. Next, with reference to FIG. 2 (b) · (c), it will be described the detection of the disparity.

2 (b) is a diagram showing the distance to the horizontal center of the image from the pixel block. dl (A), dl (B), dl (C) is a pixel block A of the left-eye image, respectively, B, the distance from C to the image center, dr (A'), dr (B'), dr (C' ) is a pixel block A'of the right-eye image, respectively, B', indicating the distance to the image center from C'. Distance is detected as the number of horizontal pixels constituting up image center from the pixel block. It is to be noted that the right direction is positive from the screen. Thus to calculate the disparity using the distance to the image center from the detected pixel block. Figure 2 (c) is a diagram showing a parallax of each pixel block. As shown in FIG. 2 (c), the disparity is determined by subtracting the distance dr to the image center from the pixel block of the right eye image from the distance dl to the image center from the pixel block of the image for the left eye. That is, the value of dl-dr is parallax. For example, the disparity in the pixel block A, the dl (A) -dr (A') = 10. As described above, it is possible to detect the parallax between the left-eye image, the right-eye image.
<1.2.3 fraud parallax detection unit 104>
Fraud parallax detection unit 104, the disparity detected by the disparity detection unit 103 makes a determination whether or incorrect parallax. First, we describe illegal parallax. In general, an excessively large positive parallax, a stereoscopic image having a negative disparity, visual fatigue, that may cause discomfort, stereoscopic sickness like are known, in the present invention such an observer the parallax of the stereoscopic image that may cause visual fatigue, discomfort, stereoscopic sickness such as incorrect parallax. For example, in the safety guidelines 3D Consortium issued (Non-Patent Document 3), it is recommended for comfortable stereoscopic viewing that parallax angle is within 1 degree. Therefore, between the upper limit value H, the lower limit value L and incorrect parallax D of predetermined to not cause visual fatigue, discomfort, stereoscopic sickness such disparity (comfortable parallax), D> H, the relationship between the D <L It holds.

Next, a description will determine incorrect parallax. Determination of a wrong or parallax compares the parallax to the value of the predetermined threshold, performing based on the comparison result. That is compared with the upper limit H and the lower limit value L of comfort parallax. Here, the value of threshold, the number of horizontal pixels based on the guideline values ​​of 3D Consortium recommended, defined as a predetermined threshold, determination may be performed in a wrong or parallax. In this embodiment, since the generation of more comfortable stereoscopic image, the number of horizontal pixels corresponding number of horizontal pixels corresponding to the upper limit of the range of comfortable parallax parallax angle of one degree, the lower limit on the parallax angle of 0 degrees to. The number of horizontal pixels corresponding to the parallax angle, the size of the display screen, the number of horizontal pixels of the image data, the distance to the display screen and the viewer, vary with the interpupillary distance of the observer.

For a typical 5 megapixel photos (2592 × 1944) (pupillary distance E: 65 mm), the number of horizontal pixels corresponding to the parallax angle of one degree becomes 102 pixels (see Non-Patent Document 3). In other words, in the viewing conditions determined, the disparity> 102 pixels, the case of the parallax <0 pixels, illegal parallax. Also, in the case of image and video of 1920 × 1080 pixels (pupillary distance E: 65 mm), the number of horizontal pixels corresponding to the parallax angle of one degree becomes 57 pixels (see Non-Patent Document 3). That is, in the viewing conditions of the determining, parallax> 57 pixels, the case of the parallax <0 pixels, illegal parallax. As described above, it is possible to determine the unauthorized parallax.
<1.2.4 blurred image generation unit 105 and 106>
Blurred image generating unit 105 performs blur processing on the pixel blocks having incorrect parallax of the input image from the image data input terminal 101. Blurred image generating unit 106 performs blur processing on the pixel blocks having incorrect parallax image input from the image data input terminal 103. Blurring uses a smoothing filter, for example 7 × 7. Below with reference to FIG. 3 illustrates the formation of blurred images.

Figure 3 is a diagram showing the addition of a blur disparity for negative incorrect parallax area. 3 (a) shows disparity negative incorrect parallax area. Hatched portion is determined to be the pixel blocks as an unauthorized parallax area. In this example, portions of the tree are determined as invalid disparity region, adding blur in this portion. That is, the smoothing filter process of 7 × 7 with respect to pixels constituting the tree. 3 (b) is a diagram showing a viewing stereoscopic image subjected to blurring processing. The left-eye image by 3D glasses or the like in the left eye, when viewed right eye image with the right eye, the stereoscopic image as shown in the figure is recognized. Thus part of the tree is incorrect parallax region is completely blurred. As described above, the blurred image generating unit 105 and 106, by performing blur processing for each pixel region constituting the unauthorized parallax region, is possible to reduce the visual fatigue, discomfort caused by incorrect parallax it can.

Further, blur processing according to the present embodiment is not merely hiding illegal parallax area. Simply hide or remove the incorrect parallax, it becomes an unnatural image to the viewer, blurring processing according to this embodiment, as an image is a so-called blur, so that it can not recognize the viewer unauthorized parallax it is intended to. This will be described below.

First it will be described the relationship between the distance and the parallax to the subject from the imaging position of the image data with reference to FIG. Figure 4 is a graph showing the relationship between the distance and parallax. a camera distance (base length), F is the focal length of the camera, dl is the distance from the image center in the left-eye image to the corresponding pixel C, dr is the distance from the image center in the right-eye image to the corresponding pixel C', x indicates the distance to the object appearing in the corresponding pixels from the imaging position. Since the parallax d is the deviation amount between the left-eye image, the right-eye image, minus the distance from the distance from the image center in the left-eye image to the corresponding pixel C to the corresponding pixel C'from the image center in the right-eye image , that is expressed by dl-dr. The length between the points C and C'is a a-dl + dr = a-d. Turning now to the similarity relationship △ DCC' and △ DAB, a: x = a-d: relationship x-F is obtained. Rearranging this equation, the relationship between the distance x and the parallax d can be expressed as x = aF / d. That is, the proportional relationship holds between the distance and parallax. This concludes the description of the relationship between the distance and parallax. Then, based on the proportional relationship established between the parallax and the distance, a description will be given of the relationship between blurring and distance.

Figure 5 is a graph showing the relationship between the intensity and the distance of adding the blurring by the blurring process. In the virtual space of distance 0-6, corresponds to the distance 1 is an upper limit value H comfort parallax, if the lower limit value L corresponds to the distance 5, the proportional relationship between the parallax and the distance is established, incorrect parallax range is the range of distances 0-1 and distance 5-6. Thus, as shown in FIG. 5 performs blurring processing on the foreground (distance 0-1) and distant (5-6). Thus, the result of performing blurring processing on unauthorized parallax area, will be carried out blurring processing on the near view and a distant view, as an image is a so-called blur, can be unrecognizable to a viewer fraud parallax. Since blurred except for a region of comfortable disparity, the object in the region of comfortable disparity is recognized observer as being more three-dimensional appearance.

Thus to identify unauthorized disparity regions, by adding a blur, as an image is a so-called blur, can be unrecognizable to a viewer unauthorized parallax, visual fatigue, discomfort caused by incorrect parallax it is possible to reduce the.
<1.2.5 image data output terminal 107 and 108>
Image data output terminal 107, the blurred image generated by the blurred image generating unit 105, the image data output terminal 108 outputs a blurred image generated by the blurred image generating unit 106, respectively. This concludes the description of the configuration of the image processing apparatus 100. Next, the operation of the image processing apparatus 100 will be described.
<1.3 Operation>
6 is a flow diagram illustrating the operation of the image processing apparatus 100. As shown in this figure, the parallax detection unit 103 first performs the search of the pixel block of the right-eye image corresponding to the pixel block of the left-eye image (step S601). Specifically, a pixel to first corresponding to one pixel belonging to the pixel block of the left-eye image, calculated by performing the corresponding point search for the right eye image, then the pixel blocks belonging of this corresponding point pixel, for the left eye searching for a corresponding pixel block by pixel block corresponding to the pixel blocks of the image. Then the disparity detection unit 103, for each left-eye image, the right-eye image, and calculates the number of pixels constituting the inter image center from the pixel block (step S602). Then by subtracting the number of pixels dr constituting the inter image center from the pixel block of the right eye image from the number of pixels dl constituting the inter image center from the pixel block of the left-eye image, and calculates the number of pixels constituting the parallax (step S603 ). Fraud parallax detection unit 104, the number of pixels constituting the parallax calculated by the parallax detection unit 103 determines whether the incorrect parallax (step S604). This determination compares the parallax with a predetermined threshold (upper limit value and lower limit value of a range of comfortable disparity) is performed based on the comparison result. Repeated for all the pixels block operation from step S601 described above to step S604. After the operation of step S604 from step S601 for all the pixel blocks, the blurred image generating unit 105, 106 performs blur processing on the pixel block (invalid disparity region) having the incorrect parallax (step S605). This concludes the description of the operation of the image processing apparatus 100.

According to the present embodiment as described above, by specifying the region having the parallax of stereoscopic images which can cause visual fatigue, discomfort, stereoscopic sickness like the viewer, it adds a blur to this region, the so-called as an image there is a blur, so as not to be recognized by the observer unauthorized parallax, it is possible to reduce the visual fatigue, discomfort caused by the illegal parallax.
«Embodiment 2 >>
<2.1 Overview>
The image processing apparatus according to the second embodiment, similarly to the image processing apparatus 100 according to the first embodiment, an image processing apparatus for processing a plurality of images constituting a stereoscopic using binocular parallax, incorrect parallax region by performing blurring processing on, as an image is a so-called blur, so unrecognizable to the observer unauthorized parallax, but is intended to reduce the visual fatigue, discomfort caused by incorrect parallax, incorrect parallax region in addition, since the crowded reflected only on one of the left-eye image, the right-eye image, flickering is generated by one eye visible at the time of stereoscopic playback, may cause visual fatigue, discomfort, stereoscopic sickness like the viewer the image area (one eye visible pixel area), transferring the other image area on the same image coordinate range and one eye visible pixel area (overwritten) to. Then performing blurring processing on the transfer source pixel region and the transcriptional pixel region. Prevented this, an image with a so-called blurring, a difference of the color and brightness is no longer recognized by the observer, the visual fatigue, discomfort due to the image area crowded reflected only on one of the left-eye image, the right eye image can do.
<2.2 Configuration>
First, description will be given of a configuration of an image processing apparatus 700 according to the second embodiment. Figure 7 is a block diagram showing an example of a configuration of an image processing apparatus 700. The configuration and the same part of the image processing apparatus 100 according to the first embodiment shown in FIG. 1, will not be described are denoted by the same reference numerals, different parts will be described. As shown in FIG. 7, the image processing apparatus 700 includes an image data input terminal 101 and 102, the disparity detection unit 103, fraud parallax detection unit 104, the interpolation image generating unit 701, the blurred image generating unit 702, an image output terminal configured to include a 107 and 108.
<2.2.1 interpolation image generation unit 701>
During stereoscopic viewing, result of the difference of the color and brightness is recognized by the viewer, flickering occurs, it can cause visual fatigue, discomfort, stereoscopic sickness like the observer. Interpolation image generating unit 701, according one eye to identify a visible pixel area, for the one eye visible pixel area, transferring the other image area on the same image coordinate range (overwriting). It will be described with reference more specifically to FIG. Figure 8 is a diagram showing one eye visible pixel area. One visible pixel region, located at the end of the image data. As shown in FIG. 8 (a), for example, in the pixel region shown by oblique lines of the left-eye image, the disparity is not detected corresponding pixel point in the right eye image is not calculated. Determining the pixel block areas such disparity is not calculated as monocular visible pixel area. Then, the pixel region shown by oblique lines of the right eye image in the same image coordinate range and one pixel area is transferred onto the pixel region shown by oblique lines of the left eye image (overwrites). Since one eye visible pixel area located at the end of the image data, pixels of overwriting is the pixel located at the end of the image data. 8 (b) is a diagram showing a left eye image after the interpolation image generation. As shown in FIG. 8 (b), one eye visible pixel region of the left-eye image pixels of the hatched portion of the right-eye image is displayed.

<2.2.2 blurred image generation unit 702 and 703>
Blurred image generating unit 702 and 703, in addition to the pixel blocks having a bad parallax (illegal parallax region), it performs blurring processing on the transfer source pixel region and the transcriptional pixel region. Thus the difference of the color and brightness can not be recognized by the observer. The one eye visible pixel area, so located at the end of the image data, as an image is a so-called blur can remove a sense of discomfort for the observer. This concludes the description of the configuration of the image processing apparatus 700. Next, the operation of the image processing apparatus 700 will be described.
<2.3 Operation>
Figure 9 is a flow diagram illustrating the operation of the image processing apparatus 700. As shown in this figure, the parallax detection unit 103 first performs the search of the pixel block of the right-eye image corresponding to the pixel block of the left-eye image (step S901). Specifically, a pixel to first corresponding to one pixel belonging to the pixel block of the left-eye image, calculated by performing the corresponding point search for the right eye image, then the pixel blocks belonging of this corresponding point pixel, for the left eye searching for a corresponding pixel block by pixel block corresponding to the pixel blocks of the image. Then, the corresponding pixel point a determination is made or not detected in step S901 (step S902). If a corresponding pixel point has been detected (step S902, YES) parallax detection unit 103, for each left-eye image, the right-eye image, and calculates the number of pixels constituting the inter image center from the pixel block (Step S903) . Then by subtracting the number of pixels dr constituting the inter image center from the pixel block of the right eye image from the number of pixels dl constituting the inter image center from the pixel block of the left-eye image, and calculates the number of pixels constituting the parallax (Step S904 ). Fraud parallax detection unit 104, the number of pixels constituting the parallax calculated by the parallax detection unit 103 determines whether the incorrect parallax (step S905). This determination compares the parallax with a predetermined threshold (upper limit value and lower limit value of a range of comfortable disparity) is performed based on the comparison result. Repeated for all the pixels block operations from step S901 described above to step S905. After the operation in step S905 for all the pixel blocks from the step S901, the interpolation image generation unit 701 identifies a pixel block parallax information is not calculated as a monocular visible pixel area (step S906). Then, the interpolation image generating unit 701 for the one eye visible pixel area, transferring the other image area on the same image coordinate range (overwriting). (Step S907). After the above interpolation image generation, the blurred image generating unit 702 performs blur processing on the illegal disparity regions and transfer-source pixel region and the transcriptional pixel area of ​​the image for left eye, right eye image, respectively. (Step S908). This concludes the description of the operation of the image processing apparatus 700.

According to this embodiment as described above, to identify the image area (one eye visible pixel area) exists only in one of the images of the left eye image, the right-eye image for one eye visible pixel area, the image it can be the other image area in the coordinate range is transferred (overwriting). Viewer in addition to incorrect parallax region having the parallax of stereoscopic images which can cause visual fatigue, discomfort, stereoscopic sickness like, by performing blur processing for the transfer source of the pixel region and the transcriptional pixel region, as an image where there is a so-called blur, remove the discomfort to the viewer, it is possible to reduce the visual fatigue, discomfort caused by the image area yelling reflected only on one of the left-eye image, the right-eye image.
«Embodiment 3»
<3.1 Overview>
The image processing apparatus according to the third embodiment, similarly to the image processing apparatus 700 according to the second embodiment, an image processing apparatus for processing a plurality of images constituting a stereoscopic using binocular parallax, incorrect parallax region by performing blurring processing on, as an image is a so-called blur, so unrecognizable to the observer unauthorized parallax, etc., but is intended to reduce the visual fatigue, discomfort caused by incorrect parallax, the distance information comprising a generator, based on the parallax between the left-eye image, the right eye image, and generates the distance information from the imaging position of the image data for each pixel block to an object appearing in the pixel block. Then performing blurring processing on the basis of incorrect parallax area and distance information. Thus, an image with a more natural blurring, can be made unrecognizable to a viewer unauthorized parallax like.
<3.2 Configuration>
Figure 10 is a block diagram showing an example of a configuration of an image processing apparatus 1000. The configuration and the same part of the image processing apparatus 700 according to the second embodiment shown in FIG. 7, omitted will be denoted by the same reference numerals, different parts will be described. As shown in FIG. 10, the image processing unit 1000, image data input terminals 101 and 102, the disparity detection unit 103, the distance information generating unit 1001, fraud parallax detection unit 104, the interpolation image generating unit 701, the blurred image generating unit 1002 configured to include 1003, an image data output terminal 107.
<3.2.1 distance information generating unit 1001>
Distance information generating unit 1001 generates the distance information based on the disparity between the detected left-eye image, the right-eye image by the disparity detection unit 103, the imaging position of the image data for each pixel block to an object appearing in a pixel block . The distance information is the triangulation principle, that is already using the distance information and parallax relationship described with reference to FIG. 4, can be calculated from the value of the disparity.
<3.2.2 blurred image generation 1002 and 1003>
Blurred image generating unit 1002 and 1003, based on the generated distance information with incorrect parallax region, transfer source of the pixel region and transferring pixel regions in the transfer for one eye visible pixel area and distance information generating unit 1001, left-eye image · carry out the blurring process on the right-eye image. Specifically, the transfer source of the pixel regions, transcribed pixel regions in the transfer against unauthorized parallax region and one eye visible pixel area, performs a smoothing filter process of 7 × 7, a predetermined focal length in the other region the blurring of the relative distance based on the intensity of the conducted. Focal length may be the distance of the center position of the image. Further, it may be one that is determined from the camera's auto-focus operation. It is also the distance of the pixel area selected by the user.

Distance blurring based on the information, for example using a technique described in Patent Document 1. That lens point spread function; the matrix corresponding to (PSF point spread fanction), selected according to the relative distance from the focal length, imparting blur by using the selected matrix.

11, the distance 0-1, and distance illegal parallax 5-6 is a diagram showing an example of a blur intensity when detected. As shown in FIG. 11, a blurring intensity at a distance from 0 to 1, 5 to 6 is maximized, inverse hyperbolic around a focal length 3 in the region of the distances 1-5 except incorrect parallax regions incorrect parallax region It has become a blur strength. Thus, an image with a more natural blur, can be unrecognizable to a viewer unauthorized parallax like.

According to this embodiment as described above, it generates the distance information from the imaging position of the image data for each pixel block to an object appearing in the pixel block. Then, in addition to the blurring process with respect to fraud disparity regions, by adding blur blur intensity according to the distance information regions other than invalid parallax area, an image with a more natural blurring, the observer illegal parallax, etc. «embodiment can so that it can not be recognized 4»
<4.1 Overview>
The image processing apparatus according to the fourth embodiment, similarly to the image processing apparatus according to the third embodiment, an image processing apparatus for processing a plurality of images constituting a stereoscopic using binocular parallax, incorrect parallax region, but it performs a shading process based on the distance information includes a subject detection unit in addition to the configuration of an image processing apparatus according to the third embodiment, detects a pixel area (object area) in which a predetermined object objects appear. And based on the incorrect parallax area and distance information and subject area, it performs the blurring process. Distance in addition to performing processing blur based on subject region information, does not become blurred image object to be focused is blurred uniformly in distance positional relationship, an image with good blurring visibility with respect to the object , it can be unrecognizable to a viewer unauthorized parallax like.
<4.2 Configuration>
Figure 12 is a block diagram showing an example of a configuration of an image processing apparatus 1200. Note the configuration and the same part of the image processing apparatus 1000 shown in FIG. 10, not explained are denoted by the same reference numerals, different parts will be described. As shown in FIG. 12, the image processing apparatus 1200, the image data input terminals 101 and 102, the disparity detection unit 103, the distance information generating unit 1001, the object detection unit 1201, fraud parallax detection unit 104, the interpolation image generating unit 701, blurred image generating unit 1202 and 1203, and includes an image data output terminal 107.
<4.2.1 subject detection unit 1201>
Subject detection unit 1201 detects the pixel area (object area) in which a predetermined object objects appear. In this embodiment, the detection of the subject region by template matching. Figure 13 is a diagram showing a mechanism of template matching. As shown in FIG. 13, it is prepared in advance face, as a template image 1301 specific object such as a person. Then, against the input image data 1202 detects a subject region by the similarity between the template image 1301 to detect certain level of object 1203.
<4.2.2 blurred image generation unit 1202 - 1203>
Blurred image generating unit 1202, 1203 is based on the object area detected by the fraud disparity regions, transcription source of the pixel region and transferring pixel regions in the transfer for one eye visible pixel area, distance information, and the subject detection unit 1201, the left eye performing a blurring process to use image-right-eye image. Specifically, performs a smoothing filter process of the unauthorized parallax region and one eye visible pixels 7 × the transfer source of the pixel regions, transcribed pixel regions in the transfer to the area 7, the distance information and subject region to the other region the blurring process on the basis of to do. 14, the distance 0-1, and the distance 5-6 incorrect parallax region is detected, the focus object at a distance 2,3,4 is a diagram showing an example of a blur intensity when detected. As shown in the figure, by characteristics such as dropped blur strength in a section of length 2, 3, 4 focused object is located, there is no blurring of the focused object image is obtained which does not impair the elite feeling It is.

According to the present embodiment as described above, it is possible to specify the pixel area (object area) in which a predetermined object objects appear. Then, by performing blurring processing on the basis of addition to the subject region in the incorrect parallax area and distance information, it does not become blurred image object to be focused is blurred uniformly in distance positional relationship, good visibility of the object as an image is blurred taste, it can be unrecognizable to a viewer unauthorized parallax like.
«Embodiment 5»
<5.1 Overview>
The image processing apparatus according to the fifth embodiment, similar to the image processing apparatus according to the fourth embodiment, an image processing apparatus for processing a plurality of images constituting a stereoscopic using binocular parallax, incorrect parallax region like by performing based blurring process, but is intended to reduce the visual fatigue, discomfort caused by incorrect parallax, with a high resolution converter, the input is low resolution image data other high-resolution image data converting the resolution image data. Then the image data after the high resolution conversion, image processing according to the first to fourth embodiments of the detection and blur processing of illegal disparity regions, one eye visible pixel area and distance information and subject area. Thus, for different resolution image data, it is possible to perform image processing according to the first to fourth embodiments, it is possible to reduce the visual fatigue, discomfort caused by incorrect parallax.
<5.2 Configuration>
First, description will be given of a configuration of an image processing apparatus according to the fifth embodiment. The configuration and the same part of the image processing apparatus 1200 according to the fourth embodiment shown in FIG. 12, not explained are denoted by the same reference numerals, different parts will be described. Figure 15 is a block diagram showing an example of a configuration of an image processing apparatus 1500. As shown in FIG. 15, the image processing apparatus 1500 includes an image data input terminal 1501, 1502, high resolution conversion unit 1503, the disparity detection unit 103, the distance information generating unit 1001, the object detection unit 1201, fraud parallax detection unit 104, composed interpolation image generation unit 701, the blurred image generating section 1504, 1505, include an image data output terminal 107 - 108.
<5.2.1 image data input terminal 1501 - 1502>
Image data input terminal 1501, one of the high resolution image data of the left-eye image, the right-eye image used for display of stereoscopic is input. Image data input terminal 1502 is a the inputted image into the image data input terminal 1501 images of different viewpoints, the image data of low resolution is input.
<5.2.2 high-resolution conversion unit 1503>
High resolution converter 1503 converts the low resolution image data input to the image data input terminal 1502, the high-resolution image data at the same resolution image data input to the image data input terminal 1501. The generated from the low resolution image high resolution image, using the nearest neighbor. That is, the high-resolution conversion by arranging closest pixel value interpolation points. It is also possible to generate a high-resolution image by other resolution conversion techniques such as bilinear interpolation. The low-resolution image data input to the image data input terminal 1502, by converting the high resolution image data with the same resolution image data of the input to the image data input terminal 1501, between the left-eye image, the right eye image in it is possible to perform the corresponding point search, it is possible to detect the parallax. Also it is possible to detect illegal parallax region from parallax. Further, it is possible to calculate the distance information from the parallax. Further, it is possible to identify a pixel block parallax information is not calculated, it is possible to detect the one eye visible pixel area. Furthermore, it is possible to detect the subject region by template matching.
<5.2.3 blurred image generation unit 1504 - 1505>
Blurred image generating unit 1504, 1505, the image data subjected to the high-resolution conversion at a high resolution image data and high-resolution conversion unit 1503 of the input to the image data input terminal 1501, illegal disparity regions, one eye visible pixel transfer source of the pixel region and transferring pixel regions in the transfer to the area, the distance information, the blurring process on the basis of the subject region performed.

According to this embodiment as described above, also, it is possible to detect illegal disparity regions, one eye visible pixel area and distance information and subject area as image data to be input is an image data of different resolutions, incorrect disparity regions, transcription source of the pixel region and transferring pixel regions in the transfer for one eye visible pixel area, distance information, by performing a blurring process on the basis of the subject region, due to incorrect parallax as an image with a so-called blurring it is possible to reduce the visual fatigue, discomfort. In particular, in a system including an image processing apparatus according to the present embodiment (e.g., a stereo camera), a portion of the plurality of cameras included in the system may be a low-performance camera can be reduced in cost.
«Embodiment 6»
<6.1 Overview>
The image processing apparatus according to the sixth embodiment, the image can be performed in the same manner as the image processing apparatus according to the fifth embodiment, the input image data having different resolutions, the image processing according to the first to fourth embodiments is a processing device, the implementation is different.

Converting the high resolution image data inputted to the image data of the other low-resolution image data at the same resolution, the image data after the low-resolution conversion, illegal disparity regions, one eye visible pixel area and distance information and subject the detection of the area detection. And each information incorrect parallax region, one eye visible pixel area and distance information and subject area detected for the low-resolution image data, after conversion to the information for the high-resolution image data that performs blurring processing, the information the blurring processing on the basis of the performed. Processing each process of detection of heavy illegal disparity regions, one eye visible pixel area and distance information and object area, it is possible to perform a low resolution image size, it is possible to reduce the processing.
<6.2 Configuration>
First, description will be given of a configuration of an image processing apparatus 1600 according to the sixth embodiment. The configuration and the same part of the image processing apparatus 1200 according to the fourth embodiment shown in FIG. 12, not explained are denoted by the same reference numerals, different parts will be described. Figure 16 is a block diagram showing an example of a configuration of an image processing apparatus 1600. As shown in FIG. 16, the image processing apparatus 1600 includes an image data input terminal 1501, 1502, the low-resolution conversion unit 1601, the disparity detection unit 103, the high-resolution conversion unit 1602, the distance information generating unit 1001, the object detection unit 1201, fraud parallax detection unit 104, the interpolation image generating unit 701, a conversion unit 1603, the blurred image generating section 1604, 1605, include an image data output terminal 107, 108 constructed.
<6.2.1 Low resolution conversion section 1601>
Low resolution conversion unit 1601 converts the high-resolution image data input to the image data input terminal 1501, the low-resolution image data at the same resolution image data input to the image data input terminal 1502. The generated from the high resolution image of the low resolution image, using the nearest neighbor. That is, the high-resolution conversion by arranging closest pixel value interpolation points. It is also possible to generate a high-resolution image by other resolution conversion techniques such as bilinear interpolation. The high-resolution image data input to the image data input terminal 1501, by converting the low-resolution image data at the same resolution image data of the input to the image data input terminal 1502, between the left-eye image, the right eye image in it is possible to perform the corresponding point search, it is possible to detect the parallax. Also it is possible to detect illegal parallax region from parallax. Further, it is possible to calculate the distance information from the parallax. Further, it is possible to identify a pixel block parallax information is not calculated, it is possible to detect the one eye visible pixel area. Furthermore, it is possible to detect the subject region by template matching. Processing each process of detection of heavy illegal disparity regions, one eye visible pixel area and distance information and object area, it is possible to perform a low resolution image size, it is possible to reduce the processing.
<6.2.2 High resolution converter 1602>
High resolution conversion unit 1602, the low-resolution image data input to the image input terminal 1502 is converted into high-resolution image data subjected to blurring processing. Specifically, into image data of high resolution image data and the same resolution. The generated from the low resolution image high resolution image, using the nearest neighbor. That is, the high-resolution conversion by arranging closest pixel value interpolation points. It is also possible to generate a high-resolution image by other resolution conversion techniques such as bilinear interpolation.
<6.2.3 conversion unit 1603>
Conversion unit 1603, each information of the high resolution unauthorized disparity regions, one eye visible pixel area and distance information and subject area of ​​the low-resolution image data input to the image data and the image input terminal 1502 of performing the low-resolution conversion is converted into the information in the incorrect parallax region, one eye visible pixel area and distance information and subject area of ​​the low-resolution image data subjected to the high resolution image data and high-resolution conversion of input to the image input terminal 1501. Specifically, the interpolated pixel at the time of the resolution conversion to the corresponding pixel, high resolution of each information incorrect parallax region, one eye visible pixel area and distance information and subject region corresponding pixels of the low resolution image data has pixel of the image data is invalid disparity region, one eye visible pixel area and distance information and subject area having. Then, with respect to one eye visible pixel area where it was converted to transfer the other image area on the same image coordinate range and one eye visible pixel area.
<6.2.4 blurred image generation unit 1604 - 1605>
Blurred image generating unit 1604, 1605, a high-resolution image data and high-resolution conversion of input to the image input terminal 1501 with respect to the low-resolution image data was performed, illegal parallax region-eye converted by the conversion unit 1603 performing based blurring process on the visible pixel area and distance information and subject the information of the region.

This concludes the description of the configuration of the image processing apparatus 1600. Next, the operation of the image processing apparatus 1600 will be described.
<6.3 Operation>
Figure 17 is a flowchart showing an operation of the image processing apparatus 1600. As shown in this figure, the low-resolution conversion unit 1601 First, the high-resolution input image data input to the image data input terminal 1501, is input to the image data input terminal 1502 are low-resolution input image data and the resolution It is converted into image data (step S1701). Then, the high-resolution input image data and the low resolution input image data subjected to the low-resolution conversion, to detect the respective information incorrect parallax region, one eye visible pixel area and distance information and object area (Step S1702). The details of each processing of the unauthorized disparity regions, one eye visible pixel area and distance information and object area detected, so described in the first to fourth embodiments, the description thereof is omitted here. Then, the high-resolution conversion unit 1602 converts the low resolution input image data input to the image data input terminal 1502, which is input to the image data input terminal 1501 to the low-resolution input image data with the same resolution image data (step S1703). Conversion unit 1603, the information on the high-resolution input image data of each information incorrect parallax region, one eye visible pixel area and distance information and subject area of ​​the high-resolution input image data subjected to the detected low-resolution conversion at step S1702 to convert the incorrect parallax region & monocular each information of the visible pixel area and distance information and subject area of ​​the low resolution input image data to each information of the low resolution input image data subjected to the high-resolution conversion (step S1704). Specifically, the interpolated pixel at the time of the resolution conversion to the corresponding pixel, high resolution of each information incorrect parallax region, one eye visible pixel area and distance information and subject region corresponding pixels of the low resolution image data has pixel of the image data is invalid disparity region, one eye visible pixel area and distance information and subject area having. The conversion unit 1603, in step S1704, with respect to one eye visible pixel area where it was converted to transfer the other image area on the same image coordinate range and one eye visible pixel area. (Step S1705). After the above processing, the blurred image generating section 1604, 1605 performs the blurring process on the basis of the information of the illegal disparity regions, one eye visible pixel area and distance information and object area converted by the converting unit 1603 performs blurring processing ( step S1706). Specifically, the blurred image generating unit 1604, the high-resolution input image data, a blurring process on the basis of the converted incorrect parallax region, one eye visible pixel area and distance information and subject the information of the area in the conversion unit 1603 performed, the blurred image generating unit 1605, the high-resolution transformed low resolution input image data, blurring based on the converted incorrect parallax region, one eye visible pixel area and distance information and subject the information of the area in the conversion unit 1603 processing is carried out. This concludes the description of the operation of the image processing apparatus 700.

According to this embodiment as described above, since the processing is the processing of detecting heavy illegal disparity regions, one eye visible pixel area and distance information and subject area can be performed at a low resolution image size, implementation the image processing according to embodiments 1 to 4 can be executed at high speed. Further, it is possible to perform processing at a processor of low performance.
«Embodiment 7»
<7.1 Overview>
The image processing apparatus according to the seventh embodiment, similarly to the image processing apparatus according to the first to fourth embodiments, against unauthorized disparity regions, by performing blurring processing, visual fatigue, discomfort caused by incorrect parallax but it is intended to reduce, but differs in the input image data is one. Receiving one input image and the distance information corresponding to the input image, to detect illegal parallax based on the parallax calculated from the distance information, it performs blurring processing on the parallax image generated from the input image and distance information. The detection of the subject region is performed on the parallax image generated from the input image and distance information. Thus, it is possible to perform blurring processing based on the incorrect parallax area and distance information and object area, from the distance information corresponding to the input image as one of the input image.
<7.2 Configuration>
First, description will be given of a configuration of an image processing apparatus 1800 according to the seventh embodiment. Figure 18 is a block diagram showing an example of a configuration of an image processing apparatus 1800. The configuration and the same part of the image processing apparatus 1200 according to the fourth embodiment shown in FIG. 12, not explained are denoted by the same reference numerals, different parts will be described. As shown in FIG. 18, the image processing apparatus 1800 includes an image data input terminal 1801, the distance information input terminal 1802, the disparity calculation unit 1803, a parallax image generator 1804, fraud parallax detection unit 104, subject detection unit 1201, the blurred image generator 1805, 1806, and include an image data output terminal 107 - 108.
<7.2.1 Image data input terminal 1801, the distance information input terminal 1802>
Image data input terminal 1801, one two-dimensional image data is input. Distance information input terminal 1802, the distance information for each pixel block of the image data input to the image data input terminal 1801 is input. Here the distance information and refers to the distance from the imaging position of the image data to an object appearing in the pixel block. Incidentally distance information may be acquired by TOF (Time Of Flight) type distance range sensor such as a sensor. Further, the distance information from the focus position of the lens may be obtained when the camera is in an autofocus operation. The distance information, an external recording device, a broadcast wave, may be acquired through a network or the like.
<7.2.2 parallax calculation unit 1803>
Parallax calculating unit 1803, based on the distance information inputted from the distance information input terminal 1802 and calculates the length to be the horizontal shift Upon parallax image created for each pixel block (disparity). Parallax can be calculated by using the distance information and parallax relationship described with reference to FIG.
<7.2.4 parallax image generating unit 1804>
Parallax image generator 1804, by shifting each pixel block only in the horizontal direction length of the calculated parallax disparity calculation unit 1803 to generate a parallax image.
<7.2.5 blurred image generation unit 1805 - 1806>
Blurred image generating unit 1805, the image data inputted from image data input terminal 1801, based on the incorrect parallax area and distance information and subject area, performs the blurring process. Blurred image generating unit 1806, to the disparity image generated by the parallax image generator 1804, based on the incorrect parallax area and distance information and subject area, performs the blurring process. This concludes the description of the configuration of the image processing apparatus 1800. Next, the operation of the image processing apparatus 1800 will be described.
<7.3 Operation>
Figure 19 is a flowchart showing an operation of the image processing apparatus 1800. As shown in this figure, the disparity calculation unit 1803 calculates the parallax based on the distance information inputted into the distance information input terminal 1802 (step S1901). Then parallax image generator 1804 generates a parallax image on the basis of the parallax calculated by the parallax calculating unit 1803 (step S1902). The unauthorized parallax region with the incorrect parallax detection unit 104 detects each input image data and parallax image object area by the object detection unit 1201 (Step S1903). After the above processing, the blurred image generating unit 1805 performs blurring on the basis of incorrect parallax area and distance information and subject area processing (step S1904).

According to this embodiment as described above, from the distance information corresponding to the image data and one two-dimensional image data, to create a stereoscopic image, performs blurring processing based on the incorrect parallax area and distance information and subject area it can be, it is possible to reduce the visual fatigue, discomfort caused by the illegal parallax.
«Embodiment 8»
<8.1 Overview>
The image processing apparatus according to the eighth embodiment is different in that it does not accept the distance information and the image processing apparatus according to the seventh embodiment. Distance information is extracted from one image data input. Accordingly, the blurring processing based on the incorrect parallax area and distance information and subject area, can be carried out from a single input image.
<8.2 Configuration>
Figure 20 is a block diagram showing an example of a configuration of an image processing apparatus 2000. The configuration and the same part of the image processing apparatus 1800 according to the seventh embodiment shown in FIG. 18, not explained are denoted by the same reference numerals, different parts will be described. As shown in FIG. 20, the image processing apparatus 2000, the image data input terminal 1801, the distance information extraction unit 2001, the disparity calculation unit 1803, fraud parallax detection unit 104, a parallax image generator 1804, the subject detection unit 1201, the blurred image generator 1805, 1806, and include an image data output terminal 107 - 108.

The following describes the distance information extraction unit 2001. The distance information extraction unit 2001, from the two-dimensional image data inputted from image data input terminal 1801, and extracts the distance information. Specifically, using the technique shown in Non-Patent Document 2. That is, the color is first called the image a "super pixel", divided into attribute very homogeneous set of pixels, such as brightness, as compared to the super pixels adjacent to this super pixel, by analyzing the changes such as texture gradient , to estimate the distance from the observer.

According to this embodiment as described above, from one two-dimensional image data, to create a stereoscopic image, it is possible to carry out blurring processing based on the incorrect parallax area and distance information and subject area, due to incorrect parallax it is possible to reduce the visual fatigue, discomfort.
«Embodiment 9»
<9.1 Overview>
Embodiment 9 shows an embodiment for an imaging apparatus including any one of the image processing apparatus from the first embodiment up to 8. Imaging device includes selecting and changing the range of disparity recognized as incorrect parallax by a camera and a user operation, selection and change of the target object, the user interface that can be selected or changed blur intensity based on distance information. To any of the image subjected to the blurring process by the image processing apparatus from the first embodiment up to 8, the user selects and changes the range of disparity recognized as incorrect parallax of the target object selection and change, based on the distance information by making a selection and change of the blur strength, it is possible to generate an image desired blur is added.
<9.2 Configuration>
First, it will be described using the form of an imaging apparatus according to the present embodiment. FIG, 22 is a diagram showing an example of an imaging apparatus according to the present embodiment. Figure 21 is a digital still camera, Figure 22 is a mobile terminal device, FIG. 23 shows a stereo camera connected to a television. Imaging apparatus shown in the figures, the input image from the photographed images or external by the camera performs blurring processing by any of the image processing apparatus from the first embodiment up to 8, after the image processing to display the image on the display. Then, the touch panel accepts a user operation by the remote controller or the like, and selecting and changing the range of disparity recognized as illegal parallax, selection and change of the target object, the selection and change of the blurring intensity based on distance information. Based on user operation, blurred image displayed on the display is updated. Then specifically describes the structure of the imaging device.

Figure 24 is a block diagram showing an exemplary configuration of an imaging apparatus 2400. As shown in FIG. 24, the imaging apparatus 2400, the image signal processing unit 2401, control unit 2402, a camera portion 2403, 2404, the image decoding unit 2405, 2406, the image encoding unit 2407, 2408, the image recording unit 2409, data transmission unit 2410,3D / 2D display unit 2411, a touch panel 2412, and includes a vibration unit 2413.
<9.2.1 image signal processing unit 2101>
The image signal processing section 2001 is provided with any of the image processing apparatus according to the first to eighth embodiments, performing blurring processing on the captured image data and the like in the camera. Note that fraud parallax area, distance information, the detection of one eye visible pixel area, the detection of the subject region may be processed in parallel. By performing processing in parallel, it can be performed at high speed of blurring.
<9.2.2 control unit 2102>
Control unit 2102 controls the operation of each component of the imaging apparatus 2100.
<9.2.3 camera unit 2003 - 2004>
Camera unit 2003, 2004 is provided with an image sensor, for imaging the left-eye image, the right-eye image.
<9.2.4 image decoding unit 2005 - 2006>
The image decoding unit 2005, 2006, JPEG, MPEG, H. 264, decodes the compressed image data such as FLASH video.
<9.2.5 image encoding unit 2007 - 2008>
The image coding unit 2007, 2008, JPEG, MPEG, H. 264, performs compression of FLASH video to such.
<9.2.6 image recording section 2409>
The image recording unit 2409 stores the image data processed by the image data the image signal processing section 2401 taken by the camera unit 2403, 2404, memory card, HDD, the recording medium such as an optical disk.
<9.2.7 data transmission section 2410>
Data transmission unit 2410, the Internet, computer data communication network such as an intranet, or a network transceiver corresponding to the mobile phone communication network or the like, a tuner for receiving broadcast waves, image data an image signal captured by a camera unit 2403, 2404 transmitting image data obtained by image processing by the processing unit 2401.
<9.2.8 3D / 2D display section 2411>
3D / 2D display unit 2411, image processing and image liquid crystal display with the image data the image signal processing section 2401 taken by the camera unit 2403, 2404, a plasma display, and displays using organic EL or the like.
<9.2.9 touch panel 2412>
The touch panel 2412, select or change the range of disparity recognized as illegal parallax by a user, selection and change of the target object, the selection and change operation of the blur strength based on distance information accepted. Specifically will be described in <9.3> section image displayed in 3D / 2D display unit 2411, while viewing the rendered menu screen control unit 2401, intuitively, imaging via the touch panel 2412 it is possible to perform an operation instruction to the apparatus 2400.
<9.2.10 vibrating section 2413>
Vibrating unit 2413, in response to a user operation through the touch panel 2412, and performs the appropriate vibration operation. In addition to the intuitive operation by screen, use the response from the clear image signal processing system according to sensory perception by the vibration user can sense.

This concludes the description of the configuration of the imaging apparatus 2400. Then, select or change the range of disparity recognized as illegal parallax by a user, selection and change of the target object, selecting and changing operation of the blur strength based on distance information will be described.
<9.3 user operation and processing>
Figure 25 is selected and changed in the range of disparity recognized as illegal parallax by the user, is a diagram showing an example of a display screen in selecting and changing operation of the blur strength based on the selection and change, distance information of the target object. As shown in FIG. 25, by touching a part of the "Boke", blurring processing is carried out. First, the display screen, blurred image blur processing simple has been performed is displayed. Here, the simple blur processing, an image center and the focus position, which adds the strength of blur corresponding to the length from the center of the image can be processed at high speed. (A) Simple blur processing and display, (b) incorrect parallax region detection, (c) the distance information calculation, (d) each processing object area detection performed in parallel, blurred image every time each information is detected to update. Distance information calculation processing after obtains change-determining operation of the blur processing based on the distance information to the user, the user can, for example <9.3.3> Change-determination of the blur processing based on the distance information by a user operation which will be described later in the section it can be carried out. Distance if it is changed and determined in blurring processing based on the information, updates the blurred image on the screen. Further, after the object area detection process, prompts the user to change or decision operation of the target object, the user can make changes, determination of the object region by a user operation to be described later, for example, <9.3.1> sections. If the change-determining the object region is performed, it updates the blurred image on the screen.

Thus (a) Simple blur processing and display, based on (b) incorrect parallax area detection performed (c) the distance information calculation, the processes of (d) the subject region detected in parallel, ends the information processing by performing sequential updating user manipulation of blurred images, it can be displayed blurred image to the user in real time.
<9.3.1 subject area change and deciding operation>
It will now be described subject area change-decision operation. In Figure 25, focused subject detected by the subject detection unit is displayed in boxes. Here by touching an object, or set focused object, it is possible to release the focused object. Further, by the work with the vibration unit 2413, when the user around the detected subject is touched, the vibration unit 2413 is activated, the user may be intuitively sense the subject by vibration. If the interest object is changed, again blurring processing on the basis of the changed object region is performed. Thus, the user selects and releases the focused object, by performing again blurring processing on the basis of the changed object area, it is possible to obtain an image of good visibility of the focused object.
<9.3.2 unauthorized parallax range change and deciding operation>
Next, a description will be given of change and deciding operation of illegal parallax range. In Figure 25, the "Option" portion, it is possible to determine the range of disparity recognized as incorrect parallax. Figure 26 is a diagram showing the safety level of the change screen. For example, providing a safe level of three stages as shown in the figure. Each safety levels, are determined in the range of disparity recognized as illegal parallax, the user can further select a safe level, it is possible to change the range of fraud parallax. For example, the safety level 1, defined to certify parallax corresponding to the above parallax angle 1.0 - -1.0 or less illegal parallax, safety level 2, or more parallax angle 1.2 - -1.2 below determining the corresponding disparity to be recognized as illegal parallax. If the safety level is changed, it is performed again to detect illegal disparity regions, blurring processing is performed on the modified illegally disparity regions.

Further, the display size envisaged Upon display of the stereoscopic image, may be provided an interface that may be entered by the user. Based on the entered here display size, and updates the pixels constituting the unauthorized parallax, again to detect illegal parallax area.
<Blurring process change and deciding operation based on the 9.3.3 distance information>
Then, blurring process change-determining operation based on the distance information will be described. Figure 27 is a diagram for explaining the blurring process changes, determination operation based on the distance information. Can be changed by touching the portion of the "DEPTH", the blurring processing based on the distance information. Specifically, it is possible to change the position of the distance which is not subjected positions of the vertices of the blur strength curve, i.e. the blurring process by the distance information illustrated in Figure 11. By sliding the scroll bar displayed on the right side of the "DEPTH", you can change the position of the apex of the blurring intensity curve. 28 to 35, by sliding the scroll bar, a diagram displays a blurring process changes. Indicating an area where the display portion by the hollow not subjected to blurring processing by the distance information. That is, the blurring processing is performed around the displayed object portion by an outline. Thus, by sliding the scroll bar, can change the position of the vertex of the blurring intensity curve, sequential blur processed image is displayed. Figure 36 is a diagram showing a blurring intensity curve is changed by the user operation. Be stopped scrolling at the portion of distance 3, is added blur shown in blurring intensity curve 3501, be stopped scrolling in part of the distance 2, it is added blur shown in blurring intensity curve 3502.

This concludes the description of the user operation. Thus, the user can select or change the blurring intensity based on selection and change, select or change of interest subject, distance information ranging disparity recognized as incorrect parallax by each operation. Then, based on each information selected and changed by the each operation, by performing the blurring process again, it is possible to generate an image that the user desired blurring processing is performed.
«Supplement»
Incidentally, have been described based on the above embodiments, the present invention is of course not limited to the above embodiments. The following cases are also included in the present invention.
(A) The present invention may be the application executing method processing procedures described in the embodiments is disclosed. The present invention may be a computer program comprising program code for operating a computer in the processing procedure.
(B) The present invention may be implemented as an LSI for controlling the application executing device. Figure 36 shows an example embodying the image processing apparatus according to the present invention in LSI. As shown in the figure for example, CPU, DSP, ENC / DEC (encoder / decoder), AIF (audio interface), VIF (video interface), PERI (control module for controlling the peripheral equipment), NIF (Network Interface) , MIF (memory interface), configured to include a RAM / ROM. Processing steps described in the embodiments is stored in RAM / ROM as a program code, executed by the CPU or DSP.

Here, although the LSI, depending on differences in integration, IC, system LSI, super LSI, referred to as ultra LSI.

Further, the method of circuit integration is not limited to LSI, a dedicated circuit or may be implemented with a general purpose processor. Which can be programmed after LSI manufacturing FPGA (Field Programmable Gate Array) or may utilize reconfigurable reconfigurable processor where connections and settings of circuit cells in the LSI.

Furthermore, when a technology for the integrated circuit replacing LSI is developed to progressive or derivative semiconductor technology, it is also possible to integrate the functional blocks and components using that technology. Such techniques, likely possibly apply of biotechnology.
(C) In the first to ninth embodiments, an example has been described for performing image processing on the two input image data, the input image data may be three or more multi-view image. Even for three or more input images, it is possible to perform the same image processing.
(D) In ​​the first to ninth embodiments, although the detection of the disparity was performed in pixel block units of 16 × 16 pixels, the detection of the disparity in other pixel blocks (e.g. 32 × 32 pixels, 8 × 8 pixels) it may be carried out. Also it may detect disparity in units of pixels.

According to the image processing apparatus according to the present invention, it identifies the risk pixel region having a parallax (illegal disparity) that cause visual fatigue, discomfort, stereoscopic sickness, and performs blurring processing on the pixel area to the observer Accordingly, as an image is a so-called blur, so unrecognizable to the observer unauthorized parallax is beneficial can reduce the visual fatigue, discomfort caused by incorrect parallax.

100,600,900,1200,1500,1600,1800,2000, the image processing apparatus 101,102,1501,1502,1801 image data input terminal 103 parallax detection unit 104 fraud parallax detection unit 105,106,602,603,902 , 903,1202,1203,1504,1505,1604,1605,1805,1806 blurred image generating unit 107 and the image data output terminal 601 interpolation image generation unit 901 distance information generating unit 1201 subject detection unit,
1301 template image 1302 input image data 1303 subject 1503,1602 High resolution converter 1601 low-resolution conversion unit 1603 converting unit 1802 distance information input terminal 1803 parallax calculating unit 1804 parallax image generator 2001 distance information extraction unit 2400 imaging apparatus 2401 image signal processing part 2402 control unit 2403 and 2404 a camera portion 2405 and 2406 the image decoding unit 2407,2408 image coding unit 2409 image recording unit 2410 data transmission unit 2411 3D / 2D display unit 2412 touch panel 2413 vibrating unit 3501,3502,3503 blur intensity curve

Claims (23)

  1. An image processing apparatus that performs image processing on a plurality of image data used for displaying three-dimensional images,
    And accepting means for accepting input of the plurality of image data,
    For each image data, and disparity detection unit for detecting the disparity with the other image data,
    Among the plurality of pixel regions in the pair of images subjected to parallax detection, detection means for detecting what parallax becomes invalid as illegal disparity region,
    The image processing apparatus characterized by comprising a processing means for performing a blurring process with respect to each pixel region constituting the unauthorized parallax area.
  2. Said detection means compares the parallax with a predetermined threshold value, according to the comparison result, the image processing apparatus according to claim 1, characterized in that detecting fraudulent parallax area.
  3. Was present at the end of the image data of one of said plurality of pixel data, due to the absence of the other image data, it could cause such a flickering caused by eye visible in stereoscopic, present only in the image data of the one If the pixel regions was defined as one eye visible pixel area,
    The image processing apparatus further includes
    The image data of the other image data side to be subjected to stereoscopic shared with the pixel region adjacent to the piece eye visible pixel area, overwriting the piece eye visible image area, the interpolation image generating unit that performs interpolation of one eye visible provided,
    It said processing means further
    Pixel area is overwritten for monocular visual interpolation and, according to claim 2, characterized in that the pixel data used for overwriting to process blurring both image area of ​​the other image data present image processing apparatus.
  4. The image processing apparatus further includes
    Based on the parallax, comprising a distance information generation unit which generates the distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region,
    It said processing means further
    Wherein for a pixel area excluding the illegal parallax area, the image processing apparatus according to claim 2, characterized by applying blurring processing based on the distance information.
  5. The image processing apparatus further includes
    Based on the parallax, the distance information generating unit that generates the distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region,
    Includes a subject detection unit for detecting an object pixel area of ​​a pixel region constituting the predetermined object,
    It said processing means further
    The against unauthorized parallax region and the pixel region except for the subject pixel area, the image processing apparatus according to claim 2, characterized by applying blurring processing based on the distance information.
  6. A receiving means for receiving an input of one of the image data,
    For each pixel area of ​​the image data, a parallax calculating section which calculates the parallax on the basis of the distance information indicating the distance from the imaging position of the image data to an object appearing in the pixel region,
    And a stereo image generating unit that generates a stereo image consisting of the left eye image and the right eye image based on the parallax between the image data,
    Among the plurality of pixel regions in the left eye image and the right-eye image, and detecting means for detecting what parallax becomes invalid as illegal disparity region,
    The image processing apparatus characterized by comprising a processing means for performing a blurring process with respect to each pixel region constituting the unauthorized parallax area.
  7. Said detection means compares the parallax with a predetermined threshold value, according to the comparison result, the image processing apparatus according to claim 6, characterized in that detecting fraudulent parallax area.
  8. It said processing means further
    The image processing apparatus according to claim 7, characterized in that for the pixel area excluding the illegal parallax region in the left-eye image and the right-eye image, subjected to blurring processing based on the distance information.
  9. The image processing apparatus further includes
    Includes a subject detection unit for detecting an object pixel area of ​​a pixel region constituting the predetermined subject in the left-eye image and the right-eye image,
    It said processing means further
    The image processing apparatus according to claim 7, characterized in that for the pixel area excluding the illegal parallax region and the object pixel regions in the left eye image and the right-eye image, subjected to blurring processing based on the distance information.
  10. The image processing apparatus further includes
    The image processing apparatus according to claim 6, characterized in that it comprises a distance information calculation unit for calculating the distance information from the one image data.
  11. An image processing apparatus that performs image processing on a plurality of image data used for displaying three-dimensional images,
    Wherein the plurality of image data includes image data of at least one high resolution image data and low resolution,
    And accepting means for accepting input of the plurality of image data,
    A resolution converter for converting the low-resolution image data, the image data of the high resolution image data of the same resolution,
    And disparity detection unit for detecting the parallax between the image data subjected to image data and resolution conversion of the high resolution,
    Among the plurality of pixel regions in the image data subjected to the image data and the resolution conversion of the high resolution, the detection means for detecting what parallax becomes invalid as illegal disparity region,
    The image processing apparatus characterized in that it comprises a processing means for performing a blurring process with respect to each pixel region constituting the unauthorized parallax area in the image data of the low resolution of performing the image data and the resolution conversion of the high resolution.
  12. Said detection means compares the parallax with a predetermined threshold value, according to the comparison result, the image processing apparatus according to claim 11, characterized in that detecting fraudulent parallax area.
  13. Was present at the end of the image data of one of said plurality of pixel data, due to the absence of the other image data, it could cause such a flickering caused by eye visible in stereoscopic, present only in the image data of the one If the pixel regions was defined as one eye visible pixel area,
    The image processing apparatus further includes
    The image data subjected to the image data or the resolution conversion of the other of the high resolution to be subjected to stereoscopic shared with the pixel region adjacent to the piece eye visible pixel area, were the high-resolution image data or the resolution conversion of the image overwriting the piece eye visible image area data includes an interpolation image generating unit that performs interpolation of one eye visible,
    Processing apparatus according to claim 12, wherein the performing blur processing for both other pixel region is overwritten on the image data and overwrite the original pixel area for one eye visible interpolation.
  14. The image processing apparatus further includes
    Based on the parallax, relative to the high resolution image data and the image data resolution conversion was carried out, the distance information for generating distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region equipped with a generator,
    It said processing means further
    The image processing apparatus according to claim 12, characterized in that for the pixel area excluding the illegal parallax region in the image data subjected to the image data and the resolution conversion of the high resolution, subjected to blurring processing based on the distance information .
  15. The image processing apparatus further includes
    Based on the parallax, relative to the high resolution image data and the image data resolution conversion was carried out, the distance information for generating distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region a generation unit,
    Includes a subject detection unit for detecting an object pixel area of ​​a pixel region constituting the predetermined object in the image data subjected to the image data and the resolution conversion of the high resolution,
    It said processing means further
    With respect to the pixel area excluding the illegal parallax region and the object pixel region in the image data subjected to the image data and the resolution conversion of the high resolution, to claim 12, characterized in that performing blurring processing based on the distance information the image processing apparatus according.
  16. An image processing apparatus that performs image processing on a plurality of image data used for displaying three-dimensional images,
    Wherein the plurality of image data includes image data of at least one high resolution image data and low resolution,
    And accepting means for accepting input of the plurality of image data,
    The image data of the high resolution, low resolution converter for converting the low resolution image data and the resolution image data of,
    And disparity detection unit for detecting the disparity between the low-resolution image data with the image data subjected to low resolution conversion,
    Among the plurality of pixel areas in the image data subjected to low resolution conversion the low-resolution image data, detecting means for detecting what parallax becomes invalid as illegal disparity region,
    The image data of the low resolution, high resolution converter for converting the high resolution image data with the same resolution image data of,
    Incorrect parallax area of ​​the image data subjected to low resolution conversion to an incorrect parallax area of ​​the image data of the high resolution, converts the incorrect parallax area of ​​the image data of the low resolution incorrect parallax area of ​​the image data subjected to high-resolution conversion and fraud parallax domain conversion unit for,
    The image processing apparatus characterized in that it comprises a processing means for performing a blurring process with respect to each pixel region constituting the unauthorized parallax area in the image data subjected to the image data and the high resolution conversion of the high resolution.
  17. Said detection means compares the parallax with a predetermined threshold value, according to the comparison result, the image processing apparatus according to claim 16, characterized in that detecting fraudulent parallax area.
  18. Was present at the end of the image data of one of said plurality of pixel data, due to the absence of the other image data, it could cause such a flickering caused by eye visible in stereoscopic, present only in the image data of the one If the pixel regions was defined as one eye visible pixel area,
    The image processing apparatus further includes
    Piece of converting the image data subjected to the low-resolution conversion of one eye visible pixel region between the low-resolution image data, one eye visible pixel area among the image data subjected to the high resolution image data and high-resolution conversion of and eyes visible pixel domain converter,
    The other image data to be subjected to stereoscopic shared with the pixel region adjacent to one eye visible pixel region after the conversion, and overwrite one eye visible pixel region after the conversion, the interpolation image generating unit that performs interpolation of one eye visible provided,
    It said processing means further
    Processing apparatus according to claim 17, wherein the performing blur processing for both other pixel region is overwritten on the image data and overwrite the original pixel area for one eye visible interpolation.
  19. The image processing apparatus further includes
    Based on the parallax, the distance to generate for the image data subjected to the low-resolution conversion the low-resolution image data, a distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region and information generating unit,
    Distance information of the image data subjected to low resolution conversion on the distance information of the image data of the high resolution, the distance information conversion for converting the distance information of the image data of the low resolution of the distance information of the image data subjected to high-resolution conversion and a part,
    It said processing means further
    The image processing according to claim 17 for the pixel region excluding the illegal parallax region in the image data subjected to the image data and the high resolution conversion of the high resolution, and characterized by applying blurring processing based on the distance information apparatus.
  20. The image processing apparatus further includes
    Based on the parallax, the distance to generate for the image data subjected to the low-resolution conversion the low-resolution image data, a distance information indicating the distance from the imaging position of the image data for each pixel region to an object appearing in the pixel region and information generating unit,
    A subject detection unit for detecting an object pixel area of ​​a pixel region constituting the predetermined subject in the low-resolution image data with the image data subjected to low resolution conversion,
    Distance information of the image data subjected to low resolution conversion on the distance information of the image data of the high resolution, the distance information conversion for converting the distance information of the image data of the low resolution of the distance information of the image data subjected to high-resolution conversion and parts,
    The object region of the image data subjected to low resolution conversion to the subject area of ​​the image data of the high resolution, the object area conversion for converting the subject area of ​​the low-resolution image data of the object region of the image data subjected to high-resolution conversion and a part,
    It said processing means further
    With respect to the pixel area excluding the illegal parallax region and the object pixel region in the image data subjected to the image data and the high resolution conversion of the high resolution, blurring processing based on the distance information to claim 17, characterized by applying the image processing apparatus according.
  21. An image processing apparatus according to claim 1, an image pickup apparatus having an image sensor.
  22. An image processing method for performing image processing on a plurality of image data used for displaying three-dimensional images,
    A receiving step of receiving an input of the plurality of image data,
    For each image data, and disparity detection step of detecting a disparity with other image data,
    Among the plurality of pixel regions in the pair of images subjected to parallax detection, a detection step of detecting what parallax becomes invalid as illegal disparity region,
    Image processing method characterized by comprising a processing step of performing a blurring process with respect to each pixel region constituting the unauthorized parallax area.
  23. A program for executing the image processing in the computer for a plurality of image data used for displaying three-dimensional images,
    A receiving step of receiving an input of the plurality of image data,
    For each image data, and disparity detection step of detecting a disparity with other image data,
    Among the plurality of pixel regions in the pair of images subjected to parallax detection, a detection step of detecting what parallax becomes invalid as illegal disparity region,
    Program for executing a processing step of performing a blurring process with respect to each pixel region constituting the unauthorized disparity regions the computer.
PCT/JP2011/006380 2010-12-24 2011-11-16 Image processing apparatus, image pickup apparatus, image processing method, and program WO2012086120A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2010287821 2010-12-24
JP2010-287821 2010-12-24

Publications (1)

Publication Number Publication Date
WO2012086120A1 true true WO2012086120A1 (en) 2012-06-28

Family

ID=46313415

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/006380 WO2012086120A1 (en) 2010-12-24 2011-11-16 Image processing apparatus, image pickup apparatus, image processing method, and program

Country Status (1)

Country Link
WO (1) WO2012086120A1 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014014076A (en) * 2012-07-03 2014-01-23 Woodman Labs Inc Image blur based on 3d depth information
US9787862B1 (en) 2016-01-19 2017-10-10 Gopro, Inc. Apparatus and methods for generating content proxy
US9792502B2 (en) 2014-07-23 2017-10-17 Gopro, Inc. Generating video summaries for a video using video summary templates
US9838730B1 (en) 2016-04-07 2017-12-05 Gopro, Inc. Systems and methods for audio track selection in video editing
US9871994B1 (en) 2016-01-19 2018-01-16 Gopro, Inc. Apparatus and methods for providing content context using session metadata
US9916863B1 (en) 2017-02-24 2018-03-13 Gopro, Inc. Systems and methods for editing videos based on shakiness measures
US9922682B1 (en) 2016-06-15 2018-03-20 Gopro, Inc. Systems and methods for organizing video files
US9953224B1 (en) 2016-08-23 2018-04-24 Gopro, Inc. Systems and methods for generating a video summary
US9953679B1 (en) 2016-05-24 2018-04-24 Gopro, Inc. Systems and methods for generating a time lapse video
US9967515B1 (en) 2016-06-15 2018-05-08 Gopro, Inc. Systems and methods for bidirectional speed ramping
US9972066B1 (en) 2016-03-16 2018-05-15 Gopro, Inc. Systems and methods for providing variable image projection for spherical visual content

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06194602A (en) * 1992-12-24 1994-07-15 Nippon Telegr & Teleph Corp <Ntt> Binocular stereoscopic viewing device
JPH10134187A (en) * 1996-10-31 1998-05-22 Nec Corp Three-dimensional structure estimating device
JP2010171628A (en) * 2009-01-21 2010-08-05 Nikon Corp Image processing device, program, image processing method, recording method, and recording medium
WO2012001970A1 (en) * 2010-06-30 2012-01-05 富士フイルム株式会社 Image processing device, method, and program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06194602A (en) * 1992-12-24 1994-07-15 Nippon Telegr & Teleph Corp <Ntt> Binocular stereoscopic viewing device
JPH10134187A (en) * 1996-10-31 1998-05-22 Nec Corp Three-dimensional structure estimating device
JP2010171628A (en) * 2009-01-21 2010-08-05 Nikon Corp Image processing device, program, image processing method, recording method, and recording medium
WO2012001970A1 (en) * 2010-06-30 2012-01-05 富士フイルム株式会社 Image processing device, method, and program

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014014076A (en) * 2012-07-03 2014-01-23 Woodman Labs Inc Image blur based on 3d depth information
US9185387B2 (en) 2012-07-03 2015-11-10 Gopro, Inc. Image blur based on 3D depth information
US9792502B2 (en) 2014-07-23 2017-10-17 Gopro, Inc. Generating video summaries for a video using video summary templates
US9787862B1 (en) 2016-01-19 2017-10-10 Gopro, Inc. Apparatus and methods for generating content proxy
US9871994B1 (en) 2016-01-19 2018-01-16 Gopro, Inc. Apparatus and methods for providing content context using session metadata
US9972066B1 (en) 2016-03-16 2018-05-15 Gopro, Inc. Systems and methods for providing variable image projection for spherical visual content
US9838730B1 (en) 2016-04-07 2017-12-05 Gopro, Inc. Systems and methods for audio track selection in video editing
US9953679B1 (en) 2016-05-24 2018-04-24 Gopro, Inc. Systems and methods for generating a time lapse video
US9922682B1 (en) 2016-06-15 2018-03-20 Gopro, Inc. Systems and methods for organizing video files
US9967515B1 (en) 2016-06-15 2018-05-08 Gopro, Inc. Systems and methods for bidirectional speed ramping
US9953224B1 (en) 2016-08-23 2018-04-24 Gopro, Inc. Systems and methods for generating a video summary
US9916863B1 (en) 2017-02-24 2018-03-13 Gopro, Inc. Systems and methods for editing videos based on shakiness measures

Similar Documents

Publication Publication Date Title
US20080240549A1 (en) Method and apparatus for controlling dynamic depth of stereo-view or multi-view sequence images
Yamanoue et al. Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images
US20100231689A1 (en) Efficient encoding of multiple views
US20110298898A1 (en) Three dimensional image generating system and method accomodating multi-view imaging
US20090237494A1 (en) Apparatus, method, and program for displaying stereoscopic images
US20120038745A1 (en) 2D to 3D User Interface Content Data Conversion
US20110126159A1 (en) Gui providing method, and display apparatus and 3d image providing system using the same
WO2007116549A1 (en) Image processing device
US20120154551A1 (en) Stereo image display system, stereo imaging apparatus and stereo display apparatus
EP2393298A1 (en) Method and apparatus for generating multiple image views for a multiview autostereoscopic display device
US20130101263A1 (en) Image capture device, player, system, and image processing method
JP2011188004A (en) Three-dimensional video imaging device, three-dimensional video image processing apparatus and three-dimensional video imaging method
US20110090323A1 (en) Image processing apparatus, image processing method, and program
US20110199459A1 (en) Method and system for processing an input three dimensional video signal
US20140333739A1 (en) 3d image display device and method
CN1956555A (en) Apparatus and method for processing 3d picture
US20110199465A1 (en) Method of processing parallax information comprised in a signal
US20130050187A1 (en) Method and Apparatus for Generating Multiple Image Views for a Multiview Autosteroscopic Display Device
JP2011250059A (en) Image processing device, image display device and image pickup device
JP2009251141A (en) Stereoscopic image display
US20130169633A1 (en) Stereoscopic image generation device, stereoscopic image display device, stereoscopic image adjustment method, program for causing computer to execute stereoscopic image adjustment method, and recording medium on which the program is recorded
US20110181593A1 (en) Image processing apparatus, 3d display apparatus, and image processing method
US20130076749A1 (en) Image processing device, image processing method, program, and integrated circuit
US20120293489A1 (en) Nonlinear depth remapping system and method thereof
US20100302234A1 (en) Method of establishing dof data of 3d image and system thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11850386

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11850386

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: JP