JP5444065B2 - Compound eye imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to a compound eye imaging apparatus and a control method thereof.

  One of the basic problems in a compound-eye imaging apparatus is that an inclination shift occurs between images due to a position (direction) shift of an optical member such as a lens or an image pickup device (CCD or the like). A compound-eye imaging device is provided with a plurality of optical members (imaging means), and usually the plurality of optical members are positioned relatively precisely in the factory production process, but the position (direction) deviation of the plurality of optical members is completely eliminated. It is difficult. In addition, positional displacement may occur or increase due to deterioration with time.

  Japanese Patent Application Laid-Open No. H10-228561 describes that the rotational shift amount θ and the like are calculated using a captured image obtained by capturing an adjustment pattern. Japanese Patent Application Laid-Open No. H10-228561 describes a technique for adjusting the displacement of the camera installation position using an image obtained by capturing an image of a cube as a subject. In Patent Documents 1 and 2, it is essential to capture a dedicated adjustment pattern or adjustment subject (cube), and it takes time and effort to adjust the displacement.

  Patent Document 3 describes using a projection matrix that adjusts a pair of (left and right) images in a matching direction. There is no reference in the adjustment direction, so there is a possibility that a pair of images may be adjusted in the wrong direction (direction).

JP-A-10-307352 JP-A-8-251627 JP 2004-334819 A

  SUMMARY OF THE INVENTION An object of the present invention is to suppress an inclination shift between a plurality of images without using an adjustment pattern or an adjustment subject.

  It is another object of the present invention to prevent the image after the tilt shift is adjusted from causing a sense of incongruity due to the tilt shift adjustment.

  The compound-eye imaging device according to the present invention has at least two imaging means including a solid-state imaging device, which are provided at positions where a common area can be imaged and images a subject from different positions, and at least two image data obtained by the imaging means. Image rotating means for independently rotating the image, non-reference represented by non-reference image data as remaining image data with respect to a reference image represented by reference image data that is one of the at least two image data Inclination deviation detecting means for detecting each inclination deviation angle amount of the image, and the image rotation means so that each of the non-reference images is rotated by an angle corresponding to the inclination deviation angle amount detected by the inclination deviation detection means. First tilt control means for controlling

  A control method for a compound eye image pickup device according to the present invention is a method for controlling a compound eye image pickup device including at least two image pickup means including a solid-state image pickup element that is provided at a position where a common region can be imaged and picks up an object from different positions. In this case, the tilt deviation detecting means is the remaining image data for the reference image represented by the reference image data which is one of at least two image data obtained by imaging by the at least two imaging means. Each inclination deviation angle amount of the non-reference image represented by the reference image data is detected, and each inclination control means detects an angle corresponding to the inclination deviation angle amount detected by the inclination deviation detection means. Rotate at least two image data obtained by the imaging means independently so that The image rotation unit, and controls.

  At least two image data are obtained by each of at least two imaging means, and one of them is treated as reference image data. The remaining image data other than the reference image data is non-reference reference image data. According to the present invention, the amount of inclination deviation angle of each non-reference image represented by the remaining image data with respect to the reference image represented by the reference image data that is one of the plurality of image data is detected and detected. The non-reference image is rotated by an angle corresponding to the tilt angle angle amount. The amount of tilt angle includes the size and direction. Since the inclination of the non-reference image approaches or matches the inclination of the reference image, it is possible to suppress or eliminate the inclination deviation between the reference image and the non-reference image. Without using a special adjustment pattern for adjusting the tilt or a special adjustment subject, it is possible to suppress or eliminate the tilt shift between the reference image and the non-reference image. In addition, according to the present invention, the non-reference image rotates and the reference image does not rotate, so that all of the plurality of image data obtained by imaging do not rotate. There is a low possibility that the intention (image composition) of the user who uses the compound-eye imaging device is broken.

  In one embodiment, for each of at least two pieces of image data obtained by the imaging means, a line segment detecting means for detecting a straight line segment included in the image represented by the image data, and the line segment detecting means. Among the detected straight line segments, a straight line segment that is closest to one of the horizontal direction and the vertical direction corresponding to the horizontal pixel direction and the vertical pixel direction included in the solid-state image sensor is used as a reference line. A reference line determination means for determining, wherein the inclination deviation detection means selects a subject image including the reference line determined by the reference line determination means as the reference image, and the inclination deviation detection means further includes , Each inclination deviation angle of the corresponding line corresponding to the reference line in the non-reference image with respect to the reference line in the reference image To detect.

  Since the plurality of (at least two) imaging units are provided at positions where a common area can be imaged, the plurality of images represented by the plurality of image data obtained by the imaging unit include the same subject image. A straight line segment existing in a subject image included in each of a plurality of images and directed to a direction closest to either the horizontal direction or the vertical direction is set as a reference line. An image including this reference line is positioned as a reference image. The non-reference image is rotated so that the inclination of the line segment (corresponding line) corresponding to the reference line in the non-reference image approaches the inclination of the reference line in the reference image.

  The merit of tilt adjustment using an adjustment pattern or an adjustment subject can be grasped (recognized) using an adjustment pattern or the like so that an ideal state without tilt deviation can be grasped and imaged in such an ideal state. By adjusting the position or angle of the camera or the like, the tilt deviation between a plurality of cameras (a plurality of images) can be adjusted. On the other hand, since the present invention is intended to suppress the tilt shift without using the adjustment pattern or the like, it cannot necessarily be said that the accuracy is better than the tilt shift adjustment using the adjustment pattern. However, in photography, it is statistically possible that images including boundaries that extend in the horizontal direction (for example, images that include horizontal lines) and images that include boundaries that extend in the vertical direction (for example, images that include trees) are taken. Many. In this embodiment, paying attention to this statistical fact, a straight line segment close to the horizontal direction or the vertical direction included in the image obtained by imaging is set as the reference line, and the direction of the reference line is approached. Tilt deviation can be suppressed or eliminated. Tilt deviation can be suppressed or eliminated with the same accuracy as using the adjustment pattern.

  Preferably, when all of the straight line segments detected by the line segment detection unit are not oriented in the horizontal vicinity angle range and the vertical vicinity angle range, the inclination by the inclination deviation detection unit is determined. The apparatus further comprises stop means for stopping the detection of the shift angle amount and the control of the image rotation means by the first tilt control means. This is to reduce the possibility of misadjustment by avoiding the adjustment of the inclination deviation itself when the possibility of suppressing the inclination deviation is low with high accuracy.

  In another embodiment, the imaging apparatus includes a reference line / corresponding line determination unit that determines whether or not directions of the reference line and the corresponding line coincide with each other, and the reference line / corresponding line determination unit performs the reference line / corresponding line determination unit. When the coincidence of the direction of the line and the corresponding line is determined, the non-reference image including the corresponding line in the matching direction is detected by the inclination deviation detecting unit by the inclination deviation detecting unit and by the first inclination controlling unit. The image forming apparatus further includes stop means for stopping the control of the image rotation means. If the directions of the reference line in the reference image and the corresponding line in the non-reference image match, it is considered that there is no tilt shift between the reference image and the non-reference image. In this case, since the subsequent processing is stopped, useless processing and power consumption can be avoided.

  Preferably, the first inclination control means rotates the non-reference image by an angle amount obtained by multiplying the inclination deviation angle amount detected by the inclination detection means by a predetermined coefficient smaller than 1. The image rotation means is controlled. Rather than canceling the tilt shift at once, it is possible to prevent the occurrence of image failure even if there is an error in the calculation of the tilt shift angle amount by suppressing it gently.

  Of course, the tilt deviation may be eliminated at once. In this case, in one embodiment, the first inclination control means obtains an angle obtained by multiplying the inclination deviation angle amount detected by the inclination detection means by a predetermined coefficient smaller than 1 by the inclination deviation angle quantity. The image rotation means is controlled so that the non-reference image is further rotated by the angle amount of the difference from the amount.

  Preferably, for each of the image data representing at least two subject images obtained by the imaging means, a cumulative value storage means for storing a cumulative value of the tilt deviation angle amount used in the previous image rotation, and the first Prior to the control of the image rotation means by one inclination control means, the image rotation is performed so that the non-reference image is rotated by an angle corresponding to the cumulative value of the inclination deviation angle amount stored in the cumulative value storage means. Second inclination control means for controlling the means is provided. Since past adjustment results can be used, it is possible to gradually reduce the inclination deviation and finally eliminate it.

  In one embodiment, the image rotation means is an imaging means rotation actuator that rotates each of the imaging means independently. By rotating the imaging means itself, the inclination of the non-reference image represented by the image data obtained thereafter approaches or matches the inclination of the reference image.

  In another embodiment, the image rotation means is image data rotation processing means for rotating a subject image by rotationally converting image data obtained by the imaging means. By image rotation processing such as affine transformation and projective transformation, the inclination of the non-reference image represented by the image data obtained thereafter approaches or coincides with the inclination of the reference image.

  In one embodiment, the compound-eye imaging device has an inclination adjustment flag storage means for storing an inclination adjustment flag, an inclination adjustment flag control means for turning on / off the inclination adjustment flag, and when the inclination adjustment flag is on. The image processing apparatus further includes image rotation permission means for permitting detection of an inclination deviation angle amount by the inclination deviation detection means and control of the image rotation means by the first inclination control means. By preventing the inclination adjustment from being executed when the inclination adjustment flag is turned off, it is possible to reduce the power consumption and to avoid consecutive erroneous adjustments.

  Preferably, the inclination adjustment flag control means turns off the inclination adjustment flag stored in the inclination adjustment flag storage means in response to the image data being rotated by the image rotation means. Thereafter, the tilt adjustment flag may be turned on under a predetermined condition.

  The condition for turning on the tilt adjustment flag is as follows: when the compound eye imaging device is turned on, when the setting of the compound eye imaging device is changed, after image data is rotated by the image rotating means, for a predetermined time. When the period is over.

It is a block diagram which shows the electric constitution of a digital still camera. It is a flowchart which shows the calibration process of 1st Example. It is a flowchart which shows an example of an adjustment flag setting process. It is a flowchart which shows the other example of an adjustment flag setting process. It is a flowchart which shows the further another example of an adjustment flag setting process. (A) and (B) show examples of images before and after the line segment detection processing, respectively. An example of the image after a line segment detection process is shown. (A), (B), and (C) show how the image is rotated by the calibration process. It is a flowchart which shows the calibration process of the modification of 1st Example. It is a flowchart which shows the calibration process of 2nd Example. (A), (B), and (C) show cumulative adjustment parameters. (A), (B), and (C) show how the image is rotated by the calibration process. It is a flowchart which shows the calibration process of 2nd Example. It is a flowchart which shows the calibration process of 3rd Example. It is a flowchart which shows the calibration process of 3rd Example. It is a flowchart which shows the calibration process of 3rd Example.

[First embodiment]
FIG. 1 is a block diagram showing an electrical configuration of a digital still camera. The block diagram shown in FIG. 1 is applied not only to the first embodiment but also to second and third embodiments described later. The embodiment of the present invention can be applied not only to a digital still camera but also to a digital movie camera.

  The overall operation of the digital still camera is controlled by the CPU 1.

  The digital still camera is provided with two optical members (imaging members) 10L and 10R that are separated by a distance corresponding to human parallax, and the two optical members 10L and 10R are used from two viewpoints whose positions are different. This is a so-called compound-eye camera that can obtain image data representing two images (two-viewpoint images). An image that can be viewed stereoscopically (hereinafter, referred to as a stereoscopic image) can be reproduced using the two viewpoint images. For the sake of clarity, in the following description, the two optical members 10L and 10R are called and distinguished from the left optical member 10L and the right optical member 10R, respectively. The left optical member 10L and the right optical member 10R are provided at positions where a common area can be imaged in the digital still camera.

  The left optical member 10L includes a solid-state imaging device, here a CCD 15L, and an imaging lens 11L, an aperture 12L, an infrared cut filter 13L, and an optical low-pass filter (OLPF) 14L are provided in front of the CCD 15L. Similarly, the right optical member 10R includes a CCD 15R, and an imaging lens 11R, an aperture 12R, an infrared cut filter 13R, and an optical low-pass filter 14R are provided in front of the CCD 15R.

  Optical member rotation actuators 30L and 30R are provided on the left optical member 10L and the right optical member 10R, respectively. The optical member rotation actuators 30L and 30R independently control the inclinations (postures) of the optical members 30L and 30R.

  The digital still camera includes an operation unit 2. The operation device 2 includes a power button, a mode setting dial, a two-stroke type shutter release button, and the like. An operation signal output from the controller 2 is input to the CPU 1. The mode set by the mode setting dial includes a shooting mode and a playback mode. The modes set in more detail in the shooting mode include an auto mode and a manual mode.

  The digital still camera is provided with a light-emitting device 6 for strobe imaging and a light-receiving device 7 for receiving reflected light of light emitted from the light-emitting device 6.

  When the power source of the digital still camera is turned on and the photographing mode is set, the light beam representing the subject image enters both the left optical member 10L and the right optical member 10R. In the left optical member 10L, the light beam is incident on the light receiving surface of the CCD 15L through the imaging lens 11L, the stop 12L, the infrared cut filter 13L, and the optical low-pass filter 14L. A subject image is formed on the light receiving surface of the CCD 15L, and an analog signal representing the subject image is output from the CCD 15L. A subject is imaged by the CCD 15L at a constant cycle, and a video signal representing the subject image is output from the CCD 15L frame by frame at a constant cycle. The same applies to the right optical member 10R. A subject image is formed on the light receiving surface of the CCD 15R via the imaging lens 11R, aperture 12R, infrared cut filter 13R, and optical low-pass filter 14R, and an analog signal representing the subject image is output from the CCD 15R. The CCD 15L in the left optical member 10L and the CCD 15R in the right optical member 10R operate synchronously.

  The analog signal processing device 16 includes a correlated double sampling circuit, a signal amplifier, and the like. Analog signals representing the subject images output from the CCDs 15L and 15R are input to the analog signal processing device 16, where correlated double sampling, signal amplification, and the like are performed. The analog video signal output from the analog signal processing device 16 is input to the analog / digital conversion circuit 18 and converted into digital image data. Two digital image data from each of the left optical member 10L and the right optical member 10R are obtained. Hereinafter, these two image data are referred to as left image data and right image data, respectively, and a set of left image data and right image data is referred to as stereoscopic image data. The stereoscopic image data is temporarily recorded in the main memory 20 under the control of the memory control circuit 19.

  The stereoscopic image data is read from the main memory 20 and input to the digital signal processing circuit 21. The digital signal processing circuit 21 performs predetermined digital signal processing such as white balance adjustment and gamma correction. The data subjected to the digital signal processing in the digital signal processing circuit 21 is given to the display control circuit 26. By controlling the display device 27 by the display control circuit 26, a subject image (stereoscopic image) is displayed on the display screen.

  When the shutter release button is pressed in the first stage, the lenses 11R and 11L are driven by the lens driving circuits 5L and 5R to perform focusing. Luminance data is obtained in the digital signal processing circuit 21 based on at least one of left image data and right image data read from the main memory 20. The luminance data is input to the integration circuit 23 and integrated. Data representing the integrated value is given to the CPU 1 to calculate the exposure amount. The apertures of the apertures 12L and 12R are controlled by the aperture drive circuits 4L and 4R so that the calculated exposure amount is obtained, and the shutter speeds of the CCDs 15L and 15R are controlled by the image sensor drive circuits 3L and 3R.

  When the shutter release button is pressed in the second stage, the stereoscopic image data output from the analog / digital conversion circuit 18 is recorded in the main memory 20. Predetermined digital signal processing is performed on the stereoscopic image data read from the main memory 20 as described above. The stereoscopic image data output from the digital signal processing circuit 21 is compressed by the compression / decompression processing circuit 22. The compressed stereoscopic image data is recorded on the memory card 25 under the control of the external memory control circuit 24.

  When the reproduction mode is set, the compressed stereoscopic image data recorded on the memory card 25 is read. The read compressed stereoscopic image data is expanded in the compression / expansion processing circuit 22 and is given to the display control circuit 26. A reproduced image (stereoscopic image) is displayed on the display screen of the display device 27.

  The stereoscopic image data obtained by the digital still camera includes a set of left image data and right image data as described above. Since the optical members 10L and 10R are spaced apart by a distance corresponding to human parallax, the left image data and the right image data include parallax shift, and the parallax shift causes a sense of perspective, which is represented by stereoscopic image data. The displayed image (reproduced image) can be stereoscopically viewed. However, if a tilt shift other than the parallax shift is included between the left image and the right image, a sense of incongruity may occur in the stereoscopic view.

  In the digital still camera of the first embodiment, the tilt deviation between the left image and the right image is reduced by controlling the rotation of the optical members 10L and 10R. In the following description, eliminating the above-described tilt deviation is referred to as calibration.

  FIG. 2 is a flowchart showing the procedure of calibration processing in the digital still camera in the first embodiment. 3 to 5 are flowcharts showing the adjustment flag setting process in the calibration process in three different modes. The calibration process is generally performed by control by the CPU 1, but part or all of the calibration process may be performed by the digital signal processing circuit 21. For example, the calibration processing execution program stored in the main memory 20 is executed by the CPU 1, whereby the calibration processing shown in FIG. 2 is performed in the digital still camera.

  Referring to FIG. 1, the calibration process is performed on condition that the adjustment flag is on (step 42). The adjustment flag is a flag indicating either on or off, and is stored in the main memory 20. When the adjustment flag is turned on, the calibration process is started (YES in step 42). If the adjustment flag is off, the calibration process is not performed (NO in step 42). The calibration process executed when the adjustment flag is turned on turns off the adjustment flag at the end of the process (step 51).

  An adjustment flag setting process is performed before determining whether the adjustment flag is on or off (step 41). 3 to 5, the adjustment flag can be turned on at various timings. Referring to FIG. 3, in one example, when the power of the digital still camera for which the photographing mode is set is turned on, the adjustment flag is turned on (YES in step 41a, step 41b). Referring to FIG. 4, when the setting mode is changed on condition that the power of the digital still camera for which the shooting mode is set is turned on, for example, the mode is switched from the auto mode to the manual mode. Sometimes the adjustment flag may be turned on (YES in step 41c, step 41b). Referring to FIG. 5, on the condition that the power of the digital still camera for which the shooting mode is set is turned on, a predetermined time has elapsed or has already elapsed since the previous calibration process was executed. In this case, the adjustment flag may be turned on (YES in step 41d, step 41b). When the adjustment flag is turned on as the predetermined time elapses (FIG. 5), the date and time when the calibration process is finished is recorded in the main memory 20, and the adjustment flag is turned on when the predetermined time elapses with reference to the date and time. Is done. The adjustment flag ON setting processing shown in FIGS. 3 to 5 may be combined as appropriate.

  Returning to FIG. 2, if the adjustment flag is turned on in the main memory 20, the calibration process is started as described above (YES in step 42).

  The calibration process is premised on the presence of stereoscopic image data, that is, left image data and right image data. When the power of the digital still camera for which the photographing mode is set is turned on, stereoscopic image data is captured by the optical members 10L and 10R, and a through image is displayed on the display device 27. Stereoscopic image data representing this through image can be the target of calibration processing. However, the stereoscopic image data when the shutter release button is pressed in one step or the stereoscopic image data when the shutter release button is pressed in two steps may be the target of the calibration process. In any case, the stereoscopic image data (left image data and right image data) obtained simultaneously by the optical members 10L and 10R that operate in synchronization are the targets of the calibration process. The stereoscopic image data to be calibrated is input to the main memory 20 and temporarily stored (step 43).

  A straight line segment detection process is performed for each of the left image L represented by the left image data and the right image R represented by the right image data (step 44). In the line segment detection process, edge pixels (for example, pixels whose luminance gradient is larger than a predetermined value) are detected for each of the left image L and the right image R, and adjacent edge pixels are connected (line-divided). Only line segments constituting straight lines are extracted from the line segment image, and line segments constituting curves are excluded (or masked). Details of the straight line segment detection processing are described in Japanese Patent Application Laid-Open No. 2004-334819 described above.

  6A and 6B show examples of images before and after the line segment detection process. The line segment image shown in FIG. 6B is obtained from the image shown in FIG. 6A by the line segment detection process. As shown in FIG. 6B, nine straight line segments are detected from the image shown in FIG.

  Further, preferably, as shown in FIG. 7, only the straight line segments having a length equal to or longer than a predetermined length are extracted from the detected straight line segments. In this case, straight line segments having a length less than the predetermined length are excluded (masked).

  Returning to FIG. 2, the left image data and the right image data are obtained by synchronous imaging, and the left image data and the right image data are provided on the left optical member 10 </ b> L provided at a position for imaging a common area. Since the image is obtained by being picked up by the right optical member 10R, if the left image L includes a straight line segment, the right image R also includes a corresponding straight line segment. For the straight line segments detected in each of the left image L and the right image R, whether there is a straight line segment that points in the horizontal direction or a direction close thereto, and there is a straight line segment that points in the vertical direction or a direction close thereto. Is determined (step 45). Both the horizontal direction and the vertical direction correspond to the pixel arrays (horizontal pixels and vertical pixels) of the CCDs 15L and 15R. Regarding the obtained left image L and right image R, in the left image L and the right image R, Corresponds to the horizontal and vertical directions. For example, a straight line having an inclination within an angle range of ± 10 degrees from the horizontal direction is treated as a horizontal line segment. A straight line having an inclination within an angle range of ± 10 degrees from the vertical direction is treated as a vertical line segment.

  If there is no horizontal line segment or vertical line segment, the calibration process ends (NO in step 45).

  If there is a horizontal line segment or a vertical line segment (YES in step 45), one of the detected horizontal line segments and vertical line segments that points most horizontally or vertically is A reference line is determined (step 46). In the case of the image shown in FIG. 6A, the image portion corresponding to the left boundary line of the tree trunk is determined as the reference line (see FIG. 7).

  Ideally, the left image L and the right image R have no shift other than the parallax shift, for example, no tilt shift. The left optical member 10L and the right optical member 10R are tilted almost accurately at the time of shipment from the factory so as not to tilt. However, if the tilt of the optical member fluctuates due to subsequent deterioration over time, a tilt shift may occur between the left image L and the right image R.

  FIGS. 8A and 8B show examples of the left image L and the right image R, respectively, obtained when there is an inclination shift between the left optical member 10L and the right optical member 10R. The left border of the trunk of the tree (highlighted with bold lines) is inclined 3 degrees clockwise from the vertical direction in the left image L, while it is inclined 5 degrees counterclockwise from the vertical direction in the right image R. ing. In the following description, the clockwise angle is represented by +, and the counterclockwise angle is represented by-.

  FIG. 8C shows the result of the calibration process, and the digital still camera is controlled so that the inclination of the right image R in FIG. 8B approaches the inclination of the left image L by the calibration process. The image obtained by this is shown. In the following description, FIG. 8A, FIG. 8B, and FIG.

  Returning to FIG. 2, an image including a reference line is selected as a reference image (step 47). As described above, the reference line is one straight line that is most directed in the horizontal direction or the vertical direction among the horizontal direction line segment and the vertical direction line segment detected in each of the left image L and the right image R. In the case of the left image L and the right image R shown in FIGS. 8A and 8B, the left boundary line of the trunk of the tree in the left image L is oriented most vertically. Therefore, the left boundary line of the trunk of the tree in the left image L is used as the reference line. Since the image including the reference line is the left image L, the left image L is set as the reference image. The right image R is a non-reference image.

  A conversion parameter for matching the inclination of the line segment corresponding to the reference line in the non-reference image (hereinafter referred to as the corresponding line) with the inclination of the reference line in the reference image is calculated (step 48). In the example shown in FIG. 8A and FIG. 8B, the reference image is the left image L and the non-reference image is the right image R as described above, from the left boundary line of the tree trunk of R in the right image. The conversion parameter is to match the obtained line segment (corresponding line) with the reference line in the left image L. The conversion parameter represents the difference in inclination between the reference line and the corresponding line (inclination deviation angle amount) (including the magnitude and direction (clockwise or counterclockwise)). Note that the corresponding line detection process corresponding to the reference line is also described in Japanese Patent Application Laid-Open No. 2004-334819 described above.

  In the example shown in FIGS. 8A and 8B, the reference line in the left image L is inclined +3 degrees from the vertical direction. On the other hand, the corresponding line in the right image R is inclined by −3 degrees from the vertical direction. The corresponding line in the right image R has an inclination that matches the reference line in the left image L when the right image R is rotated clockwise by 8 degrees. As the conversion parameter, rotation data for matching the inclination of the corresponding line in the non-reference image with the inclination of the reference line in the reference image, in the above example, data representing the clockwise rotation of 8 degrees with respect to the right image R ( Hereinafter, “right + 8 degrees”) is determined.

  The determined conversion parameter is multiplied by a coefficient α (0 <α <1) (step 49). The conversion parameter multiplied by the coefficient α is hereinafter referred to as an adjustment parameter. For example, if the coefficient α is 0.6 and the conversion parameter is “right + 8 degrees”, the adjustment parameter is “right + 4.8 degrees”. Using the calculated adjustment parameter, a process of adjusting the inclination of the non-reference image, here, the right image R is performed. The reason for adjusting the tilt using an adjustment parameter smaller than the conversion parameter without adjusting the tilt using the conversion parameter itself is, for example, when the accuracy of the conversion parameter is poor due to inaccurate reference line detection. This is to prevent the adverse effect from being exerted strongly in the calibration process.

  As described above, the digital still camera includes optical member rotation actuators 30L and 30R that independently control the inclinations of the left optical member 10L and the right optical member 10R (see FIG. 1). The tilt adjustment of the left image L and the right image R is performed by controlling the optical member rotation actuators 30L and 30R to adjust the tilt of the left optical member 10L or the right optical member 10R. In accordance with the adjustment parameter “right + 4.8 degrees”, the CPU 1 gives a control signal for rotating the right image optical member 10R clockwise by +4.8 degrees with respect to the optical axis direction to the optical member rotation actuator 30R. As a result, the right optical member 10R is rotated clockwise by 4.8 degrees by the optical member rotation actuator 30R. Inclination deviation between the left image L and the right image R is suppressed (step 50). In the case of the example shown in FIGS. 8A to 8C, an inclination shift of 8 degrees between the reference line in the left image L and the corresponding line in the right image R (see FIGS. 8A and 8B). ) Is reduced to an inclination shift of 3.2 degrees (compare FIGS. 8A and 8C).

  Thereafter, the above-described adjustment parameter flag is turned off, and the calibration process is terminated (step 51). The left image L and the right image R in which the tilt deviation is suppressed are obtained by the digital still camera by subsequent imaging.

  The calibration process of the first embodiment is suitable for a digital still camera in which the left optical member 10L or the right optical member 10R whose tilt is adjusted is held as it is even when the power is turned off. The more the digital still camera is used, the smaller the tilt shift in the stereoscopic image (the left image L and the right image R). Of course, the calibration process of the first embodiment can be applied even to a digital still camera in which the left optical member 10L or the right optical member 10R whose inclination is adjusted when the power is turned off returns to the initial state. is there.

  In the calibration process according to the first embodiment, as described above, a line segment close to the vertical direction or the horizontal direction is used as a reference line, and the corresponding line of the other image approaches this reference line so that the stereoscopic image (left The inclination deviation between the image L and the right image R) is eliminated. In photography, a subject having a boundary line along the horizontal direction or the vertical direction is generally taken with high frequency. The tilt deviation is adjusted such that the boundary line along the horizontal direction or the vertical direction is closer to the horizontal direction or the vertical direction. For this reason, there is little possibility that a sense of incongruity will occur even if the tilt deviation is adjusted.

  Further, in the calibration process of the first embodiment, in place of the tilt control of the left optical member 10L or the right optical member 10R, the tilt shift is caused by rotating the non-reference image by image rotation processing, for example, affine transformation or projective transformation. You may make it suppress. In this case, the optical member rotation actuators 30L and 30R for controlling the inclinations of the left optical member 10L and the right optical member 10R are not necessarily required for the digital still camera.

[Modification of the first embodiment]
FIG. 9 is a flowchart showing the procedure of the calibration process in a modification of the first embodiment. The flowchart shown in FIG. 2 includes a step (step 52) for ending the process without performing the subsequent calibration process when it is determined that the inclinations of the left image L and the right image R are the same. Is different.

  If there is a horizontal line segment or a vertical line segment in each of the left image L and the right image R (YES in step 45), then the line segment that points most horizontally or vertically in the left image L and the right image It is determined whether or not the line segment that points most horizontally or vertically in R exists in the left image L and the right image R with the same inclination (or faces in the same direction) (step 52). If the inclination is the same, the left image L and the right image R are treated as having no inclination deviation without performing the calibration process. In this case, the subsequent processing of the calibration processing is skipped (YES in step 52). If the inclinations are different, the line segment that is most horizontally or vertically oriented is determined as the reference line as in the first embodiment, and an image including the reference line is selected as the reference image (NO in step 52, step 46, 47).

[Second Embodiment]
FIG. 10 is a flowchart showing the procedure of the calibration process of the second embodiment. In the calibration process of the second embodiment, the tilt adjustment is performed in advance using the adjustment parameters (hereinafter referred to as cumulative adjustment parameters) obtained by the calibration process before (previous) the previous time (steps 61 and 62). , And that the cumulative adjustment parameter is stored and held in the main memory 20 (steps 63 and 64) is different from the calibration process (FIG. 2) of the first embodiment. The same steps as those shown in the flowchart of FIG.

  The calibration process of the second embodiment is suitable for a digital still camera in which the left optical member 10L and the right optical member 10R, whose inclinations are adjusted, are restored to the initial state when the power is turned off. 11A, 11B, and 11C show how the cumulative adjustment parameter is updated. The cumulative adjustment parameter is stored in the main memory 20. FIGS. 12A to 12C show the calibration process in the second embodiment.

  The calibration process of the second embodiment starts when the power of the digital still camera for which the photographing mode is set is turned on.

  In the initial state (the state in which the tilt adjustment has not been performed yet), the cumulative adjustment parameter is not stored in the main memory 20 (0 degree) (FIG. 11A). In this case, as in the first embodiment, the conversion parameter is calculated, the conversion parameter is multiplied by the coefficient α, the adjustment parameter is calculated, and the tilt adjustment of the left optical member 10L or the right optical member 10R is performed according to the adjustment parameter. (Step 41 to Step 50, see FIGS. 8A to 8C).

  The cumulative adjustment parameter is updated using the calculated adjustment parameter (step 63). The cumulative adjustment parameter update process is calculated according to Equation 1 for each of the left image L (left optical member 10L) and the right image R (right optical member 10R).

  Cumulative adjustment parameter after update = cumulative adjustment parameter + adjustment parameter Equation 1

  For example, in the example described in the first embodiment (FIGS. 8A to 8C), “right + 4.8 degrees” is calculated as the adjustment parameter, so the cumulative adjustment parameter for the right image R is “right +4.8 degrees ". The cumulative adjustment parameter for the left image L remains 0 degrees (FIG. 11B).

  As described above, the digital still camera of the second embodiment is based on the assumption that when the power is turned off, the left optical member 10L and the right optical member 10R whose inclinations are adjusted return to the initial state. When the power of the digital still camera is turned off and then turned on again, the left optical member 10L and the right optical member 10R of the digital still camera are in the initial state.

  When the cumulative adjustment parameter is stored in the main memory 20 of the digital still camera that is powered on, first, the tilt adjustment of the optical member is performed using the cumulative adjustment parameter. That is, the cumulative adjustment parameter is read from the main memory 20 (step 61), and the inclination of the optical member is adjusted according to the read cumulative adjustment parameter (step 62). Referring to FIGS. 8A to 8C and FIG. 11B, when “right + 4.8 degrees” is stored as the cumulative adjustment parameter, the tilt adjustment of the optical member using the cumulative adjustment parameter is performed. The right optical member 10R rotates clockwise by 4.8 degrees. The left image L and the right image R in the states shown in FIGS. 12A and 12B are obtained.

  Thereafter, the calibration process is executed by the same process as in the first embodiment. In the example shown in FIGS. 12A and 12B, the left boundary line of the tree trunk of the left image L is inclined +3 degrees from the vertical direction, whereas the right image R is inclined -0.2 degrees. . Since the left border of the tree trunk of the right image R is closer to the vertical direction, the right image R is selected as the reference image (step 47). The left image L becomes a non-reference image for tilt adjustment.

  In the example shown in FIGS. 12A and 12B, the conversion parameter for matching the slope of the corresponding line in the left image L with the slope of the reference line in the right image R is −3.2 degrees. An adjustment parameter obtained by multiplying the conversion parameter by a predetermined coefficient α (= 0.6) is −1.92 degrees. The left optical member 10L is rotationally controlled so that the left image R rotates 1.92 degrees counterclockwise (step 50) (see FIG. 12C).

  Since the adjustment parameter for the left image L (left optical member 10L) has been calculated, the cumulative adjustment parameter for the left image L is updated (steps 63 and 64). According to Equation 1, the cumulative adjustment parameter for the left image L is updated from 0 degrees to -1.92 degrees (see FIG. 11C).

  In the next calibration process, the left optical member 10L is rotated 1.92 degrees counterclockwise and the right optical member 10R is 4.8 degrees clockwise according to the cumulative adjustment parameter shown in FIG. It is rotated by a minute (steps 61 and 62).

  Thus, even if the left optical member 10L and the right optical member 10R that have been tilt-adjusted return to the initial state each time the digital still camera is turned off, the cumulative adjustment parameter is used. The tilt adjustment of the optical member can be performed using the cumulative adjustment parameter representing the previous (previous) adjustment result, and then the tilt adjustment is performed again according to the adjustment parameter. The tilt shift of the optical member can be gradually eliminated, and finally the tilt shift can be eliminated.

[Modification of Second Embodiment]
FIG. 13 shows a modification of the second embodiment, and is a flowchart showing the flow of calibration processing. The flowchart of the second embodiment shown in FIG. 11 is that the tilt adjustment for eliminating (suppressing) the tilt shift is not the adjustment by rotating the optical member, but image processing, typically affine transformation or projection. The difference is that it is performed by image conversion processing using conversion.

  Cumulative adjustment parameters representing the inclination adjustment results up to the previous time are stored in the main memory 20. Therefore, if the left image L and the right image R are subjected to image rotation processing according to the cumulative adjustment parameter, a stereoscopic image in which the inclination deviation is suppressed can be obtained, and the inclination deviation gradually decreases. In a modification of the second embodiment, image rotation processing is performed on the left image L and right image R input to the main memory 20 in accordance with the cumulative adjustment parameter (see FIGS. 11B and 11C) (see FIG. 11B). Step 71). When the cumulative adjustment parameters shown in FIG. 11C are stored in the main memory 20, the left image data representing the left image L and the right image data representing the right image R input to the main memory 20, respectively. On the other hand, by affine transformation or projective transformation, 1.92 degree image rotation processing is performed counterclockwise for the left image data, and 4.8 degree image rotation processing is performed clockwise for the right image data. .

  Thereafter, the tilt adjustment using the adjustment parameters is executed on the left image L and the right image R that have been subjected to the rotation processing. The tilt adjustment based on the adjustment parameter is also performed by image rotation processing (affine transformation or projective transformation) (steps 44 to 49, 72).

  In the case of adjusting the inclination by image rotation processing by affine transformation or projective transformation, the cumulative adjustment parameter is calculated (updated) by the following equation 2 instead of equation 1.

  Cumulative adjustment parameter after update = cumulative adjustment parameter / adjustment parameter Equation 2

  When tilt adjustment is performed by image rotation processing, the optical member rotation actuators 30L and 30R for adjusting the tilt of the left optical member 10L and the right optical member 10R are not necessarily required for the digital still camera.

  When the tilt adjustment is performed by the image rotation process, the left image data and the right image data subjected to the image rotation process by the calibration process are stored in the main memory 20. The stereoscopic image after the inclination adjustment is displayed on the display device 27. Also on the memory card 25, stereoscopic image data (a set of left image data and right image data) after tilt adjustment is recorded.

[Third embodiment]
FIG. 14, FIG. 15 and FIG. 16 are flowcharts showing the flow of the calibration process of the third embodiment.

  In the calibration process (FIG. 2) of the first embodiment described above, the calibration process is performed using the adjustment parameter in which the conversion parameter is reduced by the coefficient α (0 <α <1), so that the non-reference image (its corresponding line) is obtained. ) To approximate the inclination of the reference image (its reference line). In the calibration process (FIG. 10) according to the second embodiment, the inclination adjustment based on the result of the previous calibration process (cumulative adjustment parameter) (cumulative adjustment parameter) is performed in advance using the cumulative adjustment parameter. The inclination of the corresponding line) is gradually matched with the inclination of the reference image (the reference line). The calibration process of the third embodiment is further characterized in that each time the calibration process is performed, the inclination of the non-reference image (its corresponding line) is completely matched with the inclination of the reference image (its reference line). , Different from the second embodiment.

  FIG. 14 is obtained by adding steps (steps 81 to 85) for completely matching the inclination of the non-reference image with the inclination of the reference image in the calibration process (FIG. 2) of the first embodiment. FIG. 15 is obtained by adding steps (steps 81 to 85) for completely matching the inclination of the non-reference image with the inclination of the reference image in the calibration process (FIG. 10) of the second embodiment. FIG. 16 shows a step (steps 81, 82, 84, 86, 87) for completely matching the inclination of the non-reference image with the inclination of the reference image in the calibration process (FIG. 13) of the modified example of the second embodiment. ) Is added.

  Referring to FIG. 14, the third embodiment differs from the first and second embodiments in the case where the adjustment flag is turned off (NO in step 42), and the horizontal or vertical line segment. Is not present in the left image L and the right image R (NO in step 45), tilt adjustment is performed. In both cases where the adjustment flag is off (NO in step 42) and there is no horizontal or vertical line segment (NO in step 45), step 48 is skipped, so the conversion parameter is It has not been calculated. In the third embodiment, on the condition that the conversion parameter is not calculated, the processing of step 43 to step 48 is executed to calculate the conversion parameter (step 84). That is, even if the adjustment flag is off, the left image data and the right image data are input to the main memory 20 (step 43), and a linear component is detected in each of the left image L and the right image R (step 44). . A horizontal line segment (a straight line component having an inclination within an angle range of ± 10 degrees from the horizontal direction) or a vertical line segment (a straight line component having an inclination within an angle range of ± 10 degrees from the vertical direction) is the left image L and Even if it does not exist in the right image R, the most horizontal or vertical line segment is determined as the reference line, and an image including the reference line is selected as the reference image (steps 46 and 47). A conversion parameter is calculated that matches the slope of the corresponding line in the non-reference image with the slope of the reference line in the reference image (step 48).

  Unlike the first embodiment, not the adjustment parameter but the conversion parameter itself is used to adjust the inclination of the optical member corresponding to the non-reference image (step 85). For example, when the left image L and the right image R shown in FIGS. 8A and 8B are objects of calibration processing, the slope of the corresponding line in the right image R is set as the reference in the left image L. Since the conversion parameter matched with the line is “right + 8 degrees”, the right optical member 10R is controlled to rotate by the optical member rotation actuator 30R so that the right image R is rotated clockwise by 8 degrees. Thereby, the inclination of the corresponding line in the right image R completely matches the inclination of the reference line in the left image L. The left image L and the right image R from which the tilt shift is eliminated are obtained by a digital still camera by imaging thereafter.

  On the other hand, when the conversion parameter is calculated and the inclination adjustment using the adjustment parameter described above has already been performed (YES in step 81), the difference parameter is further calculated and the inclination adjustment process using the difference parameter is performed (step 82). , 83). The difference parameter is calculated by the following equation.

  Difference parameter = (1−α) · Conversion parameter Equation 3

  In Expression 3, the coefficient α is the same as the coefficient used for calculating the adjustment parameter in Expression 1, and is a numerical value in the range of 0 <α <1.

  For example, when the coefficient α = 0.6 and the conversion parameter = right + 8 degrees, the difference parameter is (1−0.6) · right + 8 degrees = right + 3.2 degrees. The right optical member 10R is further controlled to rotate by the optical member rotation actuator 30R so that the right image R is rotated by 3.2 degrees clockwise.

  In addition to the rotation control of 4.8 degrees clockwise according to the adjustment parameter, the rotation control of 3.2 degrees clockwise is further performed, so the right optical member 10R rotates by a total of 8 degrees. As a result, the corresponding line in the right image R completely matches the inclination of the reference line in the left image L. The left image L and the right image R from which the tilt shift is eliminated are obtained by a digital still camera by imaging thereafter.

  In the calibration process shown in FIG. 14, the digital still camera that returns the initial state to the digital still camera that holds the left-side optical member 10L and the right-side optical member 10R whose tilts are adjusted even when the power is turned off. It can also be applied to.

  Referring to FIG. 15, the calibration process shown in FIG. 15 is obtained by adding steps 81 to 85 to the calibration process (FIG. 10) of the second embodiment as described above, and the power is turned off. This is suitable for a digital still camera in which the left optical member 10L and the right optical member 10R whose inclination is adjusted are restored to the initial state. As described in the second embodiment, the result of the previous tilt adjustment (cumulative adjustment parameter) is stored in the main memory 20, so that the digital still camera is turned on according to the cumulative adjustment parameter. Each time the tilt adjustment is performed according to the tilt adjustment result (cumulative adjustment parameter) until the previous time (steps 61 and 62), the remaining tilt deviation is adjusted according to the adjustment parameter (steps 41 to 51).

  Similarly to FIG. 14, also in the calibration process of FIG. 15, when the adjustment flag is turned off (NO in step 42), and when there is no horizontal or vertical line segment (NO in step 45). In this case, the tilt adjustment is performed, and in this case, not the adjustment parameter but the conversion parameter itself is used to adjust the tilt of the optical member corresponding to the non-reference image (steps 84 and 85).

  For example, in the case where the left image L and the right image R shown in FIGS. 8A and 8B are objects of calibration processing, the cumulative adjustment parameter shown in FIG. 11B is stored in the main memory 20. When (right + 4.8 degrees) is stored, the right optical member 10R is rotationally controlled by the optical member rotation actuator 30R so that the right image R rotates 4.8 degrees clockwise according to the cumulative adjustment parameter. . The left image L and the right image R shown in FIGS. 8A and 8B are those shown in FIGS. 12A and 12B (steps 61 and 62).

  As described in the second embodiment, in FIGS. 12A and 12B, the conversion parameter for matching the slope of the corresponding line in the left image L with the reference line in the right image R is “left-3. .2 degrees ". If the adjustment flag is off (NO in step 42), and if there is no horizontal or vertical line segment (NO in step 45), left according to the conversion parameter "left -3.2 degrees" The left optical member 10L is rotationally controlled by the optical member rotation actuator 30L so that the image L is rotated counterclockwise by 3.2 degrees (NO in step 81, steps 84 and 85). The corresponding line in the left image R completely matches the inclination of the reference line in the right image L. The left image L and the right image R in which the tilt deviation is eliminated are obtained by a digital still camera by subsequent imaging.

  On the other hand, when the conversion parameter is calculated and the inclination adjustment using the adjustment parameter described above has already been performed (YES in step 81), the left image shown in FIG. 12A is obtained by the inclination adjustment according to the adjustment parameter (step 50). The left optical member 10L is controlled to rotate counterclockwise so that L rotates 1.92 degrees counterclockwise (FIG. 12C). Further, the above-described difference parameter is calculated, and tilt adjustment processing using the difference parameter is performed (steps 82 and 83). For example, when the coefficient α = 0.6 and the conversion parameter = left−3.2 degrees, the difference parameter is (1−0.6) · left−3.2 degrees = left−1 from Equation 3 above. .28 degrees. The left optical member 10L is rotationally controlled by the optical member rotation actuator 30L so that the left image R is rotated counterclockwise by 1.28 degrees. Thereby, the inclination of the corresponding line in the left image R completely matches the inclination of the reference line in the right image L. The left image L and the right image R in which the tilt deviation is eliminated are obtained by a digital still camera by subsequent imaging.

  Referring to FIG. 16, the calibration process shown in FIG. 16 is obtained by adding a step of completely eliminating the tilt deviation to the calibration process (FIG. 13) of the modified example of the second embodiment as described above. Yes, the tilt shift is adjusted not by adjusting the tilt shift by rotationally driving the optical member, but by image rotation processing, typically affine transformation or projective transformation. As in FIG. 15, even if the conversion parameter is not calculated (NO in step 81) or the conversion parameter is calculated (YES in step 81), the non-reference image is displayed by image processing. The left image L and the right image R from which the inclination deviation is eliminated are obtained by making the inclination of the corresponding line completely match the inclination of the reference line in the reference image (steps 82 and 86, steps 84 and 87). Is obtained by a digital still camera.

  In the first to third embodiments described above, the digital still camera is premised on a so-called two-lens digital still camera provided with a left optical member 10L and a right optical member 10R. Of course, the present invention can be applied even to three or more eyes (three or more sets of optical members are provided). For example, in the case of three eyes, one of three images (left image, middle image, and right image) that have been captured synchronously becomes a reference image, and the remaining two are treated as non-reference images. In the embodiment (a part of the second embodiment and the third embodiment) using the cumulative adjustment parameter (see FIGS. 11A to 11C), the left image, the middle image, and the right image (three optical members) Corresponding to each of the three), three cumulative adjustment parameters are stored in the main memory 20.

  In the above-described embodiment, the example in which the CPU 1 (or the digital image processing circuit 21) executes determination of the reference line, calculation of the conversion parameter, calculation of the adjustment parameter, and the like has been described. Wear (circuit) may be provided in the digital still camera.

1 CPU
10L Left side optical member
10R Right optical member
15R, 15L CCD
20 Main memory
21 Digital signal processing circuit
30R, 30L Optical member rotation actuator

Claims (14)

At least two imaging means including a solid-state imaging device, which are provided at a position where a common area can be imaged and which image a subject from different positions
Image rotating means for independently rotating at least two image data obtained by the imaging means;
With respect to a reference image represented by the reference image data which is one of said at least two image data, detects a tilt-out shift angle of the non-reference image represented by the non-reference image data which is the remaining image data Tilt deviation detecting means ,
First inclination control means angular amount the non-reference picture image corresponding to the inclination angular deviation amount detected by the inclination shift detecting means for controlling said image rotation means to rotate,
A cumulative value storage means for storing a cumulative value of an inclination angle amount used in the previous image data rotation for each of image data representing at least two subject images obtained by the imaging means; and
Prior to the control of the image rotation means by the first inclination control means, the image is so rotated that the non-reference image is rotated by an angle corresponding to the cumulative value of the tilt deviation angle amount stored in the cumulative value storage means. Second tilt control means for controlling the rotation means;
A compound eye imaging apparatus. For each of at least two image data obtained by the imaging means, a line segment detecting means for detecting a straight line segment included in the image represented by the image data, and a straight line detected by the line segment detecting means Of the minutes, a reference line determination is made to determine, as a reference line, a straight line segment that is closest to either the horizontal direction or the vertical direction corresponding to the horizontal pixel direction and the vertical pixel direction included in the solid-state imaging device, respectively. With means,
The inclination deviation detecting means is
An image including the reference line determined by the reference line determination means is selected as the reference image;
The tilt deviation detecting means further includes
Detecting an inclination angle amount of each corresponding line corresponding to the reference line in the non-reference image with respect to the reference line in the reference image;
The compound eye imaging device according to claim 1. When all of the straight line segments detected by the line segment detection means are not oriented in the direction of the horizontal vicinity angle range and the vertical vicinity angle range, the inclination deviation angle amount by the inclination deviation detection means A stop means for stopping the detection of the image rotation means and the control of the image rotation means by the first tilt control means,
The compound eye imaging device according to claim 2. A reference line / corresponding line judging means for judging whether or not the directions of the reference line and the corresponding line match,
When the reference line / corresponding line determination means determines that the reference lines and the corresponding lines are in the same direction, the inclination deviation angle by the inclination deviation detection means for the non-reference image including the corresponding lines in the matching direction. Stop means for stopping detection of the amount and control of the image rotation means by the first tilt control means;
The compound eye imaging device according to claim 2. The first inclination control means rotates the image so that the non-reference image is rotated by an angle amount obtained by multiplying the inclination deviation angle amount detected by the inclination detection means by a predetermined coefficient smaller than 1. Control means,
The compound eye imaging device according to claim 1. The first tilt control means is an angle amount of a difference between the tilt deviation angle amount detected by the tilt detection means and an angle amount obtained by multiplying the tilt deviation angle amount by a predetermined coefficient smaller than 1. Controlling the image rotation means so that the non-reference image is further rotated;
The compound eye imaging device according to claim 5.   The compound-eye imaging apparatus according to claim 1, wherein the image rotation means is an imaging means rotation actuator that independently rotates each of the imaging means. The image rotation means is image data rotation processing means for rotating and converting image data obtained by the imaging means.
The compound eye imaging device according to claim 1. Inclination adjustment flag storage means for storing an inclination adjustment flag;
Inclination adjustment flag control means for turning on / off the inclination adjustment flag, and when the inclination adjustment flag is on, detection of the inclination deviation angle amount by the inclination deviation detection means, and the above-mentioned by the first inclination control means. Image rotation permission means for allowing control of the image rotation means;
The compound eye imaging device according to claim 1, further comprising: The inclination adjustment flag control means turns off the inclination adjustment flag stored in the inclination adjustment flag storage means in response to the image data being rotated by the image rotation means.
The compound eye imaging device according to claim 9 . The inclination adjustment flag control means turns on the inclination adjustment flag stored in the inclination adjustment flag storage means when the compound eye imaging device is powered on;
The compound eye imaging device according to claim 9 . The inclination adjustment flag control means turns on the inclination adjustment flag stored in the inclination adjustment flag storage means when the setting of the compound eye imaging device is changed;
The compound eye imaging device according to claim 9 . The inclination adjustment flag control means turns on the inclination adjustment flag stored in the inclination adjustment flag storage means when a predetermined time has elapsed after the image data is rotated by the image rotation means;
The compound eye imaging device according to claim 9 . A control method for a compound eye imaging device having at least two imaging means including a solid-state imaging device, which is provided at a position where a common area can be imaged and images a subject from different positions,
The non-reference image data which is the remaining image data with respect to the reference image represented by the reference image data which is one of the at least two image data obtained by the at least two image pickup means by the inclination deviation detection means. detecting the inclination-out shift angle of the non-reference image represented by,
First gradient control means, so that the angle component corresponding to the inclination angular deviation amount detected by the inclination shift detecting means above the non-reference picture image is rotated, at least two of the image data obtained by the image pickup means, respectively image rotation means for rotating independently control and control,
A cumulative value storage means stores, for each of the image data representing at least two subject images obtained by the imaging means, a cumulative value of the tilt angle amount used in the previous image data rotation,
The second inclination control means performs the non-reference by an angle corresponding to the accumulated value of the inclination deviation angle amount stored in the accumulated value storage means before the control of the image rotation means by the first inclination control means. Controlling the image rotation means so that the image rotates,
Control method of compound eye imaging apparatus.

Images