JP2010002280A - Imaging device, distance measuring device, and parallax calculating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device, etc. capable of calculating a minute parallax with high accuracy, and suppressing an increase in circuit scale for parallax calculation. <P>SOLUTION: This distance measuring device includes: an image cut-out section 141 which acquires the positions of optical axes respectively, concerning a standard image and a reference image created by a standard imaging system and a reference imaging system, and sets plane coordinates of the standard image and the reference image so that a position separated by a predetermined number of pixels from the position of the optical axis of the reference image in the direction of approaching to the center of the optical axis of a lens which the standard imaging system has, has the same coordinate values as those of the position of the optical axis of the standard image; a parallax calculation section 142 which calculates an evaluation value between the images of a partial region of the standard image and a partial region of the reference image on the basis of the set coordinates, and calculates a temporary parallax from the calculated evaluation value; and a parallax correction section 143 for calculating a parallax by subtracting the predetermined number of pixels from the calculated temporary parallax. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a parallax calculation method that have a plurality of imaging systems, calculate parallax generated between the imaging systems, and a distance measuring apparatus that measures the distance from the calculated parallax to a subject. The present invention relates to an imaging device, a parallax calculation method, and a distance measuring device that can calculate accurate parallax with high accuracy.

  As a conventional imaging device capable of measuring parallax between a plurality of viewpoints, there is an imaging device disclosed in Patent Document 1. The imaging apparatus of Patent Literature 1 measures parallax between viewpoints using image information of a plurality of viewpoints. The imaging device of Patent Document 1 treats image information of one viewpoint as a reference image, and treats image information of another viewpoint as a reference image. And the imaging device of patent document 1 calculates | requires the similarity of the image feature of a reference image with respect to a base image. Then, the imaging apparatus of Patent Document 1 has a similarity h (d) of the shift amount d that minimizes the similarity and a similarity (h (d−1)) of the front and rear shift amounts (d−1 and d + 1). And h (d + 1)), the parallax d ′ is obtained with a resolution finer than one pixel as shown in the following formula (1).

  Here, in a general imaging apparatus that calculates only a similarity with a shift amount of 0 or more, when the shift amount that minimizes the similarity is 0, the shift amount d− before the shift amount that gives the minimum similarity is d−. The similarity h (-1) at 1 cannot be obtained. Therefore, since a general imaging device cannot perform the calculation of Expression (1), the parallax d ′ cannot be obtained with a resolution finer than one pixel.

Therefore, the imaging apparatus of Patent Document 1 changes the shift amount from −1 to a preset positive value, and obtains the similarity in each of the changed shift amounts. As a result, the imaging apparatus disclosed in Patent Document 1 has a similarity h (−1) at the shift amount d−1 before the shift amount that gives the minimum similarity even when the shift amount d that minimizes the similarity is 0. ). Therefore, the imaging apparatus of Patent Document 1 can obtain the parallax d ′ with a resolution finer than one pixel using Expression (1). That is, the imaging apparatus of Patent Document 1 can obtain the parallax d ′ with fine resolution even when the parallax is near zero.
JP 2003-346130 A

  However, since the conventional imaging device described in Patent Document 1 needs to perform a parallax calculation using a shift amount of a negative number (−1), the calculation becomes complicated and the circuit scale for realizing the calculation There was a problem that became larger.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an imaging device and the like that can calculate a minute parallax with high accuracy and suppress an increase in circuit scale for parallax calculation. And

  In order to achieve the above object, an imaging apparatus according to the present invention includes a standard imaging system and one or more reference imaging systems each having a lens and an imaging region corresponding to the lens, and images the same subject. An imaging apparatus for calculating a parallax generated between the reference imaging system and the reference imaging system when the reference imaging system generates a reference image and a reference image generated by the reference imaging system and the reference imaging system. And optical axis position acquisition means for acquiring an optical axis position that is a position corresponding to the optical axis center of the lens respectively included in the reference imaging system, and light of the lens included in each of the standard imaging system and the reference imaging system A position obtained by moving a predetermined number of pixels from the optical axis position of the reference image in a direction approaching the optical axis center of the lens included in the reference imaging system in a base line direction that is a direction of a straight line connecting the axial centers. The coordinate setting means for setting the plane coordinates of the reference image and the reference image so that the optical axis position of the reference image has the same coordinate value, and the coordinates set by the coordinate setting means, Temporary parallax calculation means for calculating an evaluation value representing a degree of similarity between the partial area of the reference image and the partial area of the reference image, and calculating temporary parallax from the calculated evaluation value; and the temporary parallax calculation And a parallax calculating unit that calculates a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated by the unit. Specifically, the temporary parallax calculation means includes a reference block selection means for selecting a reference block that is a partial area of the reference image, and the reference indicated by the same coordinate value as the coordinate value indicating the reference block. Select an area that is shifted by a predetermined shift amount as a reference block from a partial area of the image to a direction away from the optical axis center of the lens of the reference imaging system in the baseline direction, and select the selected reference Using the evaluation value calculating means for calculating the evaluation value with the reference block for each block, and using an interpolation formula based on a plurality of evaluation values among the calculated evaluation values, the shift amount that maximizes the similarity is Temporary parallax calculation means for calculating as temporary parallax is provided.

  As a result, a minute parallax can be calculated with high accuracy without using a negative shift amount, so that it is possible to calculate a minute parallax with high accuracy and to suppress an increase in circuit scale for parallax calculation. .

  Further, the coordinate setting means includes a position moved by one pixel from the optical axis position of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system in the baseline direction, and an optical axis of the reference image The plane coordinates are preferably set so that the position has the same coordinate value.

  As a result, since it is only necessary to enlarge the reference image by one pixel, it is possible to suppress an increase in circuit scale due to an increase in the storage area used in the parallax calculation.

  Further, it is preferable that the reference image is an image larger than the reference image by the predetermined number of pixels toward the direction closer to the optical axis center of the lens of the reference imaging system in the baseline direction.

  As a result, when selecting a reference block, it is not necessary to check whether or not the reference block can be selected, so that the parallax calculation can be simplified and an increase in circuit scale for the parallax calculation is suppressed. Is possible.

  The distance measuring device according to the present invention includes a reference imaging system and one or more reference imaging systems each having a lens and an imaging region corresponding to the lens, and the reference imaging when imaging the same subject. A distance measuring device that calculates a distance to the subject based on a parallax generated between a system and the reference imaging system, wherein a reference image and a reference image generated by the standard imaging system and the reference imaging system An optical axis position acquisition unit that acquires an optical axis position that is a position corresponding to an optical axis center of the lens of each of the standard imaging system and the reference imaging system, and the standard imaging system and the reference imaging system, respectively. A predetermined number of pixels from the optical axis position of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system in a base line direction that is a direction of a straight line connecting the optical axis centers of the lens with each other Coordinate setting means for setting the plane coordinates of the reference image and the reference image so that the moved position and the optical axis position of the reference image have the same coordinate value; and the coordinates set by the coordinate setting means A temporary parallax calculating means for calculating an evaluation value indicating a degree of similarity between the partial area of the reference image and the partial area of the reference image based on the calculated evaluation value; A parallax calculating unit that calculates a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated by the temporary parallax calculating unit; and the parallax calculating unit And a distance calculating means for calculating a distance to the subject using the calculated parallax.

  As a result, it is possible to increase the accuracy of the far distance accompanying the improvement of the calculation accuracy of the minute parallax, and to suppress an increase in the circuit scale for the distance calculation.

  Note that the present invention can be realized not only as such an imaging apparatus, but also as a parallax calculation method using steps characteristic of the imaging apparatus, or by executing these steps on a computer. It can also be realized as a program to be executed. Such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  According to the present invention, in an imaging apparatus that includes two or more imaging systems and calculates parallax generated between the imaging systems when the same subject is imaged, parallax calculation using a negative shift amount becomes unnecessary. Therefore, it is possible to calculate a minute parallax with high accuracy and to suppress an increase in circuit scale for parallax calculation.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a cross-sectional view showing a configuration of a distance measuring apparatus 101 according to Embodiment 1 of the present invention. As shown in FIG. 1, the distance measuring apparatus 101 includes a circuit unit 120 and a lens module unit 110 provided above the circuit unit 120.

  The lens module unit 110 includes a cylindrical lens barrel 111, an upper cover glass 112 that covers the opening of the lens barrel 111, and a lens array 113 that is provided below the upper cover glass 112 and inside the lens barrel 111. And have. The circuit unit 120 includes a substrate 121, a package 122 provided on the substrate 121, an image sensor 123, a package cover glass 124, and a system LSI (hereinafter referred to as SLSI) 125 that is a semiconductor circuit element. have.

  The lens barrel 111 has a cylindrical shape as described above, and its inner wall surface is colored with black that has been matted to prevent irregular reflection of light. The lens barrel 111 is formed by injection molding a resin.

  The upper cover glass 112 has a disc shape, is formed of an optical glass material or a transparent resin, and is fixed to the inner wall of the upper portion of the lens barrel 111 using an adhesive or the like. Further, a protective film for preventing damage due to friction and the like and an antireflection film for preventing reflection of incident light are provided on the surface of the upper cover glass 112.

  FIG. 2 is a plan view showing the configuration of the lens array 113 of the distance measuring apparatus 101 according to Embodiment 1 of the present invention. As shown in FIG. 2, the lens array 113 has a substantially disk shape and is formed of an optical glass material, a transparent resin, or the like. The lens array 113 includes a circular first lens portion 113a, a second lens portion 113b, a third lens portion 113c, and a fourth lens portion 113d arranged in a grid pattern of 2 rows and 2 columns. It is installed.

  Here, along the arrangement direction of the first to fourth lens portions 113a to 113d, the x axis and the y axis are set as shown in FIG. In the first lens unit 113 a, the second lens unit 113 b, the third lens unit 113 c, and the fourth lens unit 113 d, light incident from the subject side is emitted to the image sensor 123 side, and then on the image sensor 123. Four images are formed.

  As shown in FIG. 2, each lens unit has an optical axis center 114a, an optical axis center 114b, an optical axis center 114c, and an optical axis center 114d, which are lens centers. A direction of a straight line connecting the optical axis center 114a, the optical axis center 114b, the optical axis center 114c, and the optical axis center 114d is referred to as a baseline direction between the lenses.

  As shown in FIG. 2, the optical axis center 114a of the first lens unit 113a and the optical axis center 114b of the second lens unit 113b are separated by Dx in the x-axis direction and in the y-axis direction. Match. The optical axis center 114a of the first lens portion 113a and the optical axis center 114c of the third lens portion 113c coincide in the x-axis direction and are separated by Dy in the y-axis direction. The optical axis center 114c of the third lens portion 113c and the optical axis center 114d of the fourth lens portion 113d are separated by Dx in the x-axis direction, and the y-axis directions coincide. The optical axis center 114d of the fourth lens unit 113d and the optical axis center 114a of the first lens unit 113a are separated by Dx in the x-axis direction and separated by Dy in the y-axis direction. Hereinafter, it is assumed that Dx and Dy are equal in distance measuring apparatus 101 according to the first embodiment.

Returning to the description of each component of the distance measuring apparatus 101 shown in FIG.
The substrate 121 is formed of a resin substrate, and the lens barrel 111 is fixed to the upper surface with an adhesive or the like in contact with the bottom surface. In this manner, in the distance measuring device 101, the lens module unit 110 and the circuit unit 120 are fixed.

  The package 122 is made of a resin having a metal terminal portion, and the metal terminal portion is soldered and fixed to the upper surface of the substrate 121 inside the lens barrel 111.

  The image pickup device 123 is a solid-state image pickup device such as a CCD sensor or a CMOS sensor, and its light receiving surface has a first lens portion 113a, a second lens portion 113b, a third lens portion 113c, and a fourth lens portion. It is arranged so as to be substantially perpendicular to the optical axis 113d. Each terminal of the image sensor 123 is connected to a metal terminal at the bottom inside the package 122 by wire bonding, and is electrically connected to the SLSI 125 through the substrate 121. Light emitted from the first lens unit 113a, the second lens unit 113b, the third lens unit 113c, and the fourth lens unit 113d forms an image on the light receiving surface of the image sensor 123, and the light Information is converted into an electrical signal by a photodiode. Then, the electric signal is transferred to the SLSI 125.

  FIG. 3 is a plan view showing the configuration of the circuit unit 120 of the distance measuring apparatus 101 according to Embodiment 1 of the present invention. The package cover glass 124 has a flat plate shape, is formed of a transparent resin, and is fixed to the upper surface of the package 122 by adhesion or the like. In FIG. 3, for the sake of convenience, description of the image sensor 123 and the like seen through the package cover glass 124 is omitted.

  The SLSI 125 performs various calculations using an electrical signal input from the image sensor 123 by a method described later. The SLSI 125 communicates with the host CPU and outputs image information, distance information, and the like to the outside. The SLSI 125 is connected to a power source (for example, 3.3V) and a ground (for example, 0V).

  FIG. 4 is a plan view showing the configuration of the image sensor of distance measuring apparatus 101 according to Embodiment 1 of the present invention. As shown in FIG. 4, the imaging device 123 includes a first imaging region 123a, a second imaging region 123b, a third imaging region 123c, and a fourth imaging region 123d. These first to fourth imaging regions 123a to 123d are arranged in two rows and two columns so that the respective light receiving surfaces are substantially perpendicular to the optical axes of the first to fourth lens portions 113a to 113d. The An imaging signal is generated in the first to fourth imaging regions 123a to 123d.

  In the first embodiment, the first lens unit 113a and the first imaging region 123a are the reference imaging system. The second lens unit 113b and the second imaging region 123b, the third lens unit 113c and the third imaging region 123c, and the fourth lens unit 113d and the fourth imaging region 123d are reference imaging systems. .

  Next, the relationship between subject distance and parallax will be described. The distance measuring apparatus 101 according to the first embodiment of the present invention includes four lens units (a first lens unit 113a, a second lens unit 113b, a third lens unit 113c, and a fourth lens unit 113d). Therefore, the relative positions of the four object images formed by the four lens units change according to the subject distance.

  FIG. 5 is a diagram for explaining the position of an object image when an object at infinity is imaged in distance measuring apparatus 101 according to Embodiment 1 of the present invention. In FIG. 5, only the first lens portion 113a and the second lens portion 113b are shown in the lens array 113 for simplicity. Typical incident light L1 from the object 10 at infinity to the first lens unit 113a and typical incident light L2 from the object 10 at infinity to the second lens unit 113b Are parallel to each other (where two objects 10 are drawn in FIG. 5 but are a single object. The object 10 at infinity cannot be drawn on a finite sheet of paper. In order to clearly indicate that the incident light L1 and the incident light L2 are parallel to each other, the object 10 at infinity at two positions is drawn. Therefore, the distance between the optical axis center 114a of the first lens unit 113a and the optical axis center 114b of the second lens unit 113b, the position where the object image 11a on the image sensor 123 is formed, and the object image The distance between the position at which 11b is imaged is equal. That is, there is no parallax.

  FIG. 6 is a diagram for explaining the position of an object image when an object at a finite distance is imaged in distance measuring apparatus 101 according to Embodiment 1 of the present invention. In FIG. 6, only the first lens portion 113a and the second lens portion 113b are shown in the lens array 113 for simplicity. Representative incident light L1 from the object 12 at a finite distance to the first lens unit 113a and representative light from the object 12 at a finite distance to the second lens unit 113b Is not parallel to the incident light L2. Therefore, compared with the distance between the optical axis center 114a of the first lens unit 113a and the optical axis center 114b of the second lens unit 113b, the position and object on which the object image 13a is formed on the image sensor 123. The distance from the position where the image 13b is formed is long. That is, there is parallax.

  Here, the distance from the principal point of the first lens portion 113a to the object image 12 (subject distance) is A, the distance between the optical axes of the first lens portion 113a and the second lens portion 113b is D, and the lens portion. Let f be the focal length of 113a and 113b. In this case, a right-angled triangle with two sides A and D sandwiching a right angle in FIG. 6 is similar to a right-angled triangle with two sides f and Δ that sandwich a right angle. Therefore, the parallax value Δ is expressed by the following formula (2) using the focal length f, the optical axis distance D, and the subject distance A.

  The same relationship is established for the other lens units. In this manner, the relative positions of the four object images formed by the four lens portions 113a, 113b, 113c, and 113d change according to the subject distance. For example, as the subject distance A decreases, the parallax value Δ increases. Therefore, the subject distance A can be obtained from the parallax value Δ by solving the equation (2) for the subject distance A as shown in the following equation (3).

  Next, the functional configuration of the distance measuring apparatus 101 according to Embodiment 1 of the present invention will be described. FIG. 7 is a block diagram showing a functional configuration of the distance measuring apparatus 101 according to Embodiment 1 of the present invention. The SLSI 125 includes a system control unit 131, an image sensor driving unit 132, an imaging signal input unit 133, an input / output unit 134, an image cutout unit 141, a parallax calculation unit 142, a parallax correction unit 143, a failure detection unit 144, and a distance calculation unit. 145.

  The system control unit 131 includes a CPU (Central Processing Unit), a logic circuit, and the like, and controls the entire SLSI 125.

  The image sensor driving unit 132 includes a logic circuit and the like, generates a signal for driving the electronic shutter or the image sensor 123 according to a command from the system control unit 131, and applies a voltage corresponding to the signal to the image sensor 123. .

  The imaging signal input unit 133 includes a CDS circuit (correlated double sampling circuit), AGC (automatic gain controller), and ADC (analog / digital converter: Analog) connected in series. (Digital Converter).

  Specifically, the imaging signal input unit 133 acquires an electrical signal from the imaging element 123 and removes fixed noise of the acquired electrical signal by the CDS circuit. Further, the imaging signal input unit 133 adjusts the gain of the electrical signal from which the fixed noise has been removed by AGC. Further, the imaging signal input unit 133 converts the electrical signal whose gain is adjusted from an analog signal to a digital signal by the ADC. As described above, the imaging signal input unit 133 generates the imaging signal I0, which is a digital signal, using the electrical signal acquired from the imaging element 123.

  The input / output unit 134 receives input from an external device and outputs the distance information calculated by the distance calculation unit 145 to the external device.

  The image cutout unit 141 is an example of an optical axis position acquisition unit, and the first image pickup signal I1 that is a reference image and the second image pickup that is a reference image in the image pickup signal I0 generated by the image pickup signal input unit 133. The optical axis positions of the signal I2, the third imaging signal I3, and the fourth imaging signal I4 are acquired.

  Here, the optical axis position is a position in the image corresponding to the optical axis center of each lens, and a line (optical axis) that passes through the optical axis center of the lens and is perpendicular to the imaging surface of the image sensor 123 is the imaging surface. It is a crossing position. This optical axis position is calculated in advance by calibrating each imaging system. Information for specifying the optical axis position is stored, for example, in a storage unit (not shown).

  The image cutout unit 141 cuts out the first image pickup signal I1 from the image pickup signal I0 so that the optical axis position of the acquired first image pickup signal I1 is the center of the image. Furthermore, the image cutout unit 141 is closer to the optical axis center 114a of the lens unit 113a in the direction of the straight line (baseline direction) connecting the optical axis center 114a and the optical axis center 114b than the first imaging signal I1. The second imaging signal I2 is cut out from the imaging signal I0 in the direction so that the distance from the optical axis position of the second imaging signal I2 is a single pixel. The image cutout unit 141 also cuts out the third image pickup signal I3 and the fourth image pickup signal I4 from the image pickup signal I0 in the same manner as the second image pickup signal I2.

  The image cutout unit 141 is also an example of a coordinate setting unit, and is a position moved by one pixel from the optical axis position of the second imaging signal I2 in a direction closer to the optical axis center 114a of the lens unit 113a in the baseline direction. The xy coordinates of the first imaging signal I1 and the second imaging signal I2 are set so that the optical axis position of the first imaging signal has the same coordinate value. Further, the image cutout unit 141 sets xy coordinates for the third image pickup signal I3 and the fourth image pickup signal I4 as well as the second image pickup signal I2.

  Here, the xy coordinate is an example of a plane coordinate, and is a coordinate whose position on the plane can be specified by the x coordinate and the y coordinate.

  Note that one pixel, which is the number of pixels to be moved from the optical axis position of the second imaging signal I2, is an example, and the distance measuring device according to the present invention has a position moved by a number of pixels larger than one pixel and light of the reference image. The xy coordinates may be set so as to correspond to the axis position. In this case, the image cutout unit 141 extracts the second image pickup signal I2, the third image pickup signal I3, and the fourth image pickup signal I4 from the image pickup signal I0 so that the image is larger by the number of pixels larger than one pixel. cut.

  The parallax calculation unit 142 is an example of a temporary parallax calculation unit, and based on the coordinates set by the image cutout unit 141, a partial region of the first imaging signal I1 and a partial region of the second imaging signal I2 An evaluation value representing the degree of similarity between the images is calculated, and a temporary parallax is calculated from the calculated evaluation value.

  Specifically, the parallax calculation unit 142 is an example of a reference block selection unit, and selects a reference block that is a partial region of the first imaging signal I1.

  Further, the parallax calculation unit 142 is an example of an evaluation value calculation unit, and the optical axis in the base line direction from a partial region of the second imaging signal I2 indicated by the same coordinate value as the coordinate value indicating the reference block. An area shifted by a predetermined shift amount in a direction away from the center 114a is selected as a reference block, and an evaluation value with respect to the reference block is calculated for each selected reference block.

  Further, the parallax calculation unit 142 is an example of a temporary parallax calculation unit, and calculates a shift amount that maximizes the degree of similarity as a temporary parallax using an interpolation formula based on a plurality of evaluation values among the calculated evaluation values. To do.

  Further, the parallax calculation unit 142 also calculates a shift amount that maximizes the degree of similarity for the third imaging signal I3 and the fourth imaging signal I4, similarly to the second imaging signal I2. Then, the parallax calculation unit 142 specifies, as the temporary parallax, the shift amount that maximizes the degree of similarity of the evaluation values among the calculated shift amounts.

  The parallax correction unit 143 is an example of a parallax calculation unit. By subtracting one pixel from the temporary parallax, the first imaging signal I1, the second imaging signal I2, the third imaging signal I3, or the fourth imaging signal is calculated. The parallax generated between the imaging signal I4 and the imaging signal I4 is calculated.

  The failure detection unit 144 is an example of a parallax determination unit, and determines whether or not the parallax calculated by the parallax correction unit 143 is a negative number.

  The failure detection unit 144 is also an example of a failure information output unit, and outputs failure information indicating that the parallax calculation has failed when the parallax determination unit determines that the number is negative.

  The distance calculation unit 145 is an example of a distance calculation unit, and calculates the distance to the subject using the parallax calculated by the parallax correction unit 143.

  Next, the basic operation of the distance measuring apparatus 101 in the present embodiment configured as described above will be described.

  FIG. 8 is a flowchart showing the operation of the distance measuring apparatus 101 according to Embodiment 1 of the present invention. The distance measuring apparatus 101 is operated according to this flowchart by the system control unit 131 of the SLSI 125.

  The distance measuring device 101 starts its operation, for example, when a host CPU (not shown) commands the distance measuring device 101 to start the operation via the input / output unit 134.

  When the operation starts, the imaging signal input unit 133 converts the electrical signal input from the imaging element 123 into an imaging signal I0. Then, the imaging signal input unit 133 outputs the imaging signal I0 to the image cutout unit 141 and the input / output unit 134 (step S1020). Specifically, the imaging signal input unit 133 outputs an imaging signal I0 (x, y) in which the number of pixels in the x direction is H0 pixels and the number of pixels in the y direction is V0 pixels, to I0 (0, 0) ( (X, y) = (0, 0)), I0 (1, 0), I0 (2, 0),..., I0 (H0-2, V0-1), I0 (H0-1, V0−) Output in the order of 1).

  Next, the image cutout unit 141 cuts out the standard image and the reference image from the input imaging signal I0 (step S1100). Specifically, the image cutout unit 141 acquires information for specifying the optical axis position stored in the storage unit, for example. Then, the image cutout unit 141 uses the information for specifying the acquired optical axis position, from the image pickup signal I0, the first image pickup signal I1 that is a reference image and the second image pickup signal I2 that is a reference image. The third imaging signal I3 and the fourth imaging signal I4 are generated.

  Next, the image cutout unit 141 sets xy coordinates for the standard image and the reference image (step S1150). In this xy coordinate, the position where a predetermined number of pixels are moved from the optical axis position of the reference image in the direction of approaching the optical axis center of the lens included in the standard imaging system in the baseline direction, and the optical axis position of the standard image are the same. The coordinates are set to be coordinate values. Then, the image cutout unit 141 outputs the first imaging signal I1 that is a reference image and the second imaging signal I2, the third imaging signal I3, and the fourth imaging signal I4 that are reference images.

  Hereinafter, the processing of the image cutout unit 141 in step S1100 and step S1150 will be described in more detail with reference to FIG.

  FIG. 9 is a diagram for explaining the cutout position of the imaging signal of the distance measuring apparatus 101 according to Embodiment 1 of the present invention. In FIG. 9, the coordinates (xc1, yc1), coordinates (xc2, yc2), coordinates (xc3, yc3), and coordinates (xc4, yc4) are respectively the first lens unit 113a and the second lens in the imaging signal I0. It is the coordinate of the position (optical axis position) corresponding to the optical axis center of the lens part 113b, the 3rd lens part 113c, and the 4th lens part 113d.

  The image cut-out unit 141 has coordinates (xc1, yc1) at the position (optical axis position) corresponding to the optical axis center of the first lens unit 113a at the center, and the number of pixels in the x direction is H1 pixels. The first imaging signal I1 is cut out from the imaging signal I0 so that the number of pixels in the y direction is V1. That is, the image cutout unit 141 cuts out the first imaging signal I1 in which the number of pixels in the x direction is H1 pixels and the number of pixels in the y direction is V1 pixels as in the following formulas (4) to (6). . Furthermore, as shown in the following formulas (4) to (6), the image cutout unit 141 sets the origin to the coordinates (x01, y01) with respect to the cut out first imaging signal I1. Set the coordinates.

  Further, the image cutout portion 141 is a pixel whose center is a coordinate (xc2, yc2) of a position (optical axis position) corresponding to the optical axis center 114b of the second lens portion 113b and whose size is in the x direction. From the region where the number is H1 pixels and the number of pixels in the y direction is V1 pixels, in the x direction (baseline direction), only one pixel is positive in the negative direction (direction approaching the optical axis center 114a of the lens portion 113a). The second imaging signal I2 is cut out from the imaging signal I0 so as to be an area larger by SB pixels (in the direction away from the optical axis center 114a of the lens portion 113a). That is, the image cutout unit 141 cuts out the second imaging signal I2 in which the number of pixels in the x direction is H2 pixels and the number of pixels in the y direction is V2 pixels as in the following formulas (7) to (11). .

  Further, the image cutout unit 141 sets the xy coordinates so that the coordinates of the origin are the coordinates (x02, y02) with respect to the cut out second imaging signal I2. That is, the image cutout unit 141 is moved by one pixel from the optical axis position (xc2, yc2) of the second imaging signal I2 toward the optical axis center 114a of the lens unit 113a in the baseline direction. 1, yc2) and the xy coordinates are set so that the optical axis position (xc1, yc1) of the first imaging signal has the same coordinate value.

  In addition, the image cutout unit 141 has coordinates (xc3, yc3) at the position (optical axis position) corresponding to the optical axis center 114c of the third lens unit 113c, and the size is the number of pixels in the x direction. Is a positive direction (only one pixel in the negative direction (direction approaching the optical axis center 114a of the lens portion 113a) in the y direction (baseline direction) from the region where the number of pixels in the y direction is V1 pixels. The third imaging signal I3 is cut out from the imaging signal I0 so as to be an area larger by the SB pixel in the direction away from the optical axis center 114a of the lens unit 113a. That is, the image cutout unit 141 cuts out the third imaging signal I3 in which the number of pixels in the x direction is H3 pixels and the number of pixels in the y direction is V3 pixels as in the following formulas (12) to (16). .

  Further, the image cutout unit 141 sets the xy coordinates so that the coordinates of the origin are the coordinates (x03, y03) with respect to the cut out third imaging signal I3. That is, the image cutout unit 141 is moved by one pixel from the optical axis position (xc3, yc3) of the third imaging signal I3 toward the optical axis center 114a of the lens unit 113a in the base line direction. The xy coordinates are set so that yc3-1) and the optical axis position (xc1, yc1) of the first imaging signal have the same coordinate value.

  Further, the image cutout part 141 has the coordinates (xc4, yc4) of the position (optical axis position) corresponding to the optical axis center 114d of the fourth lens part 113d, and the size is the number of pixels in the x direction. From the region where H1 pixel and the number of pixels in the y direction are V1 pixels, in the x direction, only one pixel in the negative direction, only SB pixels in the positive direction, and in the y direction, only one pixel in the negative direction, positive The fourth imaging signal I4 is cut out from the imaging signal I0 so as to be an area larger by SB pixels in the direction of. That is, the image cutout unit 141 has a square root of 2 in a direction closer to the optical axis center 114a of the lens unit 113a in the direction (baseline direction) in which the x axis is rotated by −45 degrees compared to the first imaging signal I1. The fourth imaging signal I4 is cut out from the imaging signal I0 so as to be an area larger by the pixel. That is, as in the following formulas (17) to (21), the fourth imaging signal I4 in which the number of pixels in the x direction is H4 pixels and the number of pixels in the y direction is V4 pixels is cut out.

  Further, the image cutout unit 141 sets the xy coordinates so that the coordinates of the origin are the coordinates (x04, y04) with respect to the cut out fourth imaging signal I4. That is, the image cutout unit 141 is moved by one pixel from the optical axis position (xc4, yc4) of the fourth imaging signal I4 toward the optical axis center 114a of the lens unit 113a in the base line direction (xc4- 1, yc4-1) and xy coordinates are set so that the optical axis position (xc1, yc1) of the first imaging signal has the same coordinate value.

Returning to the flowchart of FIG.
Next, the parallax calculation unit 142, the parallax correction unit 143, and the distance calculation unit 145 perform distance calculation (step S1200).

  Hereinafter, the details of the process of step S1200 will be described with reference to FIG. FIG. 10 is a flowchart showing an operation related to the distance calculation of the distance measuring apparatus 101 according to the first embodiment of the present invention. The flowchart of FIG. 10 shows the details of the process of step S1200 in FIG.

  First, the parallax calculation unit 142 selects a reference block that is a partial image region of the imaging signal I1 (step S1220).

  Here, details of the reference block selected by the parallax calculation unit 142 in step S1220 will be described with reference to FIG. FIG. 11 is a diagram for explaining a reference block selected by the parallax calculation unit 142 in the distance measuring apparatus 101 according to Embodiment 1 of the present invention. As illustrated in FIG. 11, the reference block selected by the parallax calculation unit 142 is an image in which the number of pixels in the x direction is HB pixels and the number of pixels in the y direction is VB pixels on the first imaging signal I1. . When step S1220 is executed for the first time after step S1150 (FIG. 8) is executed, the parallax calculation unit 142 selects a reference block whose block center is (HB / 2, VB / 2). Thereafter, when step S1220 is executed, the parallax calculation unit 142 selects a reference block in which the center of the block is shifted to the right by one pixel at a time. In addition, when the parallax calculation unit 142 selects the rightmost block (the block whose center is (H1-HB / 2, 0), (H1-HB / 2, 1),...) In FIG. The parallax calculation unit 142 selects the block at the left end of the row below one pixel (the block whose center is (HB / 2, 1), (HB / 2, 2),...) As the reference block. That is, the parallax calculation unit 142 sets an index (i, j), selects a block whose center is (i, j), the number of pixels in the x direction is HB pixels, and the number of pixels in the y direction is VB pixels as a reference block. To do. After the step S1150 (FIG. 8) is executed for the index (i, j), when the parallax calculation unit 142 executes the process of step S1220 for the first time, the index (i, j) is set to (HB / 2, VB). / 2). After the initialization, the parallax calculation unit 142 increases i by 1 each time the parallax calculation unit 142 executes the process of step S1220. When the previous i is H1-HB / 2, the parallax calculation unit 142 sets i to 0 and j to increase by 1.

Returning to the flowchart of FIG.
Next, the parallax calculation unit 142 uses the first imaging signal I1 that is a reference image, and the second imaging signal I2, the third imaging signal I3, and the fourth imaging signal I4 that are reference images. Thus, the parallax Δ (temporary parallax) in each pixel on the reference image is calculated, and the calculated parallax Δ (temporary parallax) is output (step S1230). In the calculation of the parallax Δ (temporary parallax), the parallax calculation unit 142 first calculates the parallax (temporary parallax) and the parallax reliability in the first imaging signal I1 and the second imaging signal I2.

  Hereinafter, the calculation of the parallax (temporary parallax) and the parallax reliability in the first imaging signal I1 and the second imaging signal I2 will be described in detail. First, the parallax calculation unit 142 calculates the parallax evaluation value R12 (kx) between the first imaging signal I1 and the second imaging signal I2. Here, kx is a shift amount indicating how much the reference block is shifted and selected with respect to a partial area of the reference image having the same coordinate value as that of the base block. The parallax calculation unit 142 changes kx as kx = 0, 1, 2,..., SB + 1.

  FIG. 12 illustrates a calculation area of a parallax evaluation value in the parallax calculation when the first imaging signal I1 and the second imaging signal I2 are used in the distance measuring apparatus 101 according to Embodiment 1 of the present invention. FIG.

  The area indicated by the reference block I1b is a block indicated by the index (i, j) in the first imaging signal I1. That is, the reference block I1b is a block in the first imaging signal I1 whose center coordinates are (i, j), the number of pixels in the x direction is HB pixels, and the number of pixels in the y direction is VB pixels.

  In addition, the area indicated by the reference block I2b is a block of an area that is shifted by kx in the x direction from the block having the same coordinate value as the coordinate value of the standard block I1b in the second imaging signal I2. That is, the reference block I2b is a block in which the center coordinates are (i + kx, j), the number of pixels in the x direction is HB pixels, and the number of pixels in the y direction is VB pixels in the second imaging signal I2.

  Here, when the shift amount kx is changed from 0 to SB + 1 (kx = 0 to SB + 1), the parallax calculation unit 142 calculates the sum of absolute value differences (SAD: Sum of Absolute Differences) shown in the following formula (22). Is calculated. Here, the parallax calculation unit 142 treats the calculated SAD as the parallax evaluation value R12 (kx). That is, the parallax calculation unit 142 calculates the parallax evaluation value R12 (kx) using the first imaging signal I1 as a reference image and the second imaging signal I2 as a reference image.

  In Expression (22), ΣΣ represents the total sum of each pixel in the block indicated by the index (i, j). That is, in Expression (22), x changes from i−HB / 2 to i + HB / 2-1, and y changes from j−VB / 2 to j + VB / 2-1.

  The parallax evaluation value R12 (kx) is an image of the reference block at the center coordinates (i, j) indicated by the index (i, j) in the first imaging signal I1 and the index (i in the second imaging signal I2. , J) indicates how much the image of the reference block in the region shifted by kx in the x direction from the block of the central coordinates (i, j) indicated by (i, j) is correlated (similar), and the value is small The higher the correlation, the greater the degree of similarity.

  FIG. 13 shows the relationship between the shift amount and the parallax evaluation value in the parallax calculation when the first imaging signal I1 and the second imaging signal I2 of the distance measuring apparatus 101 according to Embodiment 1 of the present invention are used. It is a figure explaining. As shown in FIG. 13, the parallax evaluation value R12 (kx) varies depending on the value of the shift amount kx, and the parallax evaluation value R12 (kx) becomes a minimum value when the shift amount kx = Δ. That is, the central coordinates indicated by the index (i, j) in the second imaging signal I2 with respect to the reference block having the central coordinates (i, j) indicated by the index (i, j) in the first imaging signal I1. The reference block in the region shifted by the shift amount Δ in the x direction from the block (i, j) is the block having the highest correlation (the highest degree of similarity). Therefore, the parallax calculation unit 142 shifts the shift amount at which the parallax evaluation value R12 (kx) between the first imaging signal I1 and the second imaging signal I2 in the block indicated by the index (i, j) is the minimum value. The quantity Δ is specified.

  Further, the parallax calculation unit 142 calculates a parallax value Δ12 (i, j) (provisional parallax) in the xy coordinates using an equiangular fitting interpolation formula as shown in the following formula (23). As a result, the parallax calculation unit 142 can obtain parallax (temporary parallax) with a resolution less than the shift amount unit (one pixel).

  Further, the parallax calculation unit 142 uses the parallax evaluation value R12 (Δ) as shown in the following formula (24) to calculate the first imaging signal I1 and the second imaging signal I2 at the coordinates (i, j). The parallax reliability C12 (i, j) is calculated.

  Next, the parallax calculation unit 142 obtains the parallax (temporary parallax) and the parallax reliability between the first imaging signal I1 and the third imaging signal I3 in the same manner as described above. However, the parallax calculator 142 shifts the reference block in the third imaging signal I3 in the direction of a straight line connecting the optical axis center of the first lens unit 113a and the optical axis center of the third lens unit 113c. Change to a certain y direction. The shift amount is ky.

  Specifically, the parallax calculation unit 142, as shown in the following formula (25), the parallax evaluation value R13 between the first imaging signal I1 and the third imaging signal I3 in the block indicated by the index (i, j). Find (ky). That is, the parallax calculation unit 142 calculates the parallax evaluation value R13 (ky) by handling the first imaging signal I1 as a reference image and handling the third imaging signal I3 as a reference image. Then, the parallax calculation unit 142 specifies the shift amount Δ at which the parallax evaluation value R13 (ky) is the minimum value. Further, the parallax calculation unit 142 uses an equiangular fitting interpolation formula as shown in the following formula (26) to calculate the first imaging signal I1 and the third imaging signal I3 at the coordinates (i, j). A parallax value Δ13 (i, j) (temporary parallax) in the xy coordinates is calculated. Further, the parallax calculation unit 142 uses the parallax evaluation value R13 (Δ) as in the following equation (27), and calculates the first imaging signal I1 and the third imaging signal I3 at the coordinates (i, j). The parallax reliability C13 (i, j) is calculated.

  Next, the parallax calculation unit 142 obtains the parallax (temporary parallax) and the parallax reliability between the first imaging signal I1 and the fourth imaging signal I4 in the same manner as described above. However, the parallax calculation unit 142 connects the reference block in the fourth imaging signal I4 in an oblique direction (connects the optical axis center 114a of the first lens unit 113a and the optical axis center 114d of the fourth lens unit 113d). Change to the direction of the straight line). The shift amount is kx in the x direction and kx in the y direction.

  Specifically, the parallax calculation unit 142, as shown in the following formula (28), the parallax evaluation value R14 between the first imaging signal I1 and the fourth imaging signal I4 in the block indicated by the index (i, j). Find (kx). That is, the parallax calculation unit 142 calculates the parallax evaluation value R14 (kx) by handling the first imaging signal I1 as a reference image and handling the fourth imaging signal I4 as a reference image. Then, the parallax calculation unit 142 specifies the shift amount Δ that the parallax evaluation value R14 (kx) gives the minimum value. Further, the parallax calculation unit 142 uses the equiangular fitting interpolation formula as in the following formula (29) to calculate the first imaging signal I1 and the fourth imaging signal I4 at the coordinates (i, j). A parallax value Δ14 (i, j) (temporary parallax) in the xy coordinates is calculated. Further, the parallax calculation unit 142 uses the parallax evaluation value R14 (Δ) as shown in the following equation (30) to calculate the first imaging signal I1 and the fourth imaging signal I4 at the coordinates (i, j). The parallax reliability C14 (i, j) is calculated.

  Then, the parallax calculation unit 142 selects the most reliable parallax value (temporary parallax) by comparing the three parallax reliability levels. That is, the parallax calculation unit 142 compares the three parallax reliability C12 (i, j), C13 (i, j), and C14 (i, j) as shown in the following formula (31), thereby obtaining the most A parallax value (temporary parallax) corresponding to a parallax reliability having a small value is selected. For example, when the parallax reliability C12 (i, j) is the smallest value, the parallax calculation unit 142 converts the parallax value Δ12 (i, j) into the parallax value Δ (i, j) ( Select as temporary parallax. For example, when the parallax reliability C13 (i, j) is the smallest value, the parallax value Δ13 (i, j) is set as the parallax value Δ (i, j) (provisional parallax) at the coordinates (i, j). select. For example, when the parallax reliability C14 (i, j) is the smallest value, the parallax value Δ14 (i, j) is set as the parallax value Δ (i, j) (provisional parallax) at the coordinates (i, j). select.

  In the first embodiment, the parallax calculation unit 142 uses the absolute value difference sum (Equation (22), Equation (25), Equation (28)) for the parallax reliability (C12, C13, C14). Although calculated, for example, it may be calculated using a normalized correlation coefficient. In this case, the parallax calculation unit 142 selects the parallax value (temporary parallax) corresponding to the parallax reliability having the largest value because the parallax reliability having the largest value is the most reliable parallax reliability.

Returning to the flowchart of FIG.
Next, the parallax correction unit 143 corrects the parallax value Δ (temporary parallax) calculated by the parallax calculation unit 142, and outputs a corrected parallax value that is a corrected parallax value.

  As shown in FIG. 9, in the first imaging signal I1, which is the reference image, the coordinates (xc1, yc1) of the position (optical axis position) corresponding to the optical axis center 114a and the coordinates (x01, y01) of the origin The difference is (H1 / 2, V1 / 2). On the other hand, in the second imaging signal I2 that is the reference image, the difference between the coordinates (xc2, yc2) of the position (optical axis position) corresponding to the optical axis center 114b and the coordinates (x02, y02) of the origin is (H1 / 2 + 1, V1 / 2). That is, the coordinate value of the optical axis position in the second imaging signal I2 (reference image) is in the x direction (reference imaging system and reference) compared to the coordinate value of the optical axis position in the first imaging signal I1 (reference image). It is larger by one pixel in the negative direction (direction approaching the optical axis center of the reference imaging system) in the negative direction of the straight line connecting the optical axis centers of the imaging system. Therefore, the parallax value Δ (i, j) (temporary parallax) is larger by one pixel than the original parallax.

  Further, in the third imaging signal I3 which is the reference image, the difference between the coordinates (xc3, yc3) of the position (optical axis position) corresponding to the optical axis center 114c and the coordinates (x03, y03) of the origin is (H1 / 2, V1 / 2 + 1). That is, the coordinate value of the optical axis position in the third imaging signal I3 (reference image) is in the y direction (reference imaging system and reference) compared to the coordinate value of the optical axis position in the first imaging signal I1 (reference image). It is larger by one pixel in the negative direction (direction approaching the optical axis center of the reference imaging system) in the negative direction of the straight line connecting the optical axis centers of the imaging system. Therefore, the parallax value Δ (i, j) (temporary parallax) is larger by one pixel than the original parallax.

  In the fourth imaging signal I4 as the reference image, the difference between the coordinates (xc4, yc4) of the position (optical axis position) corresponding to the optical axis center 114d and the coordinates (x04, y04) of the origin is (H1 / 2 + 1, V1 / 2 + 1). That is, the coordinate value of the optical axis position in the fourth imaging signal I4 (reference image) is rotated by −45 degrees relative to the coordinate value of the optical axis position in the first imaging signal I1 (reference image). Is larger by one pixel in the negative direction (direction approaching the optical axis center of the standard imaging system) in the negative direction (direction of a straight line connecting the optical axis centers of the standard imaging system and the reference imaging system). That is, the coordinate value of the optical axis position in the fourth imaging signal I4 (reference image) is one more negative in the x direction than the coordinate value of the optical axis position in the first imaging signal I1 (reference image). Only one pixel is larger by one pixel in the negative direction of the y direction. Therefore, the parallax value Δ (i, j) (temporary parallax) is larger by one pixel than the original parallax.

  Here, “one pixel larger in a predetermined direction” means that the distance of one cycle is larger when the center of the pixel periodically appears on a straight line in the predetermined direction.

  Thus, since the parallax value Δ calculated by the parallax calculation unit 142 is larger than the original parallax, the parallax correction unit 143 calculates the original parallax by correcting the parallax value Δ (temporary parallax). . For example, when the parallax value Δ12 (i, j), the parallax value Δ13 (i, j), or the parallax value Δ14 (i, j) is selected as the parallax value Δ (temporary parallax), the following equation (32) ), The parallax correction unit 143 subtracts one pixel from the parallax value Δ (i, j) at the coordinates (i, j), thereby correcting the original parallax value at the coordinates (i, j). A parallax value Δa (i, j) is calculated.

  Next, the failure detection unit 144 detects a failure in the parallax calculation using the parallax value corrected by the parallax correction unit 143. If a failure is detected, the failure detection unit 144 sets the value of the failure flag DDE to 1 and outputs the result.

  As shown in FIG. 5, since the parallax is the smallest value 0 when the subject distance is infinite, the parallax value does not become a negative number. Therefore, when the parallax value corrected by the parallax correction unit 143 becomes a negative number, the failure detection unit 144 determines that the parallax calculation has failed. Specifically, as shown in the following equation (33), the failure detection unit 144 sets the failure flag DDE to 0 (DDE) when the corrected parallax value Δa (i, j) at the coordinate (i, j) is 0 or more. = 0). On the other hand, the failure detection unit 144 sets the failure flag DDE to 1 (DDE = 1) when the corrected parallax value Δa (i, j) at the coordinates (i, j) is smaller than 0. Note that this failure may be caused by noise superimposed on the imaging signal.

  Next, the distance calculation unit 145 calculates a distance using the corrected parallax value Δa and the failure flag DDE. Then, the distance calculation unit 145 outputs the calculated distance data DIS (step S1260). Specifically, the distance calculation unit 145 substitutes the corrected parallax value Δa (i, j) calculated by the parallax correction unit 143 and the like into the equation (3), so that the distance data DIS (i, j) Is calculated. That is, the distance calculation unit 145 calculates the distance data DIS (i, j) as shown in the following formula (34).

  Note that when the failure flag DDE (i, j) is 0, since the parallax is correctly obtained, the distance calculation unit 145 performs the above calculation. On the other hand, when the failure flag DDE (i, j) is 1, since the parallax is not obtained correctly, the distance calculation unit 145 does not perform the above calculation. For example, the distance calculation unit 145 sets the distance data DIS (i, j) to 0.

  In Expression (34), f is the focal length, Dx is the distance between the optical axes in the x direction, and p is the interval between the light receiving elements of the image sensor 123. Further, in Equation (34), since the corrected parallax value Δa is in units of pixels, the corrected parallax value Δa is multiplied by p so that the corrected parallax value Δa becomes the same unit system as the focal length f and the like. Has been converted.

  Next, the distance calculation unit 145 determines whether or not all blocks in the reference image have been selected (step S1270). Here, when all the blocks have been selected (when index (i, j) is (H1-HB / 2, V1-VB / 2)) (Yes in step S1270), step S1910 in FIG. Processing is executed. On the other hand, when all the blocks are not selected (when the index (i, j) is not (H1-HB / 2, V1-VB / 2)) (No in step S1270), the process of step S1220 is executed again. Is done. That is, in step S1220, index (i, j) is initialized to (HB / 2, VB / 2), i is changed from HB / 2 to H1-HB / 2, and j is changed from VB / 2 to V1-VB /. While changing to 2, the operations of step S1230, step S1240, step S1250, and step S1260 are repeated.

Returning to the flowchart of FIG.
The input / output unit 134 outputs the image data, distance data, and failure information data to the outside of the distance measuring apparatus 101 (step S1910). Specifically, the input / output unit 134 outputs the imaging signal I0 or the first imaging signal I1 as image data. The input / output unit 134 outputs distance data DIS as distance data. Further, the input / output unit 134 outputs a failure flag DDE as failure information data.

  The system control unit 131 determines whether to end the operation (step S1920). For example, the system control unit 131 requests an instruction to end the operation by communicating with a host CPU (not shown) via the input / output unit 134. When the system control unit 131 receives an end command from the host CPU (Yes in step S1920), the process ends. On the other hand, when the system control unit 131 does not receive an end command from the host CPU (No in step S1920), the process in step S1020 is executed. That is, unless the system control unit 131 receives an end command from the host CPU, the processes of step S1020, step S1100, step S1150, step S1200, and step S1910 are repeated.

  The distance measuring apparatus 101 according to Embodiment 1 has the following effects by being configured and operating as described above.

  First, the effects of the distance measuring apparatus 101 according to the first embodiment will be described with reference to FIGS. 14A to 14D are diagrams for explaining the effect of the distance measuring apparatus 101 according to the first embodiment.

  As shown in FIGS. 14A and 14B, when the parallax between the first imaging signal I1 that is the reference image and the second imaging signal I2 that is the reference image is very small, Since the distance measuring device shifts the reference block to be selected from the same position as the reference block (position of the same distance and direction from each optical axis position), the parallax evaluation value R12 (with a minimum shift amount (0)) kx) is minimized. For this reason, when the parallax is very small (when the shift amount that minimizes the parallax evaluation value is 0), a general distance measuring device has a parallax evaluation value at a shift amount that is smaller than the shift amount of the parallax evaluation value that is minimum. The resolution and accuracy of the parallax cannot be increased by the interpolation function using.

  On the other hand, as shown by the dotted vertical axis in FIG. 14D, the conventional distance measuring device (for example, the imaging device of Patent Document 1) calculates the parallax evaluation value R12 (kx) up to a negative shift amount. By doing so, the resolution and accuracy of the parallax can be increased using the interpolation function. However, since the conventional distance measuring device needs to set the shift amount to a negative number, the scale of the circuit for executing the parallax calculation increases.

  Therefore, as shown by the solid vertical axis in FIG. 14D, the distance measuring apparatus 101 according to the first embodiment shifts the reference block to be selected from a position one pixel smaller than the reference block. As a result, the distance measuring apparatus 101 according to the first embodiment increases the resolution and accuracy of parallax in minute parallax, and suppresses an increase in circuit scale for parallax calculation because the shift amount is always a positive number. It becomes possible to do.

  Further, the distance measuring device 101 according to the first embodiment moves one pixel from the optical axis position of the reference image in the direction of approaching the optical axis center 114a of the lens unit 113a that is a lens included in the reference imaging system in the baseline direction. The xy coordinates are set so that the position thus obtained is the same as the optical axis position of the reference image. Therefore, the distance measuring apparatus 101 can always set the shift amount to a positive number only by calculating the parallax evaluation value R12 (kx) while shifting the reference block from the same coordinate value as the coordinate value of the base block. Become. Then, the distance measuring apparatus 101 performs a parallax calculation similar to the parallax calculation performed by a general distance measuring apparatus, and then subtracts one pixel from the obtained parallax (temporary parallax), thereby reducing the resolution of the parallax in a minute parallax. In addition, the accuracy can be increased. That is, since the distance measuring apparatus 101 can use the parallax calculation of a general distance measuring apparatus almost as it is, it is possible to suppress an increase in circuit scale for the parallax calculation.

  As shown in FIG. 14C, in the distance measuring apparatus 101 according to the first embodiment, the reference image is one pixel larger in the direction closer to the optical axis center 114a of the lens unit 113a than the reference image. . Therefore, in the distance measuring device 101, the base line direction is set to the optical axis center 114a of the lens unit 113a from the same position (position of the same distance and direction from each optical axis position) with respect to all pixels of the reference image. There is a reference image at a position moved by one pixel in the approaching direction. Therefore, the distance measuring apparatus 101 does not need to confirm whether or not the reference block can be selected when selecting the reference block, so that the parallax calculation can be simplified and the circuit scale for the parallax calculation can be reduced. The increase can be suppressed. The distance measuring apparatus 101 can also calculate the parallax with high accuracy for the entire region of the reference image. In addition, since the distance measuring apparatus 101 only needs to enlarge the reference image by one pixel, an increase in circuit scale due to an increase in a storage area used in the parallax calculation can be suppressed.

  In addition, since the distance measuring apparatus 101 according to the first embodiment can simplify the parallax calculation as described above, an effect of improving the speed of the parallax calculation can be expected.

Hereinafter, the above-described effects will be described more specifically.
As shown in FIG. 14A and FIG. 14B, a general image pickup apparatus uses a first image pickup signal I1 that is a reference image and a second image pickup signal I2 that is a reference image from an optical axis position. When the number of pixels in the negative direction in the x direction (baseline direction) (direction approaching the optical axis center 114a of the lens portion 113a) is the same, the shift amount Δ that minimizes the parallax evaluation value R12 (kx) (SAD) and Using the parallax evaluation values R12 (Δ−1), R12 (Δ), and R12 (Δ + 1) with the front and rear shift amounts (shift amount Δ−1, shift amount Δ + 1), the parallax Δ12 (resolution is less than one pixel) i, j). And a general imaging device does not correct | amend parallax like Formula (32), but calculates distance based on this parallax (DELTA) 12 (i, j). Therefore, as shown in FIGS. 14 (a) and 14 (b), a general imaging apparatus has a leftmost block (in FIG. 11, the indexes are (HB / 2, 0), (HB / 2, 1)). ,... (Block indicated by (HB / 2, VB / 2)), and when the shift amount Δ that minimizes the parallax evaluation value R12 (kx) (SAD) is 0 when the parallax is near 0, the shift amount The parallax evaluation value R12 (-1) of Δ-1 cannot be calculated (the parallax evaluation value R12 (-1) of the shift amount -1 cannot be calculated). Therefore, a general imaging device cannot obtain a parallax with a resolution of less than one pixel using Equation (23) in the leftmost block of the reference image.

  On the other hand, in the distance measuring apparatus 101 according to the first embodiment, as shown in FIG. 14C, the second imaging signal I2 that is the reference image is compared with the first imaging signal I1 that is the reference image. The number of pixels is larger by one pixel in the negative direction (direction approaching the optical axis center 114a of the lens portion 113a) in the x direction (baseline direction) from the optical axis center. Then, the distance measuring apparatus 101 has a shift amount Δ that minimizes the parallax evaluation value R12 (kx) (SAD) and a shift amount before and after the shift amount (shift amount Δ−1 and shift amount Δ + 1) as shown in Expression (23). The parallax Δ12 (i, j) (temporary parallax) having a resolution of less than one pixel is obtained using the parallax evaluation values R12 (Δ−1), R12 (Δ), and R12 (Δ + 1). Then, the distance measuring apparatus 101 obtains the parallax Δ12a (i, j) by subtracting one pixel from the obtained parallax Δ12 (i, j) (temporary parallax) as shown in Expression (32). Then, the distance measuring apparatus 101 calculates a distance based on the parallax Δ12a (i, j) thus obtained. Therefore, as shown in FIGS. 14 (c) and 14 (d), the distance measuring apparatus 101 has a leftmost block (in FIG. 11, the indexes are (HB / 2, 0), (HB / 2, 1), ... (Block indicated by (HB / 2, VB / 2)), and when the shift amount Δ that minimizes the parallax evaluation value R12 (kx) (SAD) is 1 when the parallax is near 0, the shift amount Δ -1 parallax evaluation value R12 (Δ-1) can be calculated (parallax evaluation value R12 (0) with a shift amount of 0 can be calculated). Therefore, the distance measuring apparatus 101 can obtain the parallax value Δ (i, j), performs correction for subtracting the parallax by 1, and obtains the parallax value Δa (i, j). The parallax value Δa (i, j) obtained in this way is near zero. That is, the distance measuring apparatus 101 has a block at the left end of the reference image (in FIG. 11, the indexes are (HB / 2, 0), (HB / 2, 1),..., (HB / 2, VB / 2). Also in the block indicated by (), it is possible to obtain a high-resolution parallax of less than one pixel near parallax 0. Therefore, the distance measuring apparatus 101 realizes highly accurate distance measurement at a distance. That is, the distance measuring apparatus 101 realizes distance measurement with high accuracy in a distant place with a wide image area.

  In the distance measuring apparatus 101 according to the first embodiment, only a shift amount of 0 or more is necessary, and a negative shift amount is not necessary. Therefore, the distance measuring apparatus 101 does not need to consider negative numbers in the calculation of the coordinates of the reference image in Expression (22), and can omit the sign bit. In other words, the distance measuring apparatus 101 can reduce the circuit scale or processing time by the number of code bits that can be omitted, so that it is possible to realize a distance measuring apparatus that is low-cost and capable of measuring distances with high accuracy. To do.

  In addition, when a negative number is introduced into the shift amount, the distance measuring device needs to sequentially verify whether there is a reference image corresponding to the negative number when the shift amount is a negative number. As a result, the circuit scale or processing time increases by the amount of logic required for the verification. Since the distance measuring apparatus 101 according to the first embodiment limits the shift amount to 0 or more, the above-described verification is unnecessary, and the circuit scale or processing time can be reduced correspondingly, and the distance can be measured with high accuracy at low cost. A distance measuring device capable of providing a distance is realized.

  In the distance measuring apparatus 101 according to the first embodiment, as illustrated in FIG. 9, the second image pickup signal I2 that is the reference image is compared with the first image pickup signal I1 that is the reference image. The number of pixels is larger by one pixel in the negative direction (direction approaching the optical axis center 114a) in the x direction (baseline direction). In the distance measuring apparatus 101, the second imaging signal I2 has a number of pixels that is larger by SB pixels in the positive direction (direction away from the optical axis center 114a) in the x direction (optical axis direction) from the optical axis position. That is, the distance measuring apparatus 101 has a block at the left end of the reference image (in FIG. 11, the indexes are (HB / 2, 0), (HB / 2, 1),..., (HB / 2, VB / 2). ) And the rightmost block of the reference image (in FIG. 11, the indexes are (H1-HB / 2, 0), (H1-HB / 2, 1),..., (H1-HB / 2). , VB / 2)), the parallax evaluation value R12 (kx) (SAD) can be calculated. If the shift amount is −1 or SB + 2, the distance measuring apparatus refers to coordinates that are not in the reference image, and thus cannot calculate the parallax evaluation value R12 (kx). The distance measuring apparatus 101 according to the first embodiment has a minimum necessary number of pixels and only a necessary minimum storage capacity for the second imaging signal I2 that is a reference image, so that the cost is low. In addition, a distance measuring device capable of measuring distances with high accuracy is realized. Further, it is not necessary to sequentially verify whether there is a reference image corresponding to the shift amount. Therefore, it is possible to reduce the circuit scale or processing time by that amount, and to realize a distance measuring device that can measure a distance with high accuracy at low cost.

  Note that the present invention can also be realized as an imaging apparatus including components other than the distance calculator 145 included in the distance measuring apparatus 101 according to the first embodiment. Furthermore, the present invention can also be realized as a three-dimensional position measurement device or an image composition device including such an imaging device.

(Embodiment 2)
Next, an imaging system according to Embodiment 2 of the present invention will be described. The imaging system according to the second embodiment of the present invention uses the distance measuring apparatus 101 according to the first embodiment of the present invention. Hereinafter, an imaging system according to Embodiment 2 of the present invention will be described with reference to the drawings.

  FIG. 15 is a block diagram showing a configuration of the imaging system 202 according to Embodiment 2 of the present invention. An imaging system 202 according to Embodiment 2 of the present invention includes a distance measuring device 101, an imaging system control unit 203, and an image recognition unit 204.

  The imaging system control unit 203 includes a CPU and the like, and controls the entire imaging system 202.

  The distance measuring device 101 is the distance measuring device described in the first embodiment of the present invention, and is controlled by a command from the imaging system control unit 203, and image data I1 (here, the first imaging signal I1), The distance data DIS and failure information data DDE are output.

  The image recognition unit 204 includes a CPU and the like. Based on the failure information data DDE, the image recognition unit 204 performs various image recognition using the image data I1 and the distance data DIS of the part that is not failed. For example, when the image recognition unit 204 captures a face, it ignores the distance data of the failed part when determining the face orientation using the image data I1 and the distance data DIS.

  As described above, according to the imaging system according to the second embodiment of the present invention, as in the first embodiment, high-precision distance measurement in a distant place is realized in a wide image area.

  Further, according to the imaging system according to the second embodiment of the present invention, recognition accuracy is achieved by performing various image recognition using the image data I1 and the distance data DIS of the part that has not failed based on the failure information data DDE. Can be improved.

  Although the distance measuring apparatus according to the present invention has been described based on the embodiments, the present invention is not limited to these embodiments. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in the present embodiment, and forms constructed by combining components in different embodiments are also included in the scope of the present invention. .

  For example, the distance measuring apparatus 101 according to the first embodiment uses SAD as the parallax evaluation value as in Expression (22), Expression (25), and Expression (28), but the distance measuring apparatus according to the present invention is The sum of squares (SSD) obtained by transforming Equation (22) into Equation (35) below may be used. Note that the distance measuring apparatus according to the present invention may use the sum of squares in the same way in the calculations of Expressions (25) and (28).

  In addition, the distance measuring apparatus 101 according to the first embodiment uses a equiangular fitting interpolation formula as shown in Expression (23), Expression (26), and Expression (29), so that the resolution is less than one pixel. However, the distance measuring apparatus according to the present invention may obtain a parallax with a resolution of less than one pixel using a parabolic fitting interpolation formula obtained by transforming the formula (23) into the following formula (36). . It should be noted that the distance measuring apparatus according to the present invention may similarly use the parabolic fitting interpolation formula in the calculations of formulas (26) and (29).

  Further, in the distance measuring apparatus 101 according to the first embodiment, the sum in the block is calculated each time as shown in Expression (22), Expression (25), and Expression (28), but the distance measuring apparatus according to the present invention is Expression (22) may be transformed into the following expression (37) and recursively calculated. The distance measuring apparatus according to the present invention may calculate recursively in the same way in the calculations of Expressions (25) and (28).

  In the lens array 113 provided in the distance measuring apparatus 101 according to the first embodiment, Dx and Dy, which are distances between the lens units, are equal. However, the distance measuring device according to the present invention is not limited to the distance measuring device including such a lens array. That is, in the distance measuring device according to the present invention, Dx and Dy may be different. In this case, the distance measuring device needs to be changed due to the difference between Dx and Dy. For example, the distance measuring device has coordinates (xc4, yc4) of the position (optical axis position) corresponding to the optical axis center 114d of the fourth lens unit 113d at the center, and the number of pixels in the x direction is H1. From the region where the number of pixels in the y direction is V1 pixels, only one pixel in the negative direction in the x direction, only SB pixels in the positive direction, only Dy / Dx pixels in the negative direction in the y direction, positive direction The fourth imaging signal I4 is cut out from the imaging signal I0 so as to be an area larger by SB * Dy / Dx pixels. That is, the distance measuring device sets the xy coordinates so that the coordinates of the origin are the coordinates (x04, y04) with respect to the cut out fourth imaging signal I4, as in the following formulas (38) and (39). To do. As a result, in the distance measuring device, the magnitude of the fourth imaging signal I4 is the number of pixels in the x direction as H4 and the number of pixels in the y direction as shown in the following equations (40) and (41). . Note that the distance measuring apparatus may appropriately perform processing such as rounding up in order to make V4 an integer in the equation (41). In addition, the distance measuring device matches the shifting direction with a straight line connecting the optical axis center 114a of the first lens portion 113a and the optical axis center 114d of the fourth lens portion 113d as shown in the following formula (42). Here, the distance measuring device obtains an image (for example, a luminance value) as appropriate using linear interpolation with respect to a block where the coordinates are decimal points. In addition, when the parallax reliability C13 (i, j) is the smallest, the distance measuring device sets Dx / Dy to parallax Δ13 (i, j) as shown in the following equation (43) in order to match the parallax in the x direction. And the parallax Δ (i, j) is multiplied by Dy / Dx as shown in the following formula (44).

  In the distance measuring apparatus 101 according to the first embodiment, the lens array 113 is formed in a substantially circular shape. However, the distance measuring apparatus according to the present invention is not limited to the distance measuring apparatus including such a lens array. 16A and 16B are plan views showing the configuration of the lens array of the distance measuring apparatus according to the modification of the first embodiment. Like the lens array 301 shown in FIG. 16A, the lens array included in the distance measuring apparatus according to the present invention may be molded into a rectangular shape. Furthermore, as in the lens array 302 shown in FIG. 16B, the lens array provided in the distance measuring apparatus according to the present invention may be provided with a protrusion or the like extending in the planar direction on the end face thereof. As described above, even if the lens array is not substantially circular, the lens unit expands almost isotropically, and thus the distance measuring device according to the modification can obtain the same effect as the distance measuring device 101 of the first embodiment. Can do.

  In the distance measuring apparatus 101 according to the first embodiment, the lens array 113 includes four lens units. However, the distance measuring apparatus according to the present invention is limited to the distance measuring apparatus including such a lens array. Not. In the distance measuring apparatus according to the present invention, the number of lens portions included in the lens array may be different. 17A and 17B are plan views showing the configuration of the lens array of the distance measuring apparatus according to the modification of the first embodiment. Like the lens array 303 shown in FIG. 17A, the distance measuring device according to the present invention may have two lens portions. Further, like the lens array 304 shown in FIG. 17B, the distance measuring device according to the present invention may have nine lens portions.

  Furthermore, the present invention is realized as a parallax calculation method and a distance measurement method for executing processing performed by the characteristic components of such an imaging device or a distance measurement device, or as a program for causing a computer to execute these methods. You can also do it. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  Since the imaging device and the distance measuring device according to the present invention are a small imaging device and a distance measuring device that calculate a minute parallax with high accuracy and suppress an increase in circuit scale, a mobile phone having a camera function It is useful for digital still cameras, in-vehicle cameras, surveillance cameras, three-dimensional measuring instruments, stereoscopic image input cameras, and the like.

It is sectional drawing which shows the structure of the distance measuring device which concerns on Embodiment 1 of this invention. It is a top view which shows the structure of the lens array of the distance measuring device which concerns on Embodiment 1 of this invention. It is a top view which shows the structure of the circuit part of the ranging apparatus which concerns on Embodiment 1 of this invention. It is a top view which shows the structure of the image pick-up element of the distance measuring device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the parallax of an object image when the object at infinite distance is imaged in the distance measuring device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the parallax of an object image when the object located in the position of a finite distance is imaged in the distance measuring device which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the ranging apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the ranging apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the cutout position of the imaging signal of the distance measuring device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation | movement of the distance calculation of the distance measuring device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating a block in the distance measuring device which concerns on Embodiment 1 of this invention. FIG. 6 is a diagram for describing a calculation area of a parallax evaluation value in a parallax calculation when the first imaging signal and the second imaging signal are used in the distance measuring device according to the first embodiment of the present invention. It is a figure for demonstrating the relationship between the shift amount and parallax evaluation value in a parallax calculation when the 1st imaging signal and 2nd imaging signal of the ranging apparatus which concern on Embodiment 1 of this invention are utilized. (A)-(d) It is a figure for demonstrating the effect by the ranging device which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the imaging system which concerns on Embodiment 2 of this invention. FIG. 6 is a plan view showing a configuration of a lens array of a distance measuring apparatus according to a modification of the first embodiment. FIG. 6 is a plan view showing a configuration of a lens array of a distance measuring apparatus according to a modification of the first embodiment. FIG. 6 is a plan view showing a configuration of a lens array of a distance measuring apparatus according to a modification of the first embodiment. FIG. 6 is a plan view showing a configuration of a lens array of a distance measuring apparatus according to a modification of the first embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Distance measuring device 110 Lens module part 111 Lens tube 112 Cover glass 113 Lens array 120 Circuit part 121 Board | substrate 122 Package 123 Image pick-up element 124 Package cover glass 125 SLSI
127 Gold wire 131 System control unit 132 Image sensor drive unit 133 Imaging signal input unit 134 Input / output unit 141 Image cutout unit 142 Parallax calculation unit 143 Parallax correction unit 144 Failure detection unit 145 Distance calculation unit 202 Imaging system 203 Imaging system control unit 204 Image recognition unit

Claims (8)

A standard imaging system and one or more reference imaging systems, each having a lens and an imaging area corresponding to the lens, are generated between the standard imaging system and the reference imaging system when imaging the same subject. An imaging device for calculating parallax,
In the standard image and the reference image generated by the standard imaging system and the reference imaging system, the optical axis position that is a position corresponding to the optical axis center of the lens respectively included in the standard imaging system and the reference imaging system is acquired. Optical axis position acquisition means for
The light of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system among the base line directions that are directions of straight lines connecting the optical axis centers of the lenses of the standard imaging system and the reference imaging system. Coordinate setting means for setting the plane coordinates of the reference image and the reference image so that the position moved by a predetermined number of pixels from the axis position and the optical axis position of the reference image have the same coordinate value;
Based on the coordinates set by the coordinate setting means, an evaluation value representing the degree of similarity between the partial area of the reference image and the partial area of the reference image is calculated, and temporary disparity is calculated from the calculated evaluation value. Temporary parallax calculation means for calculating
An imaging apparatus comprising: a parallax calculating unit that calculates a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated by the temporary parallax calculating unit. The temporary parallax calculation means includes
A reference block selecting means for selecting a reference block which is a partial area of the reference image;
A predetermined shift amount from a partial area of the reference image indicated by the same coordinate value as the coordinate value indicating the reference block to a direction away from the optical axis center of the lens of the reference imaging system in the baseline direction An evaluation value calculating means for selecting an area shifted every time as a reference block, and calculating the evaluation value of the reference block for each selected reference block;
The imaging apparatus according to claim 1, further comprising: a temporary parallax calculation unit that calculates, as the temporary parallax, a shift amount that maximizes the degree of similarity using an interpolation formula based on a plurality of evaluation values among the calculated evaluation values. . The coordinate setting means includes a position moved by one pixel from the optical axis position of the reference image in a direction closer to the optical axis center of the lens of the reference imaging system in the baseline direction, and an optical axis position of the reference image. The imaging apparatus according to claim 2, wherein the planar coordinates are set so that the two have the same coordinate value. The imaging apparatus according to claim 1, wherein the reference image is an image that is larger by the predetermined number of pixels toward the direction closer to the optical axis center of the lens of the standard imaging system in the baseline direction than the reference image. . The imaging device further includes:
Parallax determination means for determining whether or not the parallax calculated by the parallax calculation means is a negative number;
The imaging apparatus according to claim 1, further comprising failure information output means for outputting failure information indicating that the parallax calculation has failed when the parallax determination means determines that the number is negative. A standard imaging system and one or more reference imaging systems, each having a lens and an imaging area corresponding to the lens, are generated between the standard imaging system and the reference imaging system when imaging the same subject. A distance measuring device that calculates a distance to the subject based on parallax,
In the standard image and the reference image generated by the standard imaging system and the reference imaging system, the optical axis position that is a position corresponding to the optical axis center of the lens respectively included in the standard imaging system and the reference imaging system is acquired. Optical axis position acquisition means for
The light of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system among the base line directions that are directions of straight lines connecting the optical axis centers of the lenses of the standard imaging system and the reference imaging system. Coordinate setting means for setting the plane coordinates of the reference image and the reference image so that the position moved by a predetermined number of pixels from the axis position and the optical axis position of the reference image have the same coordinate value;
Based on the coordinates set by the coordinate setting means, an evaluation value representing the degree of similarity between the partial area of the reference image and the partial area of the reference image is calculated, and temporary disparity is calculated from the calculated evaluation value. Temporary parallax calculation means for calculating
Parallax calculating means for calculating a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated by the temporary parallax calculating means;
A distance measuring device comprising: distance calculating means for calculating a distance to the subject using the parallax calculated by the parallax calculating means. When the same subject is imaged in an imaging device having a lens and an imaging region corresponding to the lens and including a standard imaging system and one or more reference imaging systems, the standard imaging system and the reference imaging system A parallax calculation method for calculating a parallax that occurs in between,
In the standard image and the reference image generated by the standard imaging system and the reference imaging system, the optical axis position that is a position corresponding to the optical axis center of the lens respectively included in the standard imaging system and the reference imaging system is acquired. An optical axis position acquisition step to perform,
The light of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system among the base line directions that are directions of straight lines connecting the optical axis centers of the lenses of the standard imaging system and the reference imaging system. A coordinate setting step of setting the plane coordinates of the reference image and the reference image so that the position moved by a predetermined number of pixels from the axis position and the optical axis position of the reference image have the same coordinate value;
Based on the coordinates set in the coordinate setting step, an evaluation value representing the degree of similarity between the partial area of the reference image and the partial area of the reference image is calculated, and temporary parallax is calculated from the calculated evaluation value. A temporary parallax calculation step for calculating
A parallax calculation method including a parallax calculation step of calculating a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated in the temporary parallax calculation step. When the same subject is imaged in an imaging device having a lens and an imaging region corresponding to the lens, the imaging device including a standard imaging system and one or more reference imaging systems, the standard imaging system and the reference imaging system A program for calculating parallax between
In the standard image and the reference image generated by the standard imaging system and the reference imaging system, the optical axis position that is a position corresponding to the optical axis center of the lens respectively included in the standard imaging system and the reference imaging system is acquired. An optical axis position acquisition step to perform,
The light of the reference image in a direction approaching the optical axis center of the lens of the reference imaging system among the base line directions that are directions of straight lines connecting the optical axis centers of the lenses of the standard imaging system and the reference imaging system. A coordinate setting step for setting the plane coordinates of the reference image and the reference image so that the position moved by a predetermined number of pixels from the axis position and the optical axis position of the reference image have the same coordinate value;
Based on the coordinates set in the coordinate setting step, an evaluation value representing the degree of similarity between the partial area of the reference image and the partial area of the reference image is calculated, and temporary parallax is calculated from the calculated evaluation value. A temporary parallax calculation step for calculating
A program for causing a computer to execute a parallax calculation step of calculating a parallax generated between the reference imaging system and the reference imaging system by subtracting the predetermined number of pixels from the temporary parallax calculated in the temporary parallax calculation step .

Images