JP2017123122A - Parallax value derivation apparatus, parallax value derivation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parallax value derivation apparatus which can determine an appropriate parallax value even when an image includes a repeated pattern area, a parallax value derivation method, and a program.SOLUTION: A parallax value derivation apparatus includes: acquisition means for acquiring a first image obtained by first imaging means that images a subject, and a second image obtained by second imaging means arranged in a position different from the first imaging means, to image the subject; detection means of detecting repeated pattern areas where the same pattern appears repeatedly, from the first and second images; modification means of modifying a pixel value of a pixel constituting the repeated pattern areas in the first and second images, on the basis of the amount of change to be determined in accordance with a pixel position of the pixel; calculation means of calculating a matching degree between a reference area in the first image modified by the modification means and each of a plurality of candidate areas which are candidates corresponding to the reference area in the second image modified by the modification means; and deriving means of deriving a parallax value between the reference area and the candidate areas, on the basis of the matching degree.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a parallax value derivation device, a parallax value derivation method, and a program.

  Conventionally, in terms of safety of automobiles, body structures of automobiles have been developed from the viewpoint of how to protect pedestrians and protect passengers when pedestrians and automobiles collide. However, in recent years, with the development of information processing technology and image processing technology, technology for detecting people and cars at high speed has been developed. Automobiles that apply these technologies to automatically apply a brake before an automobile collides with an object to prevent the collision have already been developed. In order to automatically apply the brake, it is necessary to accurately measure the distance to an object such as a person or another vehicle. For this purpose, distance measurement using millimeter wave radar and laser radar, distance measurement using a stereo camera, etc. Has been put to practical use.

  By using a stereo camera equipped with an imaging means that captures images from two different viewpoints and obtains two images, the three-dimensional position and size of an object such as a person or another vehicle can be accurately determined by obtaining a parallax image. Detect and measure the distance to the object. When obtaining the parallax for a certain pixel of the captured image, a matching process is performed to find out which position in the search range the pixel corresponding to the pixel of the reference image serving as a reference is in. Run. Then, after all the search range matching processing is completed, the position with the highest matching is set as the most likely appropriate parallax value.

  However, in the process of taking an image of a similar appearance repeatedly appearing on the exterior, such as a building window, tile wall, or fence, and calculating the parallax of the object, multiple matching locations may appear. . In such a case, since it is unclear which part is correct, the parallax value may be erroneously determined. When such a phenomenon occurs, for example, an erroneous distance that is as close as possible is measured even though it is actually a distant object (having a repeated pattern). Then, for example, in a system in which a vehicle is automatically stepped on when a nearby object is detected by ranging with a stereo camera, the brake is stepped on at a place where it is not necessary to step on the brake. It will cause “wrong treading”.

  Therefore, conventionally, a technique has been proposed in which a reference image is generated and a repeated pattern region in which a similar pattern appears repeatedly is detected by comparing the reference image with an input image (Patent Document 1). Further, the disparity values that should be regarded as the same object are grouped based on the disparity value obtained by the matching process, and the region on the reference image where the object exists and the adjacent region on the comparison image corresponding to the region A technique for recognizing an erroneously detected object by performing matching processing again is proposed (Patent Document 2).

  However, with the conventional technique, it is possible to detect a repeated pattern region or identify an erroneously detected object, but it is not possible to obtain an appropriate parallax value.

  The present invention has been made in view of the above, and a parallax value deriving device, a parallax value deriving method, and a parallax value deriving method capable of obtaining an appropriate parallax value even when a repeated pattern region exists in an image The purpose is to provide a program.

  In order to solve the above-described problems and achieve the object, the present invention is arranged such that the first image capturing unit captures the subject and the first image capturing unit is located at a different position from the first image capturing unit. An acquisition means for acquiring a second image obtained by imaging the subject by the second imaging means, and a repeated pattern area in which a similar pattern appears repeatedly from the first image and the second image, respectively. A detecting unit; a changing unit that changes a pixel value of a pixel constituting the repetitive pattern region of the first image and the second image based on a change amount determined according to a pixel position of the pixel; and the changing unit. Calculation for calculating a degree of coincidence between the changed reference area of the first image and each of a plurality of candidate areas that are candidates for areas corresponding to the reference area in the second image changed by the changing unit. And stage, based on the matching degree, and a deriving means for deriving the disparity values of the reference region and the candidate region.

  According to the present invention, an appropriate disparity value can be obtained even when a repeated pattern exists in an image.

FIG. 1 is a diagram for explaining the principle of deriving the distance from an imaging unit to an object. FIG. 2 is an explanatory diagram for obtaining a corresponding pixel in a comparison image corresponding to a reference pixel in the reference image. FIG. 3 is a diagram illustrating an example of a graph of the matching processing result. FIG. 4 is a diagram illustrating an example of the reference image and the comparison image. FIG. 5 is a diagram showing the relationship between the reference region and the search range in FIG. FIG. 6 is a diagram illustrating the relationship between the reference region and the candidate region in FIG. FIG. 7 is a diagram illustrating another example of the graph of the matching processing result. FIG. 8 is a diagram illustrating an example in which the device control system according to the embodiment is mounted on a vehicle. FIG. 9 is a diagram illustrating an example of the appearance of the object recognition apparatus according to the embodiment. FIG. 10 is a diagram illustrating an example of a hardware configuration of the object recognition apparatus according to the embodiment. FIG. 11 is a diagram illustrating an example of a functional block configuration of the object recognition apparatus according to the embodiment. FIG. 12 is a diagram for explaining an example of a luminance value changing method by the processing unit of the object recognition apparatus according to the embodiment. FIG. 13 is a diagram schematically illustrating an example of the relationship between the pixel position and the amount of change in the luminance value. FIG. 14 is a diagram schematically illustrating another example of the relationship between the pixel position and the change amount of the luminance value. FIG. 15 is a diagram illustrating an example of the processed reference image and the processed comparison image. FIG. 16 is a diagram illustrating the relationship between the reference region and the candidate region in FIG. FIG. 17 is a diagram illustrating another example of the graph of the matching processing result. FIG. 18 is a diagram illustrating an example of a functional block configuration of a disparity value calculation processing unit of the object recognition device according to the embodiment. FIG. 19 is a flowchart illustrating an example of the operation of the block matching process of the disparity value deriving unit according to the embodiment.

(Outline of distance measurement method using block matching process)
First, an outline of a distance measuring method using block matching processing will be described with reference to FIGS.

(Principles of ranging)
FIG. 1 is a diagram for explaining the principle of deriving the distance from an imaging unit to an object. With reference to FIG. 1, the principle of deriving a parallax with respect to an object from the stereo camera by stereo matching processing and measuring the distance from the stereo camera to the object with the parallax value indicating the parallax will be described.

  The imaging system shown in FIG. 1 includes an imaging unit 10a (first imaging unit) and an imaging unit 10b (second imaging unit) arranged in parallel equiposition. The imaging units 10a and 10b respectively include imaging lenses 11a and 11b that refract incident light and form an image of an object on an image sensor that is a solid-state imaging device. The images captured by the image capturing unit 10a and the image capturing unit 10b are referred to as a reference image Ia and a comparative image Ib, respectively. In FIG. 1, the point S on the object E in the three-dimensional space is mapped to a position on a straight line parallel to a straight line connecting the imaging lens 11a and the imaging lens 11b in each of the reference image Ia and the comparison image Ib. Here, the point S mapped to each image is a point Sa (x, y) in the reference image Ia and a point Sb (X, y) in the comparison image Ib. At this time, the parallax value dp is expressed by the following (Equation 1) using the point Sa (x, y) at the coordinates on the reference image Ia and the point Sb (X, y) at the coordinates on the comparison image Ib. It is expressed in

  dp = X−x (Formula 1)

  In FIG. 1, the distance between the point Sa (x, y) in the reference image Ia and the intersection of the perpendicular line taken from the imaging lens 11a on the imaging surface is Δa, and the point Sb (X, y) in the comparative image Ib is When the distance from the intersection of the perpendicular line taken from the imaging lens 11b to the imaging surface is Δb, the parallax value dp can also be expressed as dp = Δa + Δb.

  Next, the distance Z between the imaging units 10a and 10b and the object E is derived by using the parallax value dp. Here, the distance Z is a distance from a straight line connecting the focal position of the imaging lens 11a and the focal position of the imaging lens 11b to the point S on the object E. As shown in FIG. 1, using the focal length f of the imaging lens 11a and the imaging lens 11b, the baseline length B that is the length between the imaging lens 11a and the imaging lens 11b, and the parallax value dp, According to 2), the distance Z can be calculated.

  Z = (B × f) / dp (Formula 2)

  From this (Equation 2), it can be seen that the larger the parallax value dp, the smaller the distance Z, and the smaller the parallax value dp, the larger the distance Z.

(Block matching process)
Next, a distance measuring method using block matching processing (hereinafter also simply referred to as matching processing) will be described.

  FIG. 2 is an explanatory diagram for obtaining a corresponding pixel in a comparison image corresponding to a reference pixel in the reference image. FIG. 3 is a diagram illustrating an example of a graph of the matching processing result. FIG. 4 is a diagram illustrating an example of the reference image and the comparison image.

  A method for calculating the cost value C (p, d) will be described with reference to FIGS. In the following description, C (p, d) represents C (x, y, d).

  2A is a conceptual diagram illustrating the reference pixel p and the reference region pb in the reference image Ia, and FIG. 2B corresponds to the reference pixel p illustrated in FIG. It is a conceptual diagram at the time of calculating the cost value C while sequentially shifting (shifting) the corresponding pixel candidates in the comparative image Ib. Here, the corresponding pixel indicates a pixel in the comparison image Ib that is most similar to the reference pixel p in the reference image Ia. The cost value C is an evaluation value (degree of coincidence) that represents the similarity or dissimilarity of each pixel in the comparison image Ib with respect to the reference pixel p in the reference image Ia. The cost value C shown below is described as an evaluation value that represents a degree of dissimilarity indicating that the smaller the value is, the more similar the pixel in the comparative image Ib is to the reference pixel p.

  As shown in FIG. 2A, a candidate that is a candidate for a reference pixel p (x, y) in the reference image Ia and a corresponding pixel on the epipolar line EL in the comparison image Ib for the reference pixel p (x, y). Based on each luminance value (pixel value) of the pixel q (x + d, y), the cost value C (p, d) of the candidate pixel q (x + d, y) corresponding to the reference pixel p (x, y) is calculated. Is done. d is a shift amount (shift amount) between the reference pixel p and the candidate pixel q. The shift amount d is shifted in units of pixels. That is, in FIG. 3, the candidate pixel q (x + d, y) and the reference pixel p are sequentially shifted by one pixel within a predetermined range (for example, 0 <d <25). A cost value C (p, d), which is the dissimilarity of the luminance value with (x, y), is calculated. In addition, in the present embodiment, block matching processing is performed as matching processing for obtaining a corresponding pixel of the reference pixel p. In the block matching process, a reference region pb that is a predetermined region centered on the reference pixel p of the reference image Ia, and a candidate region qb (size is the same as the reference region pb) centered on the candidate pixel q of the comparison image Ib. Find dissimilarity of. As the cost value C indicating the dissimilarity between the reference region pb and the candidate region qb, the average value of each block is subtracted from the value of SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), or SSD. ZSSD (Zero-mean-Sum of Squared Difference) or the like is used. These evaluation values indicate dissimilarity because the values are smaller as the correlation is higher (the degree of similarity is higher).

  As described above, since the imaging units 10a and 10b are arranged in parallel equiposition, the reference image Ia and the comparison image Ib are also in parallel equivalence relations. Accordingly, the corresponding pixel in the comparison image Ib corresponding to the reference pixel p in the reference image Ia exists on the epipolar line EL shown as a horizontal line in FIG. 2, and the corresponding pixel in the comparison image Ib is In order to obtain it, the pixel on the epipolar line EL of the comparison image Ib may be searched.

  The cost value C (p, d) calculated by such block matching processing is represented by, for example, the graph shown in FIG. 3 in relation to the shift amount d. In the example of FIG. 3, the cost value C is derived as the parallax value dp = 7 because the minimum value is obtained when the shift amount d = 7.

  However, in an image including an object such as a building window, a tile wall, or a fence that repeatedly shows the same pattern, two or more matching points may appear in the process of calculating the parallax of the object. is there. In such a case, the most probable parallax value may be erroneously determined.

  For example, when the reference image Ia is the image shown in FIG. 4A and the comparison image Ib is the image shown in FIG. 4B, the region showing the building window and the like is a repetitive pattern region in which the same pattern appears repeatedly. Become. Here, a region surrounded by a broken line in FIG. 4A is defined as a reference region pb, and a region surrounded by a broken line in FIG. 4B is defined as a search range Q of the candidate region qb. In this case, the similarity (dissimilarity) is derived by comparing the reference region pb and each candidate region qb in the search range Q as shown in FIG.

  FIG. 5 is a diagram showing the relationship between the reference region pb in FIGS. 4A and 4B and the search range Q (candidate region qb). As shown in the figure, the similarity (dissimilarity) is obtained by comparing the reference region pb with the candidate region qb existing at each position (search position) within the search range Q. In this case, as shown in FIG. 6, the image of the reference region pb is similar to the image of the candidate region qb at the search positions 4, 7,. Therefore, as shown in FIG. 7, the graph of the cost value C obtained by this matching processing periodically fluctuates (up and down) depending on the repeated pattern region included in the subject.

  When such a variation occurs, the minimum value of the cost value C cannot be specified. For example, although the object is actually a distant object (having a repetitive pattern), an erroneous distance as if it is close ( The parallax value dp) is measured. Then, for example, in a system in which a vehicle is automatically stepped on when a nearby object is detected by ranging with a stereo camera, the brake is stepped on at a place where it is not necessary to step on the brake. It will cause “wrong treading”.

  As described above, when a repeated pattern region exists in an image, it is difficult to obtain an appropriate parallax value. Therefore, in the following embodiment, a parallax value derivation device, a parallax value derivation method, and a program that can obtain an appropriate parallax value even when a repeated pattern region exists in an image will be described.

  Hereinafter, embodiments of a parallax value deriving device, a parallax value deriving method, and a program according to the present invention will be described in detail with reference to FIGS. 8 to 19. Hereinafter, a case where an object recognition device including a parallax value deriving device is mounted on an automobile will be described as an example.

(Schematic configuration of a vehicle equipped with an object recognition device)
FIG. 8 is a diagram illustrating an example in which the device control system is mounted on a vehicle. 8A is a side view of the vehicle 70 on which the device control system 60 is mounted, and FIG. 8B is a front view of the vehicle 70.

  As shown in FIG. 8, a vehicle 70 that is an automobile includes a device control system 60. The device control system 60 includes an object recognition device 1, a vehicle control device 6, a steering wheel 7, and a brake pedal 8 that are installed in a vehicle room that is a room space of the vehicle 70.

  The object recognition device 1 has an imaging function for imaging the traveling direction of the vehicle 70 and is installed, for example, in the vicinity of a rearview mirror inside the front window of the vehicle 70. Although the details will be described later, the object recognition device 1 includes a main body unit 2, an imaging unit 10a fixed to the main body unit 2, and an imaging unit 10b. The imaging units 10a and 10b are fixed to the main body unit 2 so that a subject in the traveling direction of the vehicle 70 can be imaged.

  The vehicle control device 6 is an ECU (Electronic Control Unit) that performs various vehicle controls based on the recognition information received from the object recognition device 1. As an example of vehicle control, the vehicle control device 6 executes steering control that controls a steering system (control target) including the steering wheel 7 to avoid an obstacle. Moreover, the vehicle control apparatus 6 performs the brake control which controls the brake pedal 8 (control object) and decelerates and stops the vehicle 70 as an example of vehicle control.

  Like the device control system 60 including the object recognition device 1 and the vehicle control device 6 described above, vehicle control such as steering control or brake control is executed, so that the safety of driving the vehicle 70 can be improved. it can.

  Note that, as described above, the object recognition device 1 captures the front of the vehicle 70, but is not limited thereto. That is, the object recognition apparatus 1 may be installed so as to capture the rear or side of the vehicle 70. In this case, the object recognition apparatus 1 can detect the position of a succeeding vehicle behind the vehicle 70 or another vehicle that translates laterally. And the vehicle control apparatus 6 can detect the danger at the time of the lane change of the vehicle 70, or a lane merge, etc., and can perform the above-mentioned vehicle control. Further, the vehicle control device 6 determines that there is a risk of collision based on the recognition information about the obstacle behind the vehicle 70 output by the object recognition device 1 in the back operation when the vehicle 70 is parked. In this case, the vehicle control described above can be executed.

(Configuration of object recognition device)
FIG. 9 is a diagram illustrating an example of an appearance of the object recognition apparatus according to the first embodiment. As illustrated in FIG. 9, the object recognition device 1 includes the main body unit 2, the imaging unit 10a fixed to the main body unit 2, and the imaging unit 10b as described above. The imaging units 10 a and 10 b are constituted by a pair of cylindrical cameras arranged in parallel equiposition with respect to the main body unit 2. For convenience of explanation, the imaging unit 10a illustrated in FIG. 9 is referred to as a “right” camera, and the imaging unit 10b is referred to as a “left” camera.

<Hardware configuration of object recognition device>
FIG. 10 is a diagram illustrating an example of a hardware configuration of the object recognition apparatus according to the first embodiment. A hardware configuration of the object recognition apparatus 1 will be described with reference to FIG.

  As illustrated in FIG. 10, the object recognition device 1 (an example of a parallax value derivation device) includes a parallax value derivation unit 3 (an example of a parallax value derivation device) and a recognition processing unit 5 in a main body unit 2.

  Among these, the parallax value deriving unit 3 derives a parallax value dp indicating the parallax with respect to the object E from a plurality of images obtained by imaging the object E, and outputs a parallax image indicating the parallax value dp in each pixel. . The recognition processing unit 5 performs processing such as measuring the distance from the imaging units 10 a and 10 b to the object E based on the parallax image output from the parallax value deriving unit 3.

  As illustrated in FIG. 10, the parallax value deriving unit 3 includes an imaging unit 10a, an imaging unit 10b, a signal conversion unit 20a, a signal conversion unit 20b, and an image processing unit 30.

  The imaging unit 10a is a processing unit that images an object in front and generates an analog image signal. The imaging unit 10a includes an imaging lens 11a, a diaphragm 12a, and an image sensor 13a.

  The imaging lens 11a is an optical element that refracts incident light to form an image of an object on the image sensor 13a. The diaphragm 12a is a member that adjusts the amount of light input to the image sensor 13a by blocking part of the light that has passed through the imaging lens 11a. The image sensor 13a is a semiconductor element that converts light that has entered the imaging lens 11a and passed through the aperture 12a into an electrical analog image signal. The image sensor 13a is realized by a solid-state imaging device such as a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor), for example.

  The imaging unit 10b is a processing unit that images an object in front and generates an analog image signal. The imaging unit 10b includes an imaging lens 11b, a diaphragm 12b, and an image sensor 13b. The functions of the imaging lens 11b, the diaphragm 12b, and the image sensor 13b are the same as the functions of the imaging lens 11a, the diaphragm 12a, and the image sensor 13a, respectively. Further, the imaging lens 11a and the imaging lens 11b are installed so that the respective lens surfaces are on the same plane so that the left and right cameras can be imaged under the same conditions.

  The signal conversion unit 20a is a processing unit that converts the analog image signal generated by the imaging unit 10a into digital image data. The signal conversion unit 20a includes a correlated double sampling (CDS) 21a, an auto gain control (AGC) 22a, an analog digital converter (ADC) 23a, and a frame memory 24a.

  The CDS 21a removes noise from the analog image signal generated by the image sensor 13a by correlated double sampling, a horizontal differential filter, or a vertical smoothing filter. The AGC 22a performs gain control for controlling the intensity of the analog image signal from which noise has been removed by the CDS 21a. The ADC 23a converts an analog image signal whose gain is controlled by the AGC 22a into digital image data. The frame memory 24a stores the image data converted by the ADC 23a.

  The signal conversion unit 20b is a processing unit that converts an analog image signal generated by the imaging unit 10b into digital image data. The signal conversion unit 20b includes a CDS 21b, an AGC 22b, an ADC 23b, and a frame memory 24b. The functions of the CDS 21b, AGC 22b, ADC 23b, and frame memory 24b are the same as the functions of the above-described CDS 21a, AGC 22a, ADC 23a, and frame memory 24a, respectively.

  The image processing unit 30 is a device that performs image processing on the image data converted by the signal conversion unit 20a and the signal conversion unit 20b. The image processing unit 30 includes an FPGA (Field Programmable Gate Array) 31, a CPU (Central Processing Unit) 32, a ROM (Read Only Memory) 33, a RAM (Random Access Memory) 34, and an I / F (35). And a bus line 39.

  The FPGA 31 is an integrated circuit, and here performs a process of deriving a parallax value dp in an image based on image data. The CPU 32 controls each function of the parallax value deriving unit 3. The ROM 33 stores an image processing program that the CPU 32 executes in order to control each function of the parallax value deriving unit 3. The RAM 34 is used as a work area for the CPU 32. The I / F 35 is an interface for communicating with the I / F 55 in the recognition processing unit 5 via the communication line 4. As shown in FIG. 10, the bus line 39 is an address bus, a data bus, or the like that connects the FPGA 31, the CPU 32, the ROM 33, the RAM 34, and the I / F 35 so that they can communicate with each other.

  The image processing unit 30 includes the FPGA 31 as an integrated circuit for deriving the parallax value dp. However, the image processing unit 30 is not limited to this and may be an integrated circuit such as an ASIC (Application Specific Integrated Circuit). .

  As shown in FIG. 10, the recognition processing unit 5 includes an FPGA 51, a CPU 52, a ROM 53, a RAM 54, an I / F 55, a CAN (Controller Area Network) I / F 58, and a bus line 59.

  The FPGA 51 is an integrated circuit, and here performs recognition processing on an object based on the parallax image received from the image processing unit 30. The CPU 52 controls each function of the recognition processing unit 5. The ROM 53 stores a recognition processing program for the CPU 52 to execute the recognition processing of the recognition processing unit 5. The RAM 54 is used as a work area for the CPU 52. The I / F 55 is an interface for communicating with the I / F 35 of the image processing unit 30 via the communication line 4. The CAN I / F 58 is an interface for communicating with an external controller (for example, the vehicle control device 6 shown in FIG. 10), and is connected to, for example, a CAN of an automobile. As shown in FIG. 10, the bus line 59 is an address bus, a data bus, or the like that connects the FPGA 51, the CPU 52, the ROM 53, the RAM 54, the I / F 55, and the CAN I / F 58 so that they can communicate with each other.

  With this configuration, when a parallax image is transmitted from the I / F 35 of the image processing unit 30 to the recognition processing unit 5 via the communication line 4, the FPGA 51 is converted into a parallax image by a command of the CPU 52 in the recognition processing unit 5. Based on this, the distance Z between the imaging units 10a, 10b and the object E is calculated, or recognition processing for the object is executed.

  Note that each of the above-described programs may be recorded and distributed on a computer-readable recording medium in an installable or executable format file. The recording medium is a CD-ROM (Compact Disc Read Only Memory) or an SD (Secure Digital) memory card.

<Functional block configuration of object recognition apparatus and operation of each functional block>
FIG. 11 is a diagram illustrating an example of a functional block configuration of the object recognition apparatus according to the first embodiment. As illustrated in FIG. 11, the object recognition device 1 includes a parallax value deriving unit 3 and a recognition processing unit 5. Among these, the parallax value deriving unit 3 includes an image acquisition unit 100 (acquisition unit), a conversion unit 200, pattern detection units 300a and 300b (detection unit), processing units 400a and 400b (change unit), and parallax. A value calculation processing unit 500.

  The image acquisition unit 100 is a functional unit that captures an image of a subject in front using two left and right cameras, generates analog image signals, and obtains two luminance images that are images based on the image signals. The image acquisition unit 100 is realized by the imaging unit 10a and the imaging unit 10b illustrated in FIG.

  The conversion unit 200 performs noise removal, distortion correction (parallelization of the two luminance images), and the like on the image data of the two luminance images obtained by the image acquisition unit 100, and then the digital format image. This is a functional unit that converts the data into an output. Digital image data is composed of, for example, 8-bit grayscale pixels.

  Here, of the image data of the two luminance images output from the conversion unit 200 (hereinafter simply referred to as luminance image), the reference image Ia captured by the right camera (imaging unit 10a) of the image acquisition unit 100 is displayed. Image data (hereinafter simply referred to as a reference image Ia) and image data of a comparative image Ib (hereinafter simply referred to as a comparative image Ib) captured by the left camera (imaging unit 10b). That is, the conversion unit 200 outputs the reference image Ia and the comparison image Ib based on the two luminance images output from the image acquisition unit 100. The conversion unit 200 is realized by the signal conversion units 20a and 20b illustrated in FIG.

  The pattern detection unit 300a is a functional unit that detects a repetitive pattern region in which a pattern or the like is represented as a repetitive pattern from the reference image Ia received from the conversion unit 200. Specifically, the pattern detection unit 300a outputs information as to whether or not each pixel of the reference image Ia is repeatedly included in the pattern area as a detection flag. The pattern detection unit 300a is realized by the FPGA 31 shown in FIG.

  The pattern detection unit 300b is a functional unit that repeatedly detects a pattern area from the comparison image Ib received from the conversion unit 200. Specifically, the pattern detection unit 300b outputs information as to whether or not each pixel of the reference image Ia is repeatedly included in the pattern area as a detection flag. The pattern detection unit 300b is realized by the FPGA 31 illustrated in FIG.

  In addition, the detection method of the repeated pattern area | region in the pattern detection parts 300a and 300b does not matter in particular, It is possible to use a well-known technique. For example, the pattern detection units 300a and 300b may detect a repeated pattern region by generating a reference image and comparing it with an input image. In addition, for example, the pattern detection units 300a and 300b sequentially acquire each pixel value of the reference image Ia (comparison image Ib) as a target pixel value, and based on the amount of change in luminance value between the pixels to be the target pixel. The repeated pattern area may be detected. Alternatively, the pattern detection unit 300a and the pattern detection unit 300b may be integrated, and the reference image Ia and the comparison image Ib may be compared to detect a repeated pattern region from both images.

  The processing unit 400a changes the pixel value (luminance value) of the repetitive pattern region included in the reference image Ia based on the reference image Ia received from the conversion unit 200 and the detection flag received from the pattern detection unit 300a. Execute the process for Specifically, the processing unit 400a changes the luminance value of the pixels constituting the repetitive pattern area of the reference image Ia based on the change amount determined according to the position of the pixel (hereinafter referred to as pixel position). Then, the processing unit 400a outputs the reference image Ia in which the luminance value of the repeated pattern region is changed to the parallax value calculation processing unit 500 as the processed reference image Ia. If the reference image Ia does not include a repeated pattern region, the reference image Ia is output to the parallax value calculation processing unit 500 as the processed reference image Ia. The processing unit 400a is realized by the FPGA 31 illustrated in FIG.

  Here, the brightness value changing method can take various forms. FIG. 12 is a diagram for explaining an example of a method of changing the luminance value by the processing unit 400a. First, the processing unit 400a identifies the repeated pattern region Ra from the reference image Ia based on the flag information received from the pattern detection unit 300a (FIGS. 12A and 12B). Subsequently, the processing unit 400a identifies the pixel position in the epipolar line direction (the horizontal direction in the drawing) for each pixel constituting the repetitive pattern region Ra of the reference image Ia, and sets the change amount determined according to the pixel position. Add to the luminance value of the pixel. Thereby, the luminance value of the repeated pattern region Ra continuously changes in the epipolar line direction as shown in FIG. Note that the base point for specifying the pixel position may be the left end or the right end of the repeated pattern region Ra.

  FIG. 13 is a diagram schematically illustrating an example of the relationship between the pixel position and the amount of change in the luminance value. FIG. 14 is a diagram schematically illustrating another example of the relationship between the pixel position and the change amount of the luminance value. Here, the horizontal axis means the pixel position from the base point, and the vertical axis means the change amount of the luminance value.

  Based on the relationship between the pixel position shown in FIG. 13 or FIG. 14 and the amount of change in luminance value, the processing unit 400a changes the luminance value of each pixel in the repetitive pattern region Ra corresponding to the pixel position of that pixel. Add the amount. For example, the processing unit 400a changes the luminance value P1 of the pixel located at the pixel position x1 to the new luminance value Q1 (= P1 + α1) by adding the corresponding change amount α1. Further, the processing unit 400a adds the corresponding change amounts α2, α3,..., Αn to the luminance values P2, P3,..., Pn of the pixels at the pixel positions x2, x3,. The brightness value Q2 (= P2 + α2), Q3 (= P3 + α3),..., Qn (= Pn + αn) is changed to the pixel.

  Here, when the relationship of FIG. 13 is used, the processing unit 400a increases the change amount of the luminance value according to the separation distance from the base point. When the relationship of FIG. 13 is used, the processing unit 400a decreases the amount of change in the luminance value according to the separation distance from the base point.

  Note that the relationship between the pixel position and the amount of change in the luminance value described with reference to FIGS. 13 and 14 may be held in the ROM 33 or the like in the state of a table or a relational expression. Further, the relationship between the pixel position and the amount of change in the luminance value described with reference to FIGS. 13 and 14 may be held in the ROM 33 or the like in the state of image data such as a tone image. In the former case, the processing unit 400a derives a change amount for each pixel position based on a table or a relational expression, and changes the pixel value of each pixel in the repeated pattern region. In the latter case, the processing unit 400a changes the pixel value of each pixel in the repeated pattern region by superimposing the tone image on the repeated pattern region.

  As another change method, a method is adopted in which the luminance value of the immediately preceding pixel is sequentially added to the luminance value of the next pixel from the pixel at the end portion serving as the base point toward the pixel at the opposite end portion. Also good. In this case, for example, the image processing unit 402 has pixel positions x1 (luminance value P1), x2 (luminance value P2), x3 (luminance value P3),..., Xn (luminance value) arranged continuously in the epipolar line direction. For each pixel of Pn), the luminance value is changed as follows.

  First, the image processing unit 402 adds the luminance value P1 of the pixel at the pixel position x1 to the luminance value P2 of the pixel at the pixel position x2, thereby changing the pixel at the pixel position x2 to the luminance value Q2 (= P2 + P1). . Next, the image processing unit 402 adds the luminance value Q2 of the pixel at the pixel position x2 to the luminance value P3 of the pixel at the pixel position x3, thereby changing the pixel at the pixel position x3 to the luminance value Q3 (= P3 + Q2). . Then, the image processing unit 402 adds the luminance value Qn−1 of the pixel at the pixel position xn−1 to the luminance value Pn of the pixel at the pixel position xn, thereby changing the luminance value Qn (= Pn + Qn) of the pixel at the pixel position xn. Change to -1).

  Further, the calculation method of the change amount with respect to the luminance value is not limited to addition, and may be subtraction, multiplication, or division. Note that the luminance value after the change may be saturated due to the change in the luminance value. Here, for example, when the luminance value is expressed by 256 gradations, the saturation means a state in which the luminance value reaches an extreme value such as 0 or 255 or near the extreme value. When the luminance value in the saturated state is set continuously for a plurality of pixels, overexposure or blackout occurs at that portion, and the characteristics of the image such as the pattern originally possessed by the repeated pattern area Ra are lost. Therefore, the image processing unit 402 performs control to prevent continuous setting of the saturated luminance value when the luminance value is changed.

  For example, when the luminance value is saturated due to the change of the luminance value, the image processing unit 402 switches the operator related to the change of the luminance value to the operator having the opposite meaning. Specifically, when the luminance value reaches a saturation state (upper limit value) by the calculation method for adding the change amount, the image processing unit 402 switches to the calculation method for subtracting the change amount. In addition, when the luminance value reaches a saturation state (lower limit value) by the calculation method for subtracting the change amount, the image processing unit 402 switches to the calculation method for adding the change amount. As a result, it is possible to prevent the luminance value after the change from being saturated, so that the original image characteristics of the repeated pattern region Ra can be maintained.

  As another control method, the image processing unit 402 saturates the changed luminance value based on the horizontal width in the search direction of the repeated pattern region, the distribution of the luminance value of each pixel in the repeated pattern region, and the like. The change method (change amount or calculation method) may be determined so as not to be performed. As a result, it is possible to prevent the luminance value after the change from being saturated, so that the original image characteristics of the repeated pattern region Ra can be maintained.

  Returning to FIG. 11, the processing unit 400b, based on the comparison image Ib received from the conversion unit 200 and the detection flag received from the pattern detection unit 300b, the pixel value (luminance of the repetitive pattern region included in the comparison image Ib. Value) is executed. Specifically, the processing unit 400b changes the luminance value of the pixels constituting the repetitive pattern region of the comparison image Ib to a change amount determined according to the pixel position of the pixel by the same changing method as the processing unit 400a. Change based on. Then, the processing unit 400b outputs the comparison image Ib in which the luminance value of the repeated pattern region is changed to the parallax value calculation processing unit 500 as the processed comparison image Ib. If the reference image Ia does not include a repeated pattern region, the reference image Ia is output to the parallax value calculation processing unit 500 as the processed reference image Ia. The processing unit 400b is realized by the FPGA 31 illustrated in FIG.

  Through the processing of the processing units 400a and 400b described above, the luminance values of the pixels constituting the repetitive pattern region of the reference image Ia and the comparison image Ib are changed based on the amount of change determined according to the pixel position of the pixel. Therefore, for example, when the processing units 400a and 400b process the reference image Ia and the comparative image Ib shown in FIGS. 4A and 4B, as shown in FIGS. 15A and 15B, The luminance value of each pixel in the repeated pattern areas Ra and Rb of the building window portion is changed according to the pixel position in the epipolar line direction (the horizontal direction in the drawing).

  Further, when attention is paid to the candidate regions qb of the search positions 4, 7,..., N that are similar to the reference region pb in FIG. 6, as shown in FIG. 16, at each of the search positions 4, 7,. The brightness values of the candidate areas qb are different. Therefore, in the matching process, the similarity of the candidate region qb at the search position 4 having the same luminance value as that of the reference region pb is increased. Further, since the candidate areas qb of the other search positions 7,..., N are different from the luminance value of the reference area pb, the dissimilarity is increased. As a result, in the graph of the cost value C obtained by the matching process, as shown in FIG. 17, the shift amount (search position) d = 4 becomes the minimum value, so that the minimum value of the cost value C can be specified. . Thereby, the most probable appropriate parallax value can be obtained from the minimum value of the cost value C.

  Note that the processing units 400a and 400b perform the luminance value changing process described above for each repeated pattern region detected in the reference image Ia and the comparison image Ib.

  Returning to FIG. 11, the parallax value calculation processing unit 500 calculates the parallax value for each pixel of the processed reference image Ia based on the processed reference image Ia and the processed comparison image Ib received from the processing units 400a and 400b. A functional unit that derives and generates a parallax image in which a parallax value is associated with each pixel. The parallax value calculation processing unit 500 outputs the generated parallax image to the recognition processing unit 5.

  As illustrated in FIG. 18, the parallax value calculation processing unit 500 includes a cost calculation unit 501 (calculation unit) and a derivation unit 502 (derivation unit).

  The cost calculation unit 501 performs the matching process described with reference to FIG. 2 on the reference region pb of the processed reference image Ia (first image) and the region corresponding to the reference region pb in the processed comparison image Ib (second image). The degree of coincidence with each of a plurality of candidate regions qb as candidates is calculated as a cost value C. Specifically, the cost calculation unit 501 includes a reference region pb that is a predetermined region centered on the reference pixel p of the processed reference image Ia, and a candidate region qb centered on the candidate pixel q of the processed comparison image Ib. Is calculated as a cost value C. Note that the size of the candidate region qb is the same as that of the reference region pb.

  The derivation unit 502 uses the shift amount d corresponding to the minimum value of the cost value C calculated by the cost calculation unit 501 as the parallax value dp for the pixel of the processed reference image Ia for which the cost value C is calculated. To derive.

  Further, the deriving unit 502 generates and recognizes a parallax image that is an image in which the luminance value of each pixel of the processed reference image Ia is represented by the parallax value dp corresponding to the pixel based on the derived parallax value dp. Output to the processing unit 5. In the present embodiment, the derivation unit 502 is configured to output a parallax image. However, the configuration is not limited thereto, and only the parallax value dp or a configuration in which the parallax value dp is associated with a position on the image is output. Also good.

  The cost calculation unit 501 and the derivation unit 502 shown in FIG. 18 are realized by the FPGA 31 shown in FIG. Note that part or all of the cost calculation unit 501 and the derivation unit 502 may be realized by the CPU 32 executing a program stored in the ROM 33 instead of the FPGA 31 that is a hardware circuit.

  Moreover, the cost calculation part 501 and the derivation | leading-out part 502 of the parallax value calculation process part 500 shown in FIG. 18 show the function notionally, and are not limited to such a structure. For example, a plurality of functional units illustrated as independent functional units in FIG. 18 may be configured as one functional unit. On the other hand, the function of one functional unit in FIG. 18 may be divided into a plurality of functions and configured as a plurality of functional units.

  Returning to FIG. 11, the recognition processing unit 5 performs various recognition processes such as measuring the distance from the imaging units 10 a and 10 b to the object E based on the parallax images output from the parallax value deriving unit 3, and performs the recognition process. The recognition information, which is the information indicating the result, is output to the vehicle control device 6.

  The image acquisition unit 100, the conversion unit 200, the pattern detection units 300a and 300b, the processing processing units 400a and 400b, and the parallax value calculation processing unit 500 of the parallax value deriving unit 3 illustrated in FIG. However, the present invention is not limited to such a configuration. For example, a plurality of functional units illustrated as independent functional units in the parallax value deriving unit 3 illustrated in FIG. 11 may be configured as one functional unit. On the other hand, in the parallax value deriving unit 3 shown in FIG. In the case where the parallax value dp or the parallax value dp associated with the position on the image is output from the parallax value deriving unit 3, the recognition processing unit 5 uses the information to perform various recognitions described above. Processing shall be performed.

(Block matching process of parallax value deriving unit)
Hereinafter, the operation of the parallax value deriving unit 3 will be described. FIG. 19 is a flowchart illustrating an example of the operation of the block matching process of the disparity value deriving unit 3. An operation flow of the block matching process of the parallax value deriving unit 3 of the object recognition apparatus 1 will be described with reference to FIG.

<Step S1-1>
The image acquisition unit 100 of the parallax value deriving unit 3 images a front subject with the right camera (imaging unit 10a), generates an analog image signal, and obtains a luminance image that is an image based on the image signal. . As a result, an image signal to be subjected to subsequent image processing is obtained. And it transfers to step S2-1.

<Step S1-2>
The image acquisition unit 100 of the parallax value deriving unit 3 captures a front subject with the left camera (imaging unit 10b), generates an analog image signal, and obtains a luminance image that is an image based on the image signal. . As a result, an image signal to be subjected to subsequent image processing is obtained. Then, the process proceeds to step S2-2.

<Step S2-1>
The conversion unit 200 of the parallax value deriving unit 3 performs noise removal, distortion correction, and the like on the analog image signal obtained by imaging by the imaging unit 10b, and converts the analog image signal into digital image data. In this way, by converting the image data into digital image data, image processing for each pixel can be performed on an image based on the image data. And it transfers to step S3-1.

<Step S2-2>
The conversion unit 200 of the parallax value deriving unit 3 performs noise removal, distortion correction, and the like on the analog image signal obtained by imaging by the imaging unit 10a, and converts the analog image signal into digital image data. In this way, by converting the image data into digital image data, image processing for each pixel can be performed on an image based on the image data. Then, the process proceeds to step S3-2.

<Step S3-1>
The conversion unit 200 of the parallax value deriving unit 3 outputs an image based on the digital image data converted in step S2-1 as the reference image Ia in the block matching process. Thus, an image serving as a reference for obtaining a parallax value in the block matching process is obtained. Then, the process proceeds to step S4-1.

<Step S3-2>
The conversion unit 200 of the parallax value deriving unit 3 outputs an image based on the digital image data converted in step S2-2 as the comparison image Ib in the block matching process. Thus, an image to be compared for obtaining a parallax value in the block matching process is obtained. Then, the process proceeds to step S4-2.

<Step S4-1>
The pattern detection unit 300a of the parallax value deriving unit 3 repeatedly detects a pattern area from the reference image Ia obtained in step S3-1. As a result, the pattern detection unit 300a identifies an area causing mismatching from the reference image Ia. Then, the process proceeds to step S5-1.

<Step S4-2>
The pattern detection unit 300b of the parallax value deriving unit 3 repeatedly detects a pattern area from the comparison image Ib obtained in step S3-2. As a result, the pattern detection unit 300b identifies a region that causes mismatching from the comparison image Ib. Then, the process proceeds to step S5-2.

<Step S5-1>
The processing unit 400a of the parallax value deriving unit 3 changes the luminance value of the pixel constituting the repetitive pattern area detected in step S4-1 according to the pixel position of the pixel. Thereby, the luminance value of the pixel constituting the repetitive pattern region is changed by the amount of change corresponding to the pixel position. Then, the process proceeds to step S6.

<Step S5-2>
The processing unit 400b of the parallax value deriving unit 3 changes the luminance value of the pixel constituting the repetitive pattern area detected in step S4-2 according to the pixel position of the pixel. Thereby, the luminance value of the pixel constituting the repetitive pattern region is changed by the amount of change corresponding to the pixel position. Then, the process proceeds to step S6.

<Step S6>
The cost calculation unit 501 of the parallax value calculation processing unit 500 of the parallax value deriving unit 3 determines the dissimilarity between the reference region pb of the processed reference image Ia and the candidate region qb of the processed comparison image Ib by block matching processing. Calculated as a cost value C. Then, the process proceeds to step S7.

<Step S7>
The deriving unit 502 of the parallax value calculation processing unit 500 of the parallax value deriving unit 3 uses the shift amount d corresponding to the minimum value of the cost value C calculated by the cost calculation unit 501 as the target of calculating the cost value C. Derived as a parallax value dp for the pixel of the processed reference image Ia. Then, the deriving unit 502 of the parallax value calculation processing unit 500 of the parallax value deriving unit 3 calculates the luminance value of each pixel of the processed reference image Ia based on the derived parallax value dp, and the parallax value dp corresponding to the pixel. A parallax image that is an image represented by The deriving unit 502 outputs the generated parallax image to the recognition processing unit 5.

  As described above, according to the present embodiment, the processing units 400a and 400b of the parallax value deriving unit 3 are the repetitive pattern regions detected from the reference image Ia (first image) and the comparison image Ib (second image). The pixel value (luminance value) of the pixels constituting the pixel is changed based on the amount of change determined according to the pixel position of the pixel. Then, the parallax value calculation processing unit 500 of the parallax value deriving unit 3 derives the cost value and the parallax value from the processed reference image Ia and the processed comparison image Ib whose luminance values have been changed, and based on the parallax value, the parallax value is calculated. Generate an image. Thereby, even if there is a repeated pattern in the images of the reference image Ia and the comparison image Ib, the minimum value of the cost value C can be specified, so that an appropriate parallax value can be obtained. Moreover, since a parallax image can be generated using an appropriate parallax value, “false steps” can be prevented.

  As mentioned above, although embodiment of this invention was described, the said embodiment was shown as an example and is not intending limiting the range of invention. The above embodiment can be implemented in various other forms, and various omissions, replacements, changes, additions, and the like can be made without departing from the scope of the invention. Moreover, the said embodiment and its deformation | transformation are included in the range of the invention, the summary, and the invention described in the claim, and its equal range.

  For example, in the above-described embodiment, the object recognition device 1 mounted on an automobile as the vehicle 70 has been described. However, the present invention is not limited to this. For example, it is good also as what is mounted in vehicles, such as a motorcycle, a bicycle, a wheelchair, or an agricultural cultivator, as an example of other vehicles. In addition to a vehicle as an example of a moving body, a moving body such as a robot may be used.

  Furthermore, the robot may be an apparatus such as an industrial robot that is fixedly installed in an FA (Factory Automation) as well as a moving body. Further, the device to be fixedly installed may be not only a robot but also a surveillance camera for security.

  In the above embodiment, the block matching process is described as an example of the stereo matching process. However, the present invention is not limited to this, and a process using an SGM (Semi-Global Matching) method may be used.

  In the above embodiment, when at least one of the pattern detection units 300a and 300b, the processing processing units 400a and 400b, and the parallax value calculation processing unit 500 of the object recognition apparatus 1 is realized by executing a program, the program is Provided in advance in a ROM or the like.

  The program executed by the object recognition apparatus 1 according to the above-described embodiment is a file in an installable format or an executable format and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD. You may comprise so that it may record and provide on a readable recording medium. The program executed by the object recognition apparatus 1 of the above-described embodiment may be configured to be stored by being stored on a computer connected to a network such as the Internet and downloaded via the network. Moreover, you may comprise so that the program performed with the object recognition apparatus 1 of the above-mentioned embodiment may be provided or distributed via networks, such as the internet.

  The program executed by the object recognition device 1 of the above-described embodiment has a module configuration including any of the above-described functional units. As actual hardware, the CPU 32 reads out and executes a program from the ROM 33, whereby the above-described functional units are loaded and generated on the main storage device (RAM 34 or the like).

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 2 Main body part 3 Parallax value derivation part 4 Communication line 5 Recognition processing part 6 Vehicle control apparatus 7 Steering wheel 8 Brake pedal 10a, 10b Imaging part 11a, 11b Imaging lens 12a, 12b Aperture 13a, 13b Image sensor 20a, 20b Signal converter 21a, 21b CDS
22a, 22b AGC
23a, 23b ADC
24a, 24b Frame memory 30 Image processing unit 31 FPGA
32 CPU
33 ROM
34 RAM
35 I / F
39 Bus line 51 FPGA
52 CPU
53 ROM
54 RAM
55 I / F
58 CANI / F
59 Bus Line 60 Device Control System 70 Vehicle 100 Image Acquisition Unit 200 Conversion Unit 300a, 300b Pattern Detection Unit 400a, 400b Processing Processing Unit 500 Parallax Value Calculation Processing Unit 501 Cost Calculation Unit 502 Derivation Unit

Japanese Patent No. 5409237 Japanese Patent No. 4856611

Claims (9)

A first image obtained by imaging the subject by the first imaging means, and a second image obtained by imaging the subject by the second imaging means installed at a position different from the first imaging means. And an acquisition means for acquiring
Detecting means for detecting each of repeated pattern regions in which a similar pattern repeatedly appears from the first image and the second image;
Changing means for changing a pixel value of a pixel constituting the repetitive pattern region of the first image and the second image based on a change amount determined according to a pixel position of the pixel;
The degree of coincidence between the reference area of the first image changed by the changing means and each of a plurality of candidate areas that are candidates for areas corresponding to the reference area in the second image changed by the changing means Calculating means for
Derivation means for deriving a parallax value between the reference region and the candidate region based on the degree of coincidence;
A parallax value deriving device comprising:   2. The parallax value deriving device according to claim 1, wherein the changing unit changes a pixel value of a pixel constituting the repetitive pattern region in accordance with a pixel position in an epipolar line direction between the first image and the second image. .   The parallax value deriving device according to claim 2, wherein the changing unit uses the one end in the epipolar line direction as a base point in the repetitive pattern region, and increases a change amount of the pixel value according to a separation distance from the base point.   The parallax value deriving device according to claim 2, wherein the changing unit uses the one end in the epipolar line direction as a base point in the repetitive pattern region, and reduces the change amount of the pixel value according to a separation distance from the base point.   The parallax value derivation according to claim 2, wherein the changing unit sequentially adds the pixel value of the immediately preceding pixel to the pixel value of the next pixel from one end to the other end in the epipolar line direction in the repetitive pattern region. apparatus.   The parallax value deriving device according to claim 1, wherein when the pixel value is saturated due to the change, the changing unit switches an operator related to the change of the pixel value to an operator having an opposite meaning. .   The said change means determines the change method based on the horizontal width of the said epipolar line direction of the said repeating pattern area | region, and the pixel value of each pixel in the said repeating pattern area | region. The parallax value deriving device described. A first image obtained by imaging the subject by the first imaging means, and a second image obtained by imaging the subject by the second imaging means installed at a position different from the first imaging means. And an acquisition step to acquire
A detection step of detecting a repeated pattern region in which a similar pattern repeatedly appears from the first image and the second image;
A change step of changing a pixel value of a pixel constituting the repetitive pattern region of the first image and the second image based on a change amount determined according to a pixel position of the pixel;
A degree of coincidence between the reference area of the first image changed in the changing step and a plurality of candidate areas that are candidates for areas corresponding to the reference area in the second image changed in the changing step is calculated. A calculation step;
A derivation step for deriving a parallax value between the reference region and the candidate region based on the degree of coincidence;
A method for deriving a disparity value. Computer
A first image obtained by imaging the subject by the first imaging means, and a second image obtained by imaging the subject by the second imaging means installed at a position different from the first imaging means. And an acquisition means for acquiring
Detecting means for detecting each of repeated pattern regions in which a similar pattern repeatedly appears from the first image and the second image;
Changing means for changing a pixel value of a pixel constituting the repetitive pattern region of the first image and the second image based on a change amount determined according to a pixel position of the pixel;
The degree of coincidence between the reference area of the first image changed by the changing means and each of a plurality of candidate areas that are candidates for areas corresponding to the reference area in the second image changed by the changing means Calculating means for
Derivation means for deriving a parallax value between the reference region and the candidate region based on the degree of coincidence;
Program to make it function.