JP6554943B2 - Image processing apparatus, image processing method, and evaluation method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an evaluation method.

  Conventionally, a recognition system for recognizing a predetermined target from image information obtained by mounting a camera on a vehicle and photographing a target such as a preceding vehicle is known and used for vehicle driving control and the like. It has been.

  For verification of recognition performance for such a recognition system, for example, correct answer data (correct answer data indicating the area of the predetermined target) is prepared in advance, and the recognition result (recognized in the recognition system) is prepared. This is done by comparing with the data of the target area.

Here, in order to correctly verify the recognition performance, it is required to prepare highly accurate correct answer data for each frame included in the image information. As a method for extracting correct data, a method of performing matching processing with a template image representing a target, calculating a position where the template image matches in each frame, and specifying a region (matching region) corresponding to the calculated position Is mentioned. According to the above method, for example, by performing a matching process using a plurality of template images having different magnifications, even if the distance to the target varies from frame to frame, the template image corresponding to each frame is used. it is possible to identify a matching area used. As a result, correct data can be extracted in frames with different distances to the target.

JP 2000-113158 A International Publication No. 2008/114683

  However, in the case of the above method, if the target has moved or changed in size at the sub-pixel level in each frame, the correct area of the target cannot be recognized, and the wrong area is used as the matching area. I will identify. In this case, incorrect correct answer data is extracted, and verification of recognition performance cannot be performed correctly.

  In one aspect, an object is to realize highly accurate position specification for a target that performs minute movement and size change.

  According to one aspect, there is provided an image processing method for generating correct data for verification when specifying a specific image region from an image obtained by photographing a predetermined direction of a vehicle, wherein the size of a template in pixel units and a template in a frame A process for specifying a specific image region in two stages: a primary evaluation for acquiring the position estimation result and a condition based on the primary evaluation result, and a secondary evaluation for acquiring the estimation result as time-series data between frames. Is executed by a computer.

  It is possible to realize highly accurate position specification for a target that performs minute movement and size change.

It is a figure which shows an example of the whole structure of a recognition performance verification system. It is a figure which shows an example of a function structure of a recognition system. It is a figure for demonstrating the function of a performance verification apparatus. It is a figure which shows an example of the hardware constitutions of a correct answer production | generation apparatus. It is a figure which shows an example of a function structure of a correct answer production | generation apparatus. It is a figure which shows an example of a function structure of an image tracking part. It is a flowchart of correct information generation processing. It is a flowchart of a template production | generation process. It is a figure which shows an example of template information. It is a flowchart of a matching process. It is a figure for demonstrating a matching process. It is a figure which shows an example of matching result information. It is a flowchart of a maximum frame calculation process. It is a figure for demonstrating a maximum frame calculation process. It is a figure which shows the specific example of a maximum frame calculation result. It is a flowchart of a continuation determination process. It is a flowchart of calculation processing between maximum frames. It is a figure which shows the specific example of the calculation process result between maximum frames. It is a figure for demonstrating the cause by which an incorrect matching result is obtained in a matching process.

  Each embodiment will be described below with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[First Embodiment]
First, the overall configuration of a recognition performance verification system for verifying the recognition performance of a recognition system mounted on a vehicle will be described. FIG. 1 is a diagram illustrating an example of the overall configuration of the recognition performance verification system.

  As shown in FIG. 1, the recognition performance verification system 100 includes a vehicle 110, a storage device 120, a correct answer generation device 130, and a performance verification device 140.

  The vehicle 110 is equipped with a recognition system 111. The recognition system 111 captures a target (for example, a forward vehicle) in a predetermined direction (for example, forward) of the vehicle 110, recognizes the forward vehicle from the image information, and performs a warning to the driver and driving control of the vehicle. .

  The target is not limited to a vehicle, and may be any object that can be specified from an image such as a person, an animal, or a building. Moreover, although said vehicle is a passenger car, a bus, a truck, a train etc., for example, it is not limited to this, An airplane, a ship, etc. may be sufficient. Further, the front of the vehicle 110 may be, for example, the front direction of the vehicle 110 main body, or the traveling direction (for example, the rear when the vehicle 110 is backing).

  Image information obtained by the recognition system 111 is stored in the image information storage unit 121 of the storage device 120. In addition, the recognition result for the preceding vehicle recognized by the recognition system 111 is stored in the recognition result storage unit 112.

  The storage device 120 includes an image information storage unit 121 and stores image information obtained by the recognition system 111 of the vehicle 110. Note that the image information stored in the image information storage unit 121 is read and used by the correct answer generation device 130 and the performance verification device 140.

  The correct answer generation device 130 is an image processing device that generates correct answer information (correct answer data) of each frame included in the image information. A correct answer generating program is installed in the correct answer generating device 130, and the correct answer generating device 130 functions as the correct answer generating unit 131 by executing the correct answer generating program.

  The correct answer generation unit 131 reads the image information stored in the image information storage unit 121, and generates correct answer information (information for specifying the vehicle area of the preceding vehicle) for the preceding vehicle for each frame. In addition, the correct answer generation unit 131 stores the generated correct answer information in the correct answer information storage unit 132 as correct answer information for verification for verifying the recognition result (verification target) of the recognition system 111.

  The performance verification device 140 includes a performance comparison unit 141. For each frame included in the image information 151 read from the image information storage unit 121, the performance comparison unit 141 recognizes the recognition result information 152 read from the recognition result storage unit 112, the correct answer information 153 read from the correct answer information storage unit 132, and Compare

  Further, the performance comparison unit 141 verifies the recognition performance of the recognition system 111 based on the comparison result, and stores the performance comparison result in the performance comparison result storage unit 142.

  1 recognizes that the number of vehicles is one, the number of vehicles may be plural. The storage device 120, the correct answer generation device 130, and the performance verification device 140 may be realized by, for example, one server device.

  Next, a functional configuration of the recognition system 111 mounted on the vehicle 110 will be described. FIG. 2 is a diagram illustrating an example of a functional configuration of the recognition system. As illustrated in FIG. 2, the recognition system 111 includes an imaging unit 201, a recognition unit 202, a distance calculation unit 203, a warning unit 204, and a control unit 205.

  The imaging unit 201 is, for example, a camera, and images the front vehicle 210 that travels in front of the vehicle 110. Image information captured by the imaging unit 201 is transmitted to the storage device 120 and stored in the image information storage unit 121. Further, image information captured by the imaging unit 201 is input to the recognition unit 202.

  The recognition unit 202 performs image recognition processing on each frame included in the image information, and recognizes the vehicle ahead in each frame. Further, the recognition unit 202 uses the recognition result (the position (coordinate) of the area indicating the preceding vehicle in the frame, the size of the preceding vehicle) as information for specifying the vehicle area (specific image area). 112. Further, the recognition unit 202 inputs the recognition result to the distance calculation unit 203.

  The distance calculation unit 203 calculates the distance to the preceding vehicle based on the recognition result by the recognition unit 202. The distance calculation unit 203 outputs the calculated distance to the warning unit 204 and the control unit 205.

  The warning unit 204 warns the occupant by presenting information to the occupant of the vehicle 110 according to the distance calculated by the distance calculation unit 203. The control unit 205 controls the traveling speed of the vehicle 110 according to the distance calculated by the distance calculation unit 203.

  Next, functions of the performance verification device 140 will be described. FIG. 3 is a diagram for explaining the function of the performance verification apparatus.

  As shown in FIG. 3, in the performance comparison unit 141 of the performance verification device 140, recognition result information by the recognition system 111 about the drawing of the forward vehicle in each frame included in the image information 151 (hereinafter simply referred to as the forward vehicle). 152 is read from the recognition result storage unit 112.

  As described above, the recognition result information 152 includes the position (coordinates) of the area indicating the forward vehicle 301 in the frame and the size of the area indicating the forward vehicle 301 (that is, the vehicle area (specific image area) Information to identify). In the example of FIG. 3, (Xe, Ye) is read out as the coordinates (X coordinate and Y coordinate) of the area indicating the forward vehicle 301, and (We and Weight) as the size (width and height) of the area indicating the forward vehicle 301. , He) is read.

  Further, the performance comparison unit 141 reads the correct answer information 153 from the correct answer information storage unit 132 for the preceding vehicle 301 in each frame included in the image information 151. In the example of FIG. 3, as the correct answer information 153, the coordinates (Xr, Yr) of the area indicating the preceding vehicle 301 are read, and the size (Wr, Hr) of the area indicating the preceding vehicle 301 is read. Show. That is, the case where the information for specifying the vehicle area | region of the front vehicle 301 is read is shown.

  The performance comparison unit 141 compares the recognition result information 152 and the correct answer information 153, and calculates, for example, a difference value such as the size of the area where the achieved coordinates are achieved. Further, the performance comparison unit 141 verifies the recognition performance of the recognition system 111 based on the calculated difference value, and stores it in the performance comparison result storage unit 142 as a performance comparison result. The performance comparison unit 141 stores the performance comparison result in association with the frame number. Further, when a plurality of front vehicles are included in the frame, the performance comparison unit 141 stores the vehicle identifiers in association with the vehicle identifiers indicating the respective vehicles.

  Next, details of the correct answer generation device 130 will be described. FIG. 4 is a diagram illustrating an example of a hardware configuration of the correct answer generation apparatus.

  As illustrated in FIG. 4, the correct answer generation device 130 includes a CPU 401, a ROM (Read Only Memory) 402, a RAM (Random Access Memory) 403, an auxiliary storage unit 404, and a communication unit 405. The correct answer generation device 130 includes a display unit 406, an operation unit 407, and a drive unit 408. Note that the units of the correct answer generation device 130 are connected to each other via a bus 409.

  The CPU 401 is a computer that executes various programs (for example, a correct answer generation program) installed in the auxiliary storage unit 404. The ROM 402 is a nonvolatile memory. The ROM 402 functions as a main storage unit that stores various programs, data, and the like necessary for the CPU 401 to execute various programs stored in the auxiliary storage unit 404. Specifically, the ROM 402 stores a boot program such as a basic input / output system (BIOS) or an extensible firmware interface (EFI).

  The RAM 403 is a volatile memory such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory), and functions as a main storage unit. The RAM 403 provides a work area that is expanded when various programs stored in the auxiliary storage unit 404 are executed by the CPU 401.

  The auxiliary storage unit 404 stores various programs, information generated by executing the various programs (for example, correct information 153), and the like.

  The communication unit 405 is a device for the correct answer generation device 130 to communicate with the storage device 120 and the performance verification device 140.

  The display unit 406 displays the image information read from the image information storage unit 121 of the storage device 120 and the correct answer information generated by the correct answer generating unit 131.

  The operation unit 407 is a device for an operator of the correct answer generating device 130 to input various instructions to the correct answer generating device 130.

  The drive unit 408 is a device for setting the recording medium 410. The recording medium 410 here includes a medium for recording information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, or the like. The recording medium 410 also includes a semiconductor memory that electrically records information, such as a ROM and a flash memory.

  In the present embodiment, the various programs stored in the auxiliary storage unit 404 are, for example, the distributed recording medium 410 set in the drive unit 408, and the various programs recorded in the recording medium 410 are stored in the drive unit 408. Installed by reading. Alternatively, it is installed by being downloaded from the network via the communication unit 405.

  Next, a functional configuration of the correct answer generation apparatus 130 will be described. FIG. 5 is a diagram illustrating an example of a functional configuration of the correct answer generation apparatus. As illustrated in FIG. 5, the correct answer generation unit 131 of the correct answer generation device 130 includes a video display unit 501, a frame specification reception unit 502, an area specification reception unit 503, an image tracking unit 504, and a correction unit 505.

  The video display unit 501 reads the image information designated by the operator of the correct answer generation device 130 from the image information storage unit 121 and displays it on the display unit 406. In addition, the video display unit 501 transmits the read image information to the image tracking unit 504.

  The frame designation accepting unit 502 accepts designation of one frame from each frame of the image information displayed on the display unit 406 and transmits a frame number for identifying the accepted frame to the image tracking unit 504. To do. In the present embodiment, the operator designates a frame including the preceding vehicle among the frames of the image information displayed on the display unit 406. In addition, the operator designates a frame in which the preceding vehicle is located closer (that is, a frame in which the preceding vehicle appears larger).

  The area designation accepting unit 503, when an area is designated by the operator in the frame designated via the frame designation accepting unit 502, the area information of the designated area (X coordinate and Y coordinate indicating the position of the area). And the width and height indicating the size of the area). In addition, the region designation receiving unit 503 transmits the received region information to the image tracking unit 504. The example of FIG. 5 indicates that (X0, Y0) is transmitted as the coordinates of the region and (W0, H0) is transmitted as the size of the region. In the present embodiment, the operator designates an area indicating the preceding vehicle in the designated frame.

  The image tracking unit 504 generates a template image based on the frame specified by the frame specification receiving unit 502 and the region specified by the region specification receiving unit 503. In the present embodiment, the template image is an image used when performing a matching process for extracting a region indicating a predetermined target (front vehicle) from each frame included in the image information.

  The image tracking unit 504 extracts matching regions (Xr, Yr, Wr, Hr) from each frame by performing matching processing of each frame included in the image information using the generated template image, and as correct answer information Output. In addition, the image tracking unit 504 displays the extracted matching area superimposed on each frame.

  When the operator determines that the correction information output from the image tracking unit 504 needs to be corrected, the correction unit 505 receives a correction operation by the operator for the correct information of the frame. Further, correct information corrected based on the correction operation of the operator is stored in the correct information storage unit 132 in association with the frame.

  Next, details of the image tracking unit 504 of the correct answer generating unit 131 will be described. FIG. 6 is a diagram illustrating a detailed functional configuration of the image tracking unit.

  As illustrated in FIG. 6, the image tracking unit 504 includes a template generation unit 601, a matching calculation unit 602, a maximum frame calculation unit 603, and a maximum interframe calculation unit 604.

  The template generation unit 601 extracts a frame specified based on the frame number transmitted by the frame designation reception unit 502 from the image information transmitted from the video display unit 501. Further, the template generation unit 601 extracts the region designated by the region designation receiving unit 503 from the extracted frame. Further, the template generation unit 601 generates a template image based on the extracted area and stores it in the template information storage unit 611.

  The matching calculation unit 602 performs a matching process on each frame included in the image information using each template image stored in the template information storage unit 611. The matching calculation unit 602 calculates a position matching result (“matching position”) for each template image for each frame by performing a matching process. In addition, the matching calculation unit 602 calculates a “matching degree” at a position where each template image matches for each frame by performing a matching process. In addition, the matching calculation unit 602 performs a matching process, so that for each frame, a “matching region” (in the frame in which the matching degree is calculated) based on the position where each template image matches and the size of each template image. Area).

  Further, the matching calculation unit 602 associates a template number indicating each template image with a matching position and a matching degree when matching processing is performed using each template image, and a matching result information storage unit as matching result information Stored in 612. Note that the matching calculation unit 602 stores the template number, the matching position, and the matching degree in association with the frame.

The maximum frame calculation unit 603 extracts a maximum frame based on the matching result information stored in the matching result information storage unit 612. The detailed description of the maximal frame will be described later. Here, the maximal frame is defined as follows.
(A) One frame among a plurality of frames that are continuous in a predetermined section.
(B) Among the template images, when a matching process is performed on a plurality of consecutive frames in the predetermined section using a template image having the maximum matching degree, the frame having the maximum matching degree in the predetermined section is there.
(C) In (b) above, the template image having the maximum matching degree is a template image in which the difference from the matching degree of other template images is equal to or greater than a predetermined threshold.

  The maximum frame calculation unit 603 stores the extracted frame number indicating the maximum frame in the maximum frame information storage unit 613 as maximum frame information. Note that the maximum frame calculation unit 603 associates the size of the template image with the maximum matching degree used when extracting at least the maximum frame with the matching position in the maximum frame, and sets the maximum frame information as the maximum frame information. Store in the storage unit 613.

  The maximum inter-frame calculation unit 604 corrects each frame between the maximum frames based on the size of the template image and the matching position stored in association with each maximum frame stored in the maximum frame information storage unit 613. Constraint conditions for generating information are calculated.

Note that the detailed description of the constraint condition will be described later, and here, the constraint condition is defined as follows.
(A) This is a condition for performing matching processing for each frame between consecutive maximal frames among the maximal frames stored in the maximal frame information storage unit 613.
(B) The matching process for each frame between consecutive maximal frames is constrained by the size of the template image stored in association with the maximal frame and the matching position.

  For each maximal frame, the maximal inter-frame calculation unit 604 outputs, as correct answer information, a matching area that is read from the maximal frame information storage unit 613 and specified based on the size of the template image and the matching position.

  Further, the maximum inter-frame calculation unit 604 calculates a matching area specified based on the size of the template image and the matching position, which is calculated by performing the matching process based on the constraint condition for each frame between the maximum frames. , Output as correct answer information.

  As described above, the image tracking unit 504 includes a first acquisition unit (matching calculation unit 602, maximum frame calculation unit 603) that acquires a matching processing result for each frame as a primary evaluation. Further, the image tracking unit 504 constrains the condition with the maximal frame calculated from the matching processing result (primary evaluation result), performs the matching process as time-series data between the maximal frames, and acquires the second evaluation. It has an acquisition means (maximum interframe calculation part 604). As described above, by performing the evaluation process in a plurality of stages (in the above example, two stages), the image tracking unit 504 extracts a specific image area for identifying the vehicle area of the preceding vehicle as correct information. Note that the stage of evaluation is not limited to two stages. For example, the above-described primary evaluation and secondary evaluation may be included in three or more stages of evaluation.

  Next, the flow of correct information generation processing by the image tracking unit 504 will be described. FIG. 7 is a flowchart of correct information generation processing.

  In step S <b> 701, the template generation unit 601 executes a template generation process, generates a template image, and stores the generated template image in the template information storage unit 611.

  In step S <b> 702, the matching calculation unit 602 performs matching processing on the frame of the image information, and stores the matching result information in the matching result information storage unit 612.

  In step S <b> 703, the maximal frame calculation unit 603 performs a maximal frame calculation process on the image information frame, and stores the maximal frame information in the maximal frame information storage unit 613.

  In step S704, the maximum frame calculation unit 603 executes a continuation determination process, and determines whether or not to continue the matching process (step S702) and the maximum frame calculation process (step S703) for the next frame.

  If it is determined in step S705 that the maximum frame calculation unit 603 continues (Yes in step S705), the process returns to step S702. Thereby, the matching process (step S702) and the maximum frame calculation process (step S703) are continued for the next frame.

  On the other hand, if it is determined in step S705 that the maximum frame calculation unit 603 does not continue (No in step S705), the process proceeds to step S706.

  In step S <b> 706, the maximum interframe calculation unit 604 performs a maximum interframe calculation process, and calculates correct information by performing a matching process for each frame between the maximum frames.

  In step S707, the maximum interframe calculation unit 604 outputs all correct information including the correct information calculated in step S706 to the correction unit 505, and ends the correct information generation process.

  Next, each step of the correct information generation processing (template generation processing (S701) in FIG. 7, matching processing (S702), maximum frame calculation processing (S703), continuation determination processing (S704), maximum local frame calculation processing (S706). ) Will be described in order below.

  First, the template generation process (step S701) will be described. FIG. 8 is a flowchart of a template generation process executed by the template generation unit.

  In step S801, the template generation unit 601 cuts out an initial template image. Specifically, among the frames of the image information transmitted from the video display unit 501, the frame is identified by the frame number transmitted from the frame designation receiving unit 502 and transmitted from the region designation receiving unit 503. The area specified by the area information is cut out.

  In step S802, the template generation unit 601 stores the region cut out in step S801 in the template information storage unit 611 as an initial template image.

  In step S803, the template generation unit 601 reduces the template image when the initial template image stored in step S802 is reduced by a predetermined number of pixels (for example, one pixel) in the width direction and the height direction. Calculate the size.

  In step S804, the template generation unit 601 determines whether the size of the template image calculated in step S803 satisfies a predetermined size condition. The predetermined size condition refers to the size range of the template image to be generated when performing the matching process for each frame.

  If it is determined in step S804 that the predetermined size condition is satisfied (Yes in step S804), the process proceeds to step S805, and a template image having the size calculated in step S803 is generated. Thereafter, the process returns to step S802, and the template image generated in step S805 is stored in the template information storage unit 611.

  On the other hand, if it is determined in step S804 that the predetermined size condition is not satisfied (No in step S804), the template generation process ends.

  In this way, the template generation processing is executed by the template generation unit 601, and thus the generated template image is stored in the template information storage unit 611.

  FIG. 9 is a diagram illustrating an example of template information. As shown in FIG. 9, the template information 900 includes “template number”, “template”, and “template size” as information items.

  In the “template number”, a number for specifying the template image generated by the template generation unit 601 is stored. The template number is assigned in association with the template image when the template generation unit 601 stores the generated template image in the template information storage unit 611.

  The “template” stores a template image generated by the template generation unit 601.

  The “template size” stores the size (width × height) of the template image generated by the template generation unit 601.

  Next, the matching process (step S702) will be described. FIG. 10 is a flowchart of the matching process executed by the matching calculation unit.

  In step S1001, the matching calculation unit 602 acquires a template number (α). The template number (α) is a number determined in step S703 (maximum frame calculation process) one loop before in the correct information generation process shown in FIG. Therefore, the matching calculation unit 602 acquires the template number (α) determined in step S703. If the correct information generation process shown in FIG. 7 is the first loop, the matching calculation unit 602 acquires a predetermined initial value as the template number (α).

  In the present embodiment, the matching calculation unit 602 performs matching processing on one frame using three template images.

  In step S1002, the matching calculation unit 602 acquires a frame having a frame number (n). The frame number (n) is a number determined in step S703 (maximum frame calculation process) one loop before the correct information generation process shown in FIG. Therefore, the matching calculation unit 602 acquires the frame number (n) determined in step S703. If the correct information generation process shown in FIG. 7 is the first loop, the matching calculation unit 602 acquires a predetermined initial value as the frame number (n).

  In step S1003, the matching calculation unit 602 determines whether the frame having the frame number (n) has been acquired in step S1002. If it is determined in step S1003 that acquisition has failed (No in step S1003), the matching process ends.

  On the other hand, if it is determined in step S1003 that it has been acquired (Yes in step S1003), the process proceeds to step S1004. In step S1004, the matching calculation unit 602 reads three template images specified by the template numbers (α), (α + 1), and (α + 2) from the template information storage unit 611.

  In step S1005, the matching calculation unit 602 extracts the frame specified by the frame number (n) from the image information.

  In step S1006, the matching calculation unit 602 calculates the matching degree and the matching position using each template image for the processing target frame. Specifically, the matching calculation unit 602 performs a matching process on the frame with the frame number (n) using the template image with the template number (α), and calculates the matching position and the matching degree.

  In addition, the matching calculation unit 602 performs a matching process on the frame of the frame number (n) using the template image of the template number (α + 1), and calculates the matching position and the matching degree.

  Further, the matching calculation unit 602 performs a matching process on the frame with the frame number n using the template image with the template number (α + 2), and calculates the matching position and the matching degree.

  In step S1007, the matching calculation unit 602 stores the matching position and the matching degree calculated using each template image in step S1006 in the matching result information storage unit 612.

  Next, a specific example of matching processing by the matching calculation unit 602 will be described with reference to FIG. FIG. 11 is a diagram for explaining the matching process.

FIG. 11A shows a case where the matching process is performed on the frame 1110 with the frame number (n) using the template image 1101 specified by the template number (α). In the example of FIG. 11A, it is shown that the template image 1101 is matched in the area indicated by the dotted line 1111. Note that the region indicated by the dotted line 1111 is specified based on the matching position (X _n _ (α), Y _n _ (α)) and the size of the template image 1101, and the matching degree is M _n _ (α). .

FIG. 11B shows a case where the matching process is performed on the frame 1110 with the frame number n using the template image 1102 specified by the template number (α + 1). In the example of FIG. 11B, the template image 1102 is matched in the area indicated by the dotted line 1112. Note that the region indicated by the dotted line 1112 is specified based on the matching position (X _n _ (α + 1), Y _n _ (α + 1)) and the size of the template image 1102, and the matching degree is M _n _ (α + 1). .

FIG. 11C shows a case where the matching process is performed on the frame 1110 with the frame number (n) using the template image 1103 specified by the template number (α + 2). In the example of FIG. 11C, the template image 1103 is matched in the area indicated by the dotted line 1113. The region indicated by the dotted line 1113, matching position _{(X n _ (α + 2} ), Y n _ (α + 2)) is specified based on the size of the template image 1103, the matching degree is _ _M n (α + 2) .

  Next, a specific example of the matching result information stored in the matching result information storage unit 612 by the matching processing by the matching calculation unit 602 will be described. FIG. 12 is a diagram illustrating an example of matching result information.

  As shown in FIG. 12, the matching result information 1200 includes “frame number”, “template number”, “template image size”, “matching degree”, and “matching position” as information items.

  The “frame number” stores the frame number of the frame that is the target of the matching process.

  The “template number” stores a template number for specifying a template image used when matching processing is performed on a frame to be processed.

  The “size of the template image” stores the size (width, height) of the template image specified by each template number.

  The “matching degree” stores the matching degree when matching processing is performed using each template image.

  The “matching position” stores a matching position when matching processing is performed using each template image.

  Next, the maximum frame calculation process (step S703) will be described. FIG. 13 is a flowchart of the maximum frame calculation process. Here, the maximum frame calculation process for determining whether the frame with the frame number (n−1) is a maximum frame with the frame with the frame number (n) as a processing target will be described.

  In step S <b> 1301, the maximum frame calculation unit 603 calculates a template image for which the maximum matching degree is calculated among matching degrees when each template image performs matching processing on the frame with the frame number (n−1). It is determined whether or not there is.

  If it is determined in step S1301 that there is no template image for which the maximum matching degree has been calculated (No in step S1301), the process proceeds to step S1307. On the other hand, if it is determined in step S1301 that there is a template image for which the maximum matching degree is calculated (Yes in step S1301), the process proceeds to step S1302.

  In step S <b> 1302, the maximum frame calculation unit 603 reads, from the matching result information 1200, the matching degree when matching processing is performed using each template image for the frame with the frame number (n−1). It is assumed that the template number (α), (α + 1), and (α + 2) template images are used in the matching process for the frame number (n−1).

For this reason, in step S1302, the maximum frame calculation unit 603 reads M _n−1 _ (α) as the matching degree of the template image with the template number (α). Also, M _n−1 _ (α + 1) is read as the matching degree of the template image with the template number (α + 1). Further, M _n−1 _ (α + 2) is read as the matching degree of the template image with the template number (α + 2).

It is assumed here that M _n−1 _ (α + 1) is the maximum matching degree among the matching degrees read in step S1302.

In step S1303, the maximal frame calculation unit 603 determines whether or not the difference between the maximum matching degree read out in step S1302 and the other matching degrees is greater than a predetermined threshold value TH. In other words, it is determined whether M _n−1 _ (α + 1) −TH> M _n−1 _ (α) and M _n−1 _ (α + 2) are satisfied.

  If it is determined in step S1303 that it is equal to or smaller than the predetermined threshold TH (No in step S1303), the process proceeds to step S1307. On the other hand, if it is determined that it is greater than the predetermined threshold TH (Yes in step S1303), the process proceeds to step S1304.

In step S1304, the maximal frame calculation unit 603 reads from the matching result information 1200 the matching degree when the matching process is performed on the frame with the frame number (n) using the template image with the template number (α + 1). Specifically, the matching degree M _n _ (α + 1) is read out. In addition, the maximum frame calculation unit 603 compares the read matching degree M _n _ (α + 1) with the matching degree M _n−1 _ (α + 1).

As a result of the comparison in step S1304, when the matching degree M _n−1 _ (α + 1) one frame before is equal to or lower than the matching degree M _n _ (α + 1) of the current processing target frame (No in step S1304). The process proceeds to step S1307.

On the other hand, as a result of the comparison in step S1304, when the matching degree M _n-1 _ (α + 1) one frame before is greater than the matching degree M _n _ (α + 1) of the current processing target frame (in step S1304, In Yes), it progresses to step S1305.

  In step S <b> 1305, the maximum frame calculation unit 603 stores the previous frame (frame number (n−1)) as the maximum frame in the maximum frame information storage unit 613.

  In step S1306, the maximal frame calculation unit 603 determines a template number used when executing a matching process (step S702) and a maximal frame calculation process (step S703) for the next frame (frame number (n + 1) frame). .

For example, it is assumed that the maximum matching degree is M _n−1 _ (α + 1). In this case, the maximum frame calculation unit 603 uses the template images of the template numbers (α), (α + 1), and (α + 2) in the matching process (step S702) and the maximum frame calculation process (step S703) for the next frame. To decide.

For example, assume that the maximum matching degree is M _{n —} (α). In this case, the maximal frame calculation unit 603 obtains template images of template numbers (α−1), (α), and (α + 1) in the matching process (step S702) and the maximal frame calculation process (step S703) for the next frame. Decide to use.

  That is, the maximum frame calculation unit 603 uses the template image for which the maximum matching degree has been calculated, the template image having a size one larger than the template image and the template image having a size smaller by one for matching processing of the next frame. Decide to use.

  In step S1307, the maximum frame calculation unit 603 increments the frame number, sets the next frame as a processing target, and ends the maximum frame calculation process.

  Next, a specific example of the maximal frame calculation process by the maximal frame calculation unit 603 will be described with reference to FIGS. FIG. 14 is a diagram for explaining the maximum frame calculation process. In FIG. 14, for simplification of explanation, two types of template images are used when matching processing is performed for each frame.

  In FIG. 14A, the horizontal axis represents the frame number, and the vertical axis represents the matching degree in the frame of each frame number. A polygonal line 1401 indicates a change in the matching degree when the matching process is performed using the template image of the template number (α + 1). Furthermore, a broken line 1402 indicates a change in the matching degree when the matching process is performed using the template image of the template number (α).

  As shown in FIG. 14A, in each frame included in the section 1411, the matching degree when the template image of the template number (α) is used and the matching degree when the template image of the template number (α + 1) is used. The difference from the degree is larger than a predetermined threshold value TH. Therefore, each frame included in the section 1411 is a candidate for the maximum frame.

  Therefore, attention is paid to the matching degree included in the section 1411 (a broken line 1401 of the section 1411). The broken line 1401 in the section 1411 starts to decrease for the first time when moving from the frame number (n−1) to the frame number (n). That is, in the section 1411, it can be seen that the matching degree corresponding to the frame number (n−1) is the maximum. Therefore, the maximum frame calculation unit 603 determines that the frame with the frame number (n−1) is the maximum frame.

  Similarly, in FIG. 14B, the horizontal axis represents the frame number, and the vertical axis represents the matching degree in the frame of each frame number. A broken line 1421 indicates a change in the matching degree when the matching process is performed using the template image of the template number (α). Further, a broken line 1422 indicates a change in the matching degree when the matching process is performed using the template image of the template number (α + 1).

  However, in the case of FIG. 14B, in each frame included in the section 1431, the matching degree when using the template image of the template number (α) and the matching when using the template image of the template number (α + 1). The difference from the degree is equal to or less than a predetermined threshold value TH. Therefore, each frame included in the section 1431 cannot be a maximum frame.

  FIG. 15 is a diagram illustrating a specific example of the maximum frame calculation result. In FIG. 15A, the horizontal axis indicates the frame number, and the vertical axis indicates the size of the template image (here, the width of the template image). A straight line 1501 indicates the true width of the target (the forward vehicle 301) in each frame.

  In the example of FIG. 15A, it is shown that the frame with frame number 1 is determined to be a maximal frame. Further, it is indicated that the width of the template image for which the maximum matching degree is calculated among the template images used when performing the matching process for the frame determined to be the maximal frame is W1.

  In the example of FIG. 15A, it is shown that the frame with frame number 6 is determined to be a maximal frame. Further, it is indicated that the width of the template image for which the maximum matching degree is calculated among the template images used when performing the matching process for the frame determined to be the maximal frame is W2.

  Further, in the example of FIG. 15A, it is indicated that the frame having the frame number 11 is determined to be the maximum frame. Further, it is indicated that the width of the template image for which the maximum matching degree is calculated among the template images used when performing the matching process for the frame determined to be the maximal frame is W3.

  Furthermore, in the example of FIG. 15A, it is shown that the frame with frame number 16 is determined to be a maximal frame. In addition, for the frame determined to be the maximum frame, among the matching images used when performing the matching process, the width of the template image for which the maximum matching degree is calculated is W4.

  As described above, the maximum frame calculation unit 603 determines that a frame in which the true width of the target (the forward vehicle 301) and the width of the template image (pixel-unit width) match is the maximum frame.

  FIG. 15B shows a state in which the determination result in FIG. 15A is stored in the maximum frame information storage unit 613 as maximum frame information.

  As shown in FIG. 15B, the maximum frame information 1510 includes information items such as “maximum frame number”, “frame number”, “template number”, “size of template image”, “matching degree”, “ "Matching position" is included.

  The “maximum frame number” stores a unique number assigned to a frame determined to be a maximum frame. The “frame number” stores the frame number of the frame determined to be the maximum frame.

  The “template number” stores a template number indicating a template image used when the maximum matching degree is calculated for a frame determined to be a maximal frame.

  The “size of the template image” stores the size (width, height) of the template image specified by each template number.

  The “matching degree” stores the maximum matching degree calculated for the frame determined to be the maximum frame.

  The “matching position” stores the matching position (coordinates) of the template image used when the maximum matching degree is calculated for the frame determined to be the maximum frame.

In the example of FIG. 15B, it is shown that the frame with frame number 1 is determined to be the maximal frame and stored as the maximal frame number 1 in the maximal frame information storage unit 613. Further, in the example of FIG. 15B, the maximum frame of the maximum frame number 1 is subjected to matching processing using the template image of the template number 1, thereby matching positions (X ₁ _ (1), Y ₁ _ ( 1)) and the degree of matching M ₁ _ (1) is calculated.

  Next, the continuation determination process (step S704) will be described. FIG. 16 is a flowchart of the continuation determination process executed by the maximal frame calculation unit. Here, a case where the continuation determination process is executed for the frame with the frame number (n) will be described.

In step S1601, the maximum frame calculation unit 603 obtains the maximum matching degree among the matching degrees when the matching process is performed using each template image for the frame of the frame number (n). Here, it is assumed that the matching degree when the template image of the template number (α + 1) is used is the maximum. Therefore, the maximum frame calculation unit 603 obtains the maximum matching degree M _n _ (α + 1).

In step S1602, the maximum frame calculation unit 603 determines whether or not the maximum matching degree M _n _ (α + 1) is less than a predetermined matching degree threshold Mth. If it is determined in step S1602 that the threshold value is equal to or greater than the predetermined matching degree threshold Mth (No in step S1602), the process proceeds to step S1604 and it is determined that the process is continued. In this case, in the correct answer information generation process of FIG. 7, the matching process (step SS702) and the maximum frame calculation process (step S703) are continued.

  On the other hand, if it is determined in step S1602 that it is less than the predetermined matching degree threshold Mth (Yes in step S1602), the process proceeds to step S1603, and it is determined not to continue the processing. In this case, in the correct answer information generation process of FIG. 7, the process proceeds to step S706, and the inter-maximum frame calculation process is executed.

  Next, the maximum interframe calculation process (step S706) will be described. FIG. 17 is a flowchart of the maximum inter-frame calculation process.

  In step S <b> 1701, the local maximum frame calculation unit 604 reads the local maximum frame (first local maximum frame) of the first local maximum frame number (Ns) from the local maximum frame information storage unit 613. The first maximal frame refers to the oldest maximal frame when two consecutive maximal frames are read from a plurality of maximal frames stored in the maximal frame information storage unit 613.

  In step S <b> 1702, the maximum interframe calculation unit 604 reads the maximum frame (second maximum frame) having the maximum frame number (Ns + 1) from the maximum frame information storage unit 613. The second maximal frame refers to a new maximal frame when two consecutive maximal frames are read from a plurality of maximal frames stored in the maximal frame information storage unit 613.

  In step S1703, the maximal inter-frame calculation unit 604 generates a constraint condition from the first maximal frame and the second maximal frame. In the present embodiment, the constraint condition generated by the maximum inter-frame calculation unit 604 includes the type of template image when performing the matching process and the range in the frame where the matching process is performed.

  For example, it is assumed that the type of template image used when the matching degree is calculated for the first maximal frame is the template image of template number 1 (that is, the template image of width W1 × height H1). ). Further, it is assumed that the type of the template image used when the matching degree is calculated for the second maximum frame is the template image with the template number 2 (that is, the width W2 × the height H2). To do). In this case, the constraint condition regarding the type of template image is “any of template number 1 or 2”.

The matching position for the first maximal frame is (X ₁ _ (1), Y ₁ _ (1)), and the matching position for the second maximal frame is (X ₆ _ (2), Y ₆ _ (2)). In this case, the constraint condition regarding the range in which the matching process is performed is “a predetermined range including two matching regions”. In particular,
A matching region based on the matching position (X ₁ _ (1), Y ₁ _ (1)) and the size (W1 × H1) of the template image,
A matching region based on the matching position (X ₆ _ (2), Y ₆ _ (2)) and the size of the template image (W2 × H2),
It becomes the predetermined range including.

  In step S1704, the local maximum frame calculation unit 604 performs matching processing on each frame between the local maximum frame and the second local maximum frame under the constraint condition generated in step S1703.

  In step S <b> 1705, the inter-maximum frame calculation unit 604 determines the template image type and matching position used when calculating the maximum matching degree for each frame between the first maximum frame and the second maximum frame. Based on the above, a matching area is calculated.

  In step S1706, the inter-maximum frame calculation unit 604 determines whether or not the processing from step S1702 to step S1705 has been executed for all local maximum frames. If it is determined in step S1706 that there is a maximum frame that has not yet been processed (No in step S1706), the process proceeds to step S1707.

  In step S1707, the maximum local frame calculation unit 604 increments the first maximum frame number (Ns). Then, it returns to step S1702 and performs the processing from step S1702 to step S1705.

  On the other hand, if it is determined in step S1706 that the process has been executed for all maximal frames (Yes in step S1706), the maximal frame calculation process ends.

  Next, a specific example of the maximal interframe calculation process by the maximal interframe calculation unit 604 will be described with reference to FIGS. 18 and 19.

  FIG. 18 is a diagram illustrating a specific example of the maximum inter-frame calculation processing result. In FIG. 18, the horizontal axis indicates the frame number, and the vertical axis indicates the size of the template image (here, the width of the template image).

  The example of FIG. 18A shows a specific example of the maximal inter-frame calculation process in the case where it is determined that each frame of frame numbers 1, 6, 11, and 16 is a maximal frame (FIG. 15A). Is.

  As shown in FIG. 18A, the maximum matching degree is calculated by using the template image with the width W1 for the frame with frame number 1 determined to be the maximum frame (maximum frame number (1)). ing. Further, the maximum matching degree is calculated by using the template image having the width W2 for the frame having the frame number 6 determined to be the maximum frame (maximum frame number (2)).

For this reason, when performing a matching process about the frame between maximal frames (namely, each frame of frame numbers 2-5), the constraint conditions regarding the type of template image are:
・ "Template number 1 or 2"
It becomes.

In addition, the constraint condition regarding the range to perform the matching process is
"Matching region based on matching position (X ₁ _ (1), Y ₁ _ (1)) and template image size (W1 × H1)",
· "Matching position _{(X 6 _ (2),} Y 6 _ (2)) and the size of the template image (W2 × H2) matching area based",
And a predetermined range.

  Therefore, the maximal inter-frame calculation unit 604 performs the matching process for each frame of frame numbers 2 to 5 under these constraint conditions. Thereby, correct information with a small error can be calculated for each frame of frame numbers 2 to 5.

  FIG. 18B shows a processing result when matching processing is performed regardless of the constraint condition for comparison. When matching processing is performed for each of the frame numbers 2 to 5 regardless of the constraint condition, for example, for the frame of frame number 5, the maximum matching degree is calculated from the template image with width = W3. That is, when the matching process is performed regardless of the constraint condition, correct information with a large error is calculated in the frame of frame number 5.

  Similarly, as shown in FIG. 18A, the frame with the frame number 11 determined to be the maximal frame (maximum frame number (3)) uses the template image with the width W3, so that the maximum matching degree is obtained. Is calculated.

For this reason, when performing a matching process about the frame between maximum frames (namely, each frame of the frame numbers 7-10), the constraint conditions regarding the type of template image are:
・ "Template number 2 or 3"
It becomes.

In addition, the constraint condition regarding the range to perform the matching process is
· "Matching position _{(X 6 _ (2),} Y 6 _ (2)) and the size of the template image (W2 × H2) matching area based",
· "Matching position _{(X 11 _ (3),} Y 11 _ (3)) and the matching area based on the size (W3 × H3) of the template image",
And a predetermined range.

  Therefore, the maximum inter-frame calculation unit 604 performs matching processing on each frame of frame numbers 7 to 10 under these constraint conditions. Thereby, correct information with a small error can be calculated for each frame of frame numbers 7 to 10.

  When matching processing is performed for each frame number 7 to 10 regardless of the constraint condition, for example, for a frame number 8, a template image with a width = W1 is used as shown in FIG. 18B. Is used to calculate the maximum matching degree. That is, when matching processing is performed regardless of the constraint condition, correct information with a large error is calculated in the frame of frame number 8.

  Similarly, as shown in FIG. 18A, the frame with frame number 16 determined to be the maximum frame (maximum frame number (4)) uses the template image with the width W4, so that the maximum matching degree is obtained. Is calculated.

For this reason, when performing a matching process about the frame between maximum frames (namely, each frame of frame numbers 12-15), the constraint conditions regarding the type of template image are:
・ "Template number 3 or 4"
It becomes.

In addition, the constraint condition regarding the range to perform the matching process is
· "Matching position _{(X 11 _ (3),} Y 11 _ (3)) and the matching area based on the size (W3 × H3) of the template image",
“Matching region based on matching position (X ₁₆ _ (4), Y ₁₆ _ (4)) and template image size (W4 × H4)”,
And a predetermined range.

  Therefore, the maximum inter-frame calculation unit 604 performs matching processing on each frame of frame numbers 12 to 15 under these constraint conditions. Thereby, correct information with a small error can be calculated for each frame of frame numbers 12 to 15.

  When matching processing is performed for each of the frame numbers 12 to 15 regardless of the constraint condition, for example, for the frame of frame number 15, a template image with width = W5 is shown in FIG. 18B. Is used to calculate the maximum matching degree. That is, when matching processing is performed regardless of the constraint condition, correct information with a large error is calculated in the frame of frame number 15.

  In addition, when matching processing is performed regardless of the constraint conditions as described above, correct information with a large error is calculated because, in each frame between the maximal frames, the front vehicle as the target is at the subpixel level. This is because it has moved slightly. Another reason is that the size is slightly changed at the sub-pixel level.

  As a result, an incorrect matching process is performed in the matching process, and correct answer information with a large error is calculated. Hereinafter, the relationship between the minute movement or size change at the sub-pixel level and the matching result will be described with reference to FIG.

  FIG. 19 is a diagram for explaining the cause of an incorrect matching process in the matching process. FIG. 19A shows a state in which template images 1902 and 1903 are generated from the initial template image 1901. The 4 × 4 pixel template image 1902 and the 3 × 3 pixel template image 1903 are template images generated by reducing the 8 × 8 pixel initial template image 1901.

  Of these, the template image 1902 is a template image generated by reducing the size information 1911 of the initial template image 1901 while maintaining the color information. That is, the color information of the feature area 1911 of the initial template image 1901 is kept as the color information of the feature area 1921 of the template image 1902.

  On the other hand, the template image 1903 cannot be reduced while maintaining the color information of the initial template image 1901, and pixels in which the color information of the feature region 1911 and the color information other than the feature region 1911 are mixed (pixels 1941 to 1943). Is a template image. This is because the feature region 1931 exists across the subpixel level in the template image 1903.

  On the other hand, FIG. 19B is a diagram showing that matching processing has been performed using the template images 1902 and 1903 shown in FIG. A frame 1950 indicates that the matching degree is maximized when the matching process is performed using the template image 1902. On the other hand, when the target (the vehicle ahead) moves at the subpixel level in the horizontal direction as in the frame image 1951, pixels in which the target and color information other than the target are mixed appear.

  In this case, the matching degree of the template image 1903 may be larger than the matching degree of the template image 1902. That is, an incorrect matching process is performed.

  In contrast to the minute movement and size change at the sub-pixel level as described above, in the present embodiment, the constraint condition is calculated based on the matching result in the maximum frame without the minute movement or size change. Since the matching process is performed under the calculated constraint condition, it is possible to avoid performing an incorrect matching process.

  As is clear from the above description, the correct answer generation device 130 calculates a maximum frame when performing matching processing with each frame included in the image information using a plurality of template images having different sizes. Further, when performing the matching process on the calculated frame between the maximal frames, the correct answer generation device 130 generates a constraint condition based on the matching result for the maximal frame. Further, the correct answer generation device 130 performs matching processing on each frame between the maximal frames based on the generated constraint condition.

  Thereby, even when the target moves at the sub-pixel level or when the size changes at the sub-pixel level in each frame between the maximal frames, erroneous matching processing is avoided.

  That is, highly accurate position specification can be realized for a target that performs minute movement and size change.

[Other Embodiments]
In the first embodiment, three types of template images are used for each frame when performing the matching process. However, two or more types of template images may be used for each frame. Any number of types may be used.

In addition, in the disclosed technology, forms such as the following additional notes are possible.
(Appendix 1)
An image processing device that generates correct data for verification when specifying a specific image region from an image of a predetermined direction of a vehicle,
First acquisition means for acquiring an estimation result of the size of the template in pixel units and the position of the template in the frame;
A second acquisition unit that constrains the condition with the estimation result acquired by the first acquisition unit and acquires the estimation result as time-series data between frames; and
An image processing apparatus that identifies a specific image region based on an estimation result acquired by the second acquisition unit.
(Appendix 2)
The first acquisition means includes
Among a plurality of templates having different pixel-unit sizes, the matching degree when the matching process is performed on the frame using one template, and the matching process is performed on the frame using another template The supplementary note 1 is characterized in that, when the difference from the matching degree in the case of the above is equal to or greater than a predetermined threshold, the estimation result of the size of the one template and the position of the one template in the frame is acquired. The image processing apparatus described.
(Appendix 3)
The first acquisition means includes
When the matching degree when the matching process is performed on the frame using the one template is larger than the matching degree when the matching process is performed on another frame using the one template, The image processing apparatus according to appendix 2, wherein an estimation result of the size of the one template and the position of the one template in a frame is acquired.
(Appendix 4)
An image processing method for generating correct data for verification when specifying a specific image region from an image obtained by photographing a predetermined direction of a vehicle,
A primary evaluation for obtaining the estimation result of the size of the template in pixel units and the position of the template in the frame, and a secondary evaluation for obtaining the estimation result as time series data between frames by constraining the conditions with the primary evaluation result. The specific image area is specified in two stages of
An image processing method, wherein the computer executes the processing.
(Appendix 5)
A primary evaluation for obtaining the estimation result of the size of the template in pixel units and the position of the template in the frame, and a secondary evaluation for obtaining the estimation result as time series data between frames by constraining the conditions with the primary evaluation result. Using the correct data for verification that specified the specific image area in the two stages, and evaluating the specific image area specified from the image obtained by photographing the predetermined direction of the vehicle as the verification target,
An evaluation method, wherein a computer executes processing.

  Note that the present invention is not limited to the configurations shown here, such as combinations with other elements, etc., in the configurations described in the above embodiments. These points can be changed without departing from the spirit of the present invention, and can be appropriately determined according to the application form.

100: Recognition performance verification system 110: Vehicle 120: Storage device 130: Correct answer generation device 131: Correct answer generation unit 504: Image tracking unit 601: Template generation unit 602: Matching calculation unit 603: Maximum frame calculation unit 604: Calculation between maximum frames Unit 611: Template information storage unit 612: Matching result information storage unit 613: Maximum frame information storage unit 900: Template information 1200: Matching result information

Claims (5)

An image processing device that generates correct data for verification when specifying a specific image region from an image of a predetermined direction of a vehicle,
First acquisition means for acquiring an estimation result of the size of the template in pixel units and the position of the template in the frame;
A second acquisition unit that constrains the condition with the estimation result acquired by the first acquisition unit and acquires the estimation result as time-series data between frames; and
An image processing apparatus that identifies a specific image region based on an estimation result acquired by the second acquisition unit. The first acquisition means includes
Among a plurality of templates having different pixel-unit sizes, the matching degree when a matching process is performed on a frame using one template and the matching process is performed on the frame using another template the difference between the degree of matching cases is state, and are equal to or higher than a predetermined threshold, and the matching degree in the case of performing a matching process with respect to the frame using the template of the 1, using the said other template the 2. The estimation result of the size of the one template and the position of the one template in the frame is obtained when the degree of matching is larger than the matching degree when the matching process is performed on the frame. An image processing apparatus according to 1. The first acquisition means includes
When the matching degree when the matching process is performed on the frame using the one template is larger than the matching degree when the matching process is performed on another frame using the one template, The image processing apparatus according to claim 2, wherein an estimation result of the size of the one template and the position of the one template in a frame is acquired. An image processing method for generating correct data for verification when specifying a specific image region from an image obtained by photographing a predetermined direction of a vehicle,
A primary evaluation for obtaining the estimation result of the size of the template in pixel units and the position of the template in the frame, and a secondary evaluation for obtaining the estimation result as time series data between frames by constraining the conditions with the primary evaluation result. The specific image area is specified in two stages of
An image processing method, wherein the computer executes the processing. A primary evaluation for obtaining the estimation result of the size of the template in pixel units and the position of the template in the frame, and a secondary evaluation for obtaining the estimation result as time series data between frames by constraining the conditions with the primary evaluation result. Using the correct data for verification that specified the specific image area in the two stages, and evaluating the specific image area specified from the image obtained by photographing the predetermined direction of the vehicle as the verification target,
An evaluation method, wherein a computer executes processing.