JP2012190380A - Vehicle detection device and vehicle detection method, and vehicle detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine whether or not a vehicle of a subject has an asymmetrical portion on the basis of an image of the vehicle captured from an oblique direction.SOLUTION: A vehicle detection device extracts from an image of a vehicle feature portions including portions indicating central positions of a front face of the vehicle and a plurality of line segments each indicating a linear portion extending in a horizontal direction in the front face of the vehicle; obtains centers of the line segments on the basis of relative positions of the portions indicating the center positions and the line segments adjacent to the portions indicating the center positions to calculate ratios of distances from the centers of the line segments to one ends of the line segments to distances from the centers of the line segments to the other ends of the line segments; obtains internal division points dividing the plurality of line segments on the basis of the ratios to generate a center line connecting the internal division points of the respective line segments with each other; and in the image of a portion indicating the vehicle included in the image of the vehicle, detects a region of the image having an asymmetrical feature with respect to the center line by dividing the front face of the vehicle into a left side portion and a right side portion using as a boundary the center line to compare the left side portion with the right side portion.

Description

  The present disclosure relates to a technique for extracting features of a vehicle from a portion representing the vehicle included in an image photographed by a fixed camera.

  In order to detect a specific vehicle based on an image obtained by photographing a vehicle traveling on a road with a camera installed on the road, a technique for recognizing a license plate portion included in the image has been proposed.

  In addition, a technique for analyzing an image obtained by photographing a vehicle and applying a process for reducing the amount of data of the image to be stored on the condition that the image of the vehicle body part after the tilt correction is symmetrical Has been proposed (see, for example, Patent Document 2).

International Publication WO2006 / 082644

  By the way, when a camera is installed on a facility such as a road or a vehicle inspection site, the installation location of the camera may be limited to a position where the vehicle is photographed from an oblique direction. When simple inclination correction is applied to an image obtained by photographing a vehicle from an oblique direction, the outer shape of the vehicle may be greatly distorted. It is difficult to determine whether or not there is an asymmetric part in the vehicle that is the subject based on the image that is distorted in this way.

  It is an object of the present disclosure to provide a vehicle detection device, a vehicle detection method, and a vehicle detection program that can determine whether or not there is an asymmetric part in a subject vehicle based on an image obtained by photographing the vehicle from an oblique direction. To do.

  A vehicle detection device, a vehicle detection method, and a vehicle detection program according to the present disclosure include a plurality of portions indicating a center position of a front surface of the vehicle and straight portions extending in a horizontal direction on the front surface of the vehicle from an image obtained by photographing the vehicle. A feature portion including a line segment is extracted, a center of the line segment is obtained based on a relative position of the line segment adjacent to the portion indicating the central position and the portion indicating the central position, and the center of the line segment An internal dividing point for calculating a ratio between a distance from the line segment to one end of the line segment and a distance from the center of the line segment to the other end of the line segment, and dividing the plurality of line segments based on the ratio And generating a center line connecting the internal dividing points of each line segment, and in an image of a part representing the vehicle included in an image obtained by photographing the vehicle, a left side portion in front of the vehicle with the center line as a boundary By comparing with the right part , Detects an area of an image having an asymmetrical wherein the center line as a reference.

  According to the vehicle detection device, the vehicle detection method, and the vehicle detection program of the present disclosure, it is possible to determine whether or not the subject vehicle has an asymmetric part based on an image obtained by photographing the vehicle from an oblique direction.

It is a figure which shows one Embodiment of a vehicle detection apparatus. It is a figure explaining the characteristic part extracted from an image. It is a figure explaining the relative position of a characteristic part. It is a figure explaining the process example of a detection part. It is a functional block diagram of a computer. It is a flowchart showing the process which detects an asymmetric vehicle. It is a figure which shows another embodiment of a vehicle detection apparatus. It is a flowchart showing an example of the process which extracts a characteristic part. It is a flowchart showing an example of a process of a calculation part. It is a flowchart showing an example of the process which detects an asymmetric area | region. It is a figure which shows the example of a vehicle recognition database. It is a figure which shows the example of a display of the result of having detected the vehicle which has an asymmetric characteristic. It is a figure which shows another embodiment of a vehicle detection apparatus. It is a flowchart showing another example of processing which detects an asymmetric feature.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an embodiment of a vehicle detection device. FIG. 1 shows a schematic relationship between the vehicle detection device 10 of the present disclosure and the vehicle 1, the camera 2, and the image input unit 3.

  The camera 2 shown in FIG. 1 images the vehicle 1 from, for example, an oblique front of the vehicle 1. The camera 2 may be installed at an angle at which the vehicle 1 traveling on the road can be caught in the field of view, such as on a street lamp post.

  The image input unit 3 acquires an image included in video captured by the camera 2 and inputs the acquired image to the vehicle detection device 10. The image input unit 3 may pass the image of each frame included in the video from the camera 2 to the vehicle detection device 10 or may pass the image every few frames.

  The vehicle detection device 10 illustrated in FIG. 1 includes an extraction unit 11, a calculation unit 12, a generation unit 13, and a detection unit 14.

  The extraction unit 11 indicates a line segment representing a horizontal linear component located in front of the vehicle 1 and a center position in front of the vehicle 1 from a vehicle image that is a part representing the vehicle 1 included in the input image. A feature portion including a portion is extracted.

  FIG. 2 is a diagram for explaining the characteristic part extracted from the image. Reference numeral 1g shown in FIG. 2 is a vehicle image that is a portion representing the vehicle 1 included in the image. Reference signs E1 to E6 and reference signs M1 and M2 shown in FIG. 2 are examples of characteristic parts extracted from the vehicle image 1g.

  Broken lines indicated by reference numerals E1 to E6 are examples of line segments corresponding to horizontal straight line components located in front of the vehicle 1. In the example of FIG. 2, the symbol E <b> 1 corresponds to the boundary line between the upper side of the windshield of the vehicle 1 and the roof. Moreover, the code | symbol E2 respond | corresponds to the boundary line of the lower side of the windshield of the vehicle 1, and a bonnet. The symbol E3 corresponds to the boundary line between the lower side of the front grill of the vehicle 1 and the bumper. Reference numeral E4 corresponds to the upper side of the contour of the license plate of the vehicle 1. Reference numeral E5 corresponds to a groove-like structure formed in the bumper of the vehicle 1. The symbol E6 corresponds to the lower side of the bumper contour of the vehicle 1.

  Reference numerals M <b> 1 and M <b> 2 illustrated in FIG. 2 are extraction examples of portions indicating the center position of the front surface of the vehicle 1. In the example of FIG. 2, the code | symbol M1 shows a license plate. Reference numeral M <b> 2 indicates an example of an emblem indicating a manufacturer or a brand on the vehicle 1. In addition to the number plate and emblem described above, the extraction unit 11 may extract an image representing a seal of a vehicle inspection certificate affixed to the inside of the windshield as a portion indicating the center position of the vehicle 1. . In addition, the extraction unit 11 may extract an image representing an indoor rearview mirror or the like as a portion indicating the center position of the front of the vehicle 1.

  The calculation unit 12 illustrated in FIG. 1 has a line segment indicating the center of a corresponding straight line portion in the line segment based on the relative position of the line segment adjacent to the portion indicating the center position and the portion indicating the center position. Find the center. Further, the calculation unit 12 calculates a ratio between the distance from the center of the line segment to one end of the line segment and the distance from the center of the line segment to the other end of the line segment.

  FIG. 3 is a diagram for explaining the relative positions of the characteristic portions. FIG. 3 is an example of the relative positions of the line segments E3 and E5 shown in FIG. 2 and the license plate in the image indicated by reference numeral M1.

  In the example shown in FIG. 3, in the license plate in the image indicated by reference numeral M1, the position corresponding to the center position in the horizontal direction of the actual license plate of the vehicle 1 is the position of the point C5 on the line segment E5. . In the example of FIG. 3, the position of this point C5 is indicated by a triangle indicated by reference numeral C5. This point C5 is the center of the line segment E5 corresponding to the center of the straight line portion of the vehicle 1 corresponding to the line segment E5.

  The calculation unit 12 calculates a ratio a: b between the distance a from the point C5 to one end point of the line segment E5 and the distance b from the point C5 to the other end of the line segment E5. The ratio a: b reflects an inclination angle between the shooting direction of the camera 2 shown in FIG. 1 and the front direction of the vehicle 1. Therefore, the point C3 that internally divides the line segment E3 extracted from the vehicle image 1g together with the above-described line segment E5 with the above-described ratio a: b corresponds to the center of the straight line portion of the corresponding vehicle 1 in the line segment E3. And is the center of this line segment E3. In the example of FIG. 3, the triangle indicated by the symbol C3 indicates the point at which the line segment E3 is internally divided by the ratio a: b, that is, the center position of the line segment E3. Similarly, an internal dividing point that internally divides the line segments E1, E2, E4, and E6 shown in FIG. 2 at a ratio a: b corresponds to each line segment corresponding to the center of the straight line portion of the vehicle 1 corresponding to each line segment. Is the center of The triangles denoted by reference numerals C1 to C6 in FIG. 2 indicate the centers of the line segments E1 to E6.

  The generation unit 13 illustrated in FIG. 1 obtains an internal dividing point that internally divides each line segment extracted by the extraction unit 11 based on the ratio a: b calculated by the calculation unit 12. Moreover, the production | generation part 13 produces | generates a centerline by connecting the internal dividing points calculated | required about each line segment mutually.

  As described above, the internal dividing point that internally divides each line segment extracted by the extraction unit 11 with the above-described ratio a: b is the center of each line segment corresponding to the center of the straight line portion of the vehicle 1 corresponding to each line segment. It is. Therefore, the center line in the vehicle image 1 g generated by connecting these corresponds to the center line of the front of the actual vehicle 1.

  FIG. 2 shows an example of the center line CL obtained by sequentially connecting the internal dividing points C1 to C6 obtained for the line segments E1 to E6.

  The detection unit 14 illustrated in FIG. 1 includes, in the vehicle image 1g, a portion corresponding to the left side and a portion corresponding to the right side when the vehicle 1 is viewed from the front, with the center line generated as described above as a boundary. Are compared with each other. And the detection part 14 detects the part which has an asymmetric characteristic on the basis of a centerline from the vehicle image 1g based on a comparison result.

  The detection unit 14 compares the outline of the left part and the right part of a part that is symmetrical in a normal state, such as a windshield and a bonnet, in the following manner, to compare the asymmetric part May be detected.

  FIG. 4 is a diagram illustrating a processing example of the detection unit. FIG. 4 is an example in which the edge indicating the contour of the windshield included in the vehicle image 1g is compared between the left part and the right part across the center line.

  For example, the detection unit 14 may check the symmetry of the windshield portion using the straight line Q parallel to the line segment E2 corresponding to the lower side of the windshield contour.

  The symbol Q shown in FIG. 4 is a straight line parallel to the line segment E2 corresponding to the lower side of the contour of the windshield. Reference numerals QL and QR are intersections of the left and right edges of the edge included in the vehicle image 1g corresponding to the contour of the windshield and the straight line Q described above. Reference QC is the intersection of the center line CL and the straight line Q.

  The detection unit 14 calculates distances XL and XR between the intersection points QL and QR shown in FIG. 4 and the intersection point QC, respectively. Next, the detection unit 14 calculates a distance ratio XL: XR, and compares the distance ratio XL: XR with the ratio a: b calculated by the calculation unit 12. Based on the comparison result, the detection unit 14 detects a portion where the distance ratio XL: XR deviates from the ratio a: b by a predetermined threshold or more as an asymmetric portion with respect to the center line. Similarly, the detection unit 14 can detect an asymmetric part that appears at the edge of the vehicle image 1g due to the dent of the hood of the vehicle 1 or the like.

  In addition, the detection unit 14 compares the average luminance value and the color for each predetermined region centering on a pixel whose ratio a: b is the ratio of the distance from the center line obtained by the calculation unit 12. An asymmetric region may be detected.

  The area detected in this way corresponds to a portion of the vehicle 1 that is the subject that exhibits asymmetric characteristics between the left side and the right side when viewed from the front. That is, the processing of the detection unit 14 can detect an asymmetric feature of the vehicle 1 based on the vehicle image 1g acquired by the camera 2 that captures the vehicle 1 from an oblique direction.

  As described above, according to the vehicle detection device of the present disclosure, it is possible to determine whether or not the vehicle 1 has an asymmetric portion based on an image captured from an oblique direction. That is, according to the vehicle detection device of the present disclosure, it is possible to detect a vehicle having an asymmetric feature from an image obtained by a camera that captures a vehicle traveling on a road from a direction other than the front. Thereby, the camera for acquiring the image used for vehicle recognition can be installed with a high degree of freedom. Such a feature is useful, for example, in a system that detects a vehicle having a specific feature from vehicles traveling on a road. In addition, the vehicle detection device disclosed herein can be applied to simple inspections at vehicle inspection sites and the like.

  In addition to the method described above, the processing of the detection unit 14 can be realized by various methods as described later. Further, the vehicle detection device disclosed herein can be realized using a computer device.

  FIG. 5 shows a functional block diagram of the computer apparatus. The computer apparatus shown in FIG. 5 includes a processor 21, a memory 22, a hard disk device (HDD: Hard Disk Drive) 23, a display control unit 24, a display device 25, and an input device 26. Further, the computer device includes an optical drive device 28 and a communication control unit 29.

  The processor 21, the memory 22, the HDD 23, the display control unit 24, the input device 26, the optical drive device 28, and the communication control unit 29 are connected via a bus. The communication control unit 29 is connected to the network 30. Further, the computer device may receive a video signal from the camera 2 shown in FIG. 1 via, for example, the image input interface unit 27. Further, the computer device may receive a video signal from the camera 2 via the communication control unit 29 and the network.

  The HDD 23 stores an application program for executing processing for detecting a vehicle having an asymmetric feature based on the operating system and the vehicle image. The application program includes a program for executing each process included in the vehicle detection method disclosed herein. The application program for executing the vehicle detection process can be recorded and distributed on the computer-readable removable disk 31, for example. The application program for executing the vehicle detection process is installed in the HDD 23 by mounting the removable disk 31 in the optical drive device 28 and performing the reading process. In addition, an application program for executing vehicle detection processing can be installed in the HDD 23 via the network 30 such as the Internet and the communication control unit 29.

  The computer apparatus illustrated in FIG. 5 realizes the above-described various functions by organically cooperating hardware such as the processor 21 and the memory 22 and programs such as an operating system and application programs.

  Moreover, you may implement | achieve the extraction part 11, the calculation part 12, the production | generation part 13, and the detection part 14 which were shown in FIG. 1 respectively, when the processor 21 runs the corresponding program. Similarly, the image input unit 3 shown in FIG. 1 may be realized by a combination of the processor 21, the image input interface 27, and a program for acquiring an image from a video signal, or the processor 21 and the communication control unit 29. You may implement | achieve by the combination. Further, information generated in the process of each unit shown in FIG. 1 is stored in the memory 22 or the HDD 23.

  FIG. 6 is a flowchart showing processing for detecting an asymmetric vehicle. The processing of each step shown in FIG. 6 is executed by the processor 21 in accordance with an application program for executing vehicle detection processing.

  Prior to the vehicle detection process, the processor 21 acquires an image from the video imaged by the camera 2 by cooperating with, for example, the image input interface 27 and a program for the image acquisition process (step 301). The processor 21 uses the image acquired from the video for processing for detecting an asymmetric vehicle described below. The processor 21 acquires an image corresponding to each frame included in the video imaged by the camera 2 or an image corresponding to a frame extracted every predetermined number of frames.

  The processor 21 cooperates with a program for realizing the extraction unit 11 included in the application program described above, thereby performing, for example, processing for extracting a feature part from an image input via the image input interface 27 ( Step 302). For example, the processor 21 extracts a line segment corresponding to a horizontal straight line component in front of the vehicle 1 by extracting a line segment having a length equal to or greater than a predetermined threshold from an edge image corresponding to the input image. To do. Further, the processor 21 extracts, for example, a characteristic part corresponding to a member arranged at the center of the vehicle 1 such as a license plate or an emblem from the input image using a pattern matching technique.

  Next, the processor 21 cooperates with the program for realizing the calculation unit 12 included in the above-described application program, whereby the distance from the center of the line segment corresponding to the center in the horizontal direction of the front of the vehicle 1 to both ends. A process for calculating the ratio is performed (step 303). For example, as described with reference to FIGS. 2 and 3, the processor 21 specifies the center of the line segment that is close to the portion indicating the center based on the position of the portion indicating the center of the vehicle 1. The ratio at which the center of the line segment internally divides this line segment is calculated.

  Next, the processor 21 performs a process of generating a center line by cooperating with a program for realizing the generating unit 13 included in the application program described above (step 304). For example, the processor 21 generates a center line by sequentially connecting internal dividing points that internally divide each line segment based on the ratio calculated in step 303.

  The processor 21 cooperates with the program for realizing the detection unit 14 included in the application program described above, and corresponds to the image corresponding to the left portion of the vehicle 1 and the right portion across the center line. Compare the image. Based on the comparison result, the processor 21 detects an image region having an asymmetric feature with respect to the center line (step 305). The region detected by the process of step 304 corresponds to a portion that shows asymmetric characteristics between the left side and the right side of the center line in the vehicle 1 that is the subject.

  Therefore, the processor 21 can determine whether or not the vehicle 1 has an asymmetric feature based on whether or not an image region having an asymmetric feature has been detected in the process of step 305. Then, based on the determination result, the processor 21 stores information indicating the detected asymmetric area or information indicating that there is no asymmetric feature, for example, in the memory 22 or the HDD 23 corresponding to the information for identifying the vehicle 1. It may be stored.

  Note that the processor 21 may cause the display device 25 to display the detection result obtained by the above-described processing via the display control unit 24. Further, the processor 21 may notify the detection result obtained by the above-described process to another computer device via the communication control unit 29 and the network 30.

  Hereinafter, another embodiment of the vehicle detection device disclosed herein will be described.

  FIG. 7 shows another embodiment of the vehicle detection device. 7 that are the same as those shown in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted.

  The image input 3 shown in FIG. 7 inputs the image extracted from the image | video image | photographed with the camera 2 shown in FIG. 1 to the vehicle detection part 10 and the existing license plate recognition apparatus 4. FIG.

  The license plate recognition device 4 includes a license plate extraction unit 4-a and a number recognition processing unit 4-b. The license plate extraction unit 4-a performs image processing such as pattern matching processing on the image received from the image input unit 3, thereby extracting a license plate portion included in the image. The number recognition processing unit 4-b recognizes the registered number of the vehicle 1 photographed by the camera 2 by analyzing the extracted image of the license plate portion. The license plate extraction unit 4-a may output information indicating the position of the license plate portion extracted from the image. For example, the license plate extraction unit 4-a may output a pixel address indicating the boundary of the range of the license plate portion included in the image as information indicating the position of the license plate portion in the image.

  The extraction unit 11 illustrated in FIG. 7 includes an image processing unit 212, a line segment extraction unit 213, and an emblem extraction unit 214. The image processing unit 212 extracts a vehicle image representing the vehicle 1 from the image input by the image input unit 3. The emblem extraction unit 214 extracts a portion representing the emblem from the extracted vehicle image. The emblem extraction unit 214 outputs information indicating the position of the region of the image extracted as a part representing the emblem as one of information indicating the position of the characteristic part indicating the center of the vehicle 1. Further, the image processing unit 212 extracts edge components from the extracted vehicle image to generate an edge image. The line segment extraction unit 213 extracts a line segment corresponding to a straight line portion extending in the horizontal direction in front of the vehicle 1 from the generated edge image. For example, the line segment extraction unit 213 extracts a line segment corresponding to a linear component extending in the horizontal direction from a straight line portion having a length equal to or greater than a predetermined value included in the edge image according to a predetermined condition described later. For example, the line segment extraction unit 213 may output information indicating the positions of both ends of each extracted straight line portion as an extraction processing result.

  Note that the emblem extraction unit 214 may perform processing of extracting an emblem portion included in an image extracted as a portion representing the vehicle 1 using information stored in the vehicle feature database 5.

  The vehicle feature database 5 illustrated in FIG. 7 stores, for example, feature information indicating the positions and shapes of main components included in a vehicle for a typical vehicle type. The feature information stored in the vehicle feature database 4 includes, for example, information indicating the approximate shape and average size of the windshield and bonnet for each vehicle type such as a sedan type, a wagon type, or a light vehicle. The feature information stored in the vehicle feature database 4 may further include information indicating the features of the emblems and mark shapes of representative automobile manufacturers.

  Further, as will be described later, the calculation unit 12 illustrated in FIG. 7 specifies the center of the line segment that is close to the feature part based on the position of the feature part that indicates the center of the vehicle 1, and specifies the specified line segment. The ratio of the distance from the center of the line to both ends of the line segment is calculated. The calculation unit 12 passes the calculated distance ratio to the generation unit 13.

  The generation unit 13 illustrated in FIG. 7 includes a coordinate calculation unit 216 and an equation generation unit 217. The coordinate calculation unit 216 receives the ratio calculated by the calculation unit 12, and calculates the coordinates of the points that internally divide each line segment extracted by the line segment extraction unit 213 according to this ratio. The equation generation unit 217 generates a plurality of linear equations that sequentially connect these coordinates based on the coordinates calculated by the coordinate calculation unit 216. The broken line represented by these equations corresponds to the center line of the front of the actual vehicle 1 in the vehicle image 1g. Information indicating the center line generated in this way is passed to the detection unit 14.

  The detection unit 14 illustrated in FIG. 7 includes a partial image extraction unit 218, a deformation unit 219, a collation unit 220, and a symmetry determination unit 221. The partial image extraction unit 218 extracts, from the vehicle image, for example, a portion of an image that represents each of the components having a symmetrical shape included in the vehicle 1. The partial image extraction unit 218 may use the feature information stored in the vehicle feature database 5 for the process of extracting the portion of the image representing each part. The image of each component extracted by the partial image extraction unit 218 can be divided into a left image corresponding to the left portion and a right image corresponding to the right portion as viewed from the front of the vehicle 1 with the above-described center line as a boundary. it can. The deforming unit 219 respectively deforms the left image and the right image included in the image corresponding to each extracted component based on the ratio calculated by the calculating unit 12. The collation unit 220 collates the deformed left image and right image with the center line as a reference. Based on the collation result, the symmetry determination unit 221 determines whether or not each of the above-described parts has an asymmetric feature between the left shape and the right shape with respect to the center line. The detection unit 14 outputs a detection result based on the determination result obtained by the symmetry determination unit 221 for each component.

  The result storage unit 6 illustrated in FIG. 7 stores the detection result obtained by the detection unit 14 in the vehicle recognition database 8. The result notification unit 7 performs a process of collectively reporting information indicating detection results obtained for each vehicle 1 by the detection unit 14 or information indicating detection results stored for a plurality of vehicles in the vehicle recognition database 8. .

  The control unit 9 illustrated in FIG. 7 performs processing for controlling the exchange of information between the units illustrated in FIG. For example, the vehicle image extracted by the image processing unit 212 is passed to the partial image extraction unit 218 via the control unit 9. The recognition result by the license plate recognition device 4 is also passed to the symmetry determination unit 221 via the control unit 9. In addition, the result notification unit 7 notifies the operator and other systems of information indicating the detection result described above, for example, via the control unit 9.

  The processing of each unit illustrated in FIG. 7 is realized by organic cooperation between hardware such as the processor 21 and the memory 22 included in the computer apparatus illustrated in FIG. 5 and programs such as an operating system and application programs. May be.

  Next, processing of the extraction unit 11, the calculation unit 13, the generation unit 13, and the detection unit 14 will be described.

  FIG. 8 is a flowchart showing the process of extracting the characteristic part. Each step shown in FIG. 8 is an example of the process of step 302 shown in FIG. Each step shown in FIG. 8 is executed by each unit included in the control unit 9 and the extraction unit 11, respectively.

  The image processing unit 212 that has received the image from the image input unit 3 performs a process of extracting a portion representing the vehicle 1 included in the image as a vehicle image (step 302-1). For the process of extracting the vehicle image, for example, a known technique such as a technique using a color difference from the background can be used.

  When the vehicle image cannot be extracted by the image processing unit 212 (No determination in step 302-2), the process is returned to step 301 in FIG. 6 via the control unit 211. When a new image is input from the image input unit 3, the image processing unit 212 again performs a process of extracting a vehicle image from the new image.

  On the other hand, when the vehicle image is extracted by the image processing unit 212 (Yes in Step 302-2), the emblem extraction unit 214 performs a process of extracting a portion representing the emblem from the vehicle image (Step 302-3). ). At this time, the license plate extraction unit 4-a performs processing for extracting the license plate portion from the same vehicle image. And the control part 9 receives the information which shows the position of a license plate from the license plate extraction part 4-a (step 302-4).

  When the information indicating the position of the license plate is received (affirmative determination in step 302-5), the control unit 9 determines that the detection of the license plate is successful. And the control part 9 records the information which shows the detection result of a license plate, and the information which shows the position of the said license plate as a part of information which shows the recognition result corresponding to the vehicle 1 (step 302-6). For example, the control unit 9 may record the address of the pixel corresponding to each vertex of a quadrangle representing the license plate as information indicating the position of the license plate. On the other hand, when the information indicating the position of the license plate is not obtained from the license plate extraction unit 4-a (No in Step 302-5), the control unit 9 determines that the extraction of the license plate has failed. In this case, the control unit 9 records information indicating that the detection of the license plate has failed as part of the information indicating the recognition result corresponding to the vehicle 1 (step 302-7).

  Next, the image processing unit 212 generates an edge image corresponding to the vehicle image extracted in Step 302-1 (Step 302-8). The line segment extraction unit 213 extracts, from the generated edge image, a plurality of line segments indicating a straight line portion in the horizontal direction in the outer shape of the front surface of the vehicle 1 (step 302-9). For example, the line segment extraction unit 213 selectively extracts a plurality of line segments indicating a straight line portion extending in the horizontal direction in front of the vehicle 1 on the condition that the inclination range of the straight line portion is within a predetermined range. May be. For example, the line segment extraction unit 213 limits the range of inclination based on the angle formed by the optical axis direction of the camera 2 and the direction of the road lane shown in FIG. You may exclude the line segment. Thereby, the line segment extraction part 213 selectively extracts the line segment which shows the straight line part extended in the horizontal direction of the front of the vehicle 1 from a vehicle image. Note that the line segment extraction unit 213 may output the addresses of the pixels corresponding to both ends of each extracted line segment as information indicating the position of each line segment.

  Note that the emblem extraction process in step 302-2 shown in FIG. 8, the process related to the license plate extraction in steps 302-4 to 302-7, and the line segment extraction in steps 302-8 to 302-9 are shown. The execution order related to may be different from that shown in FIG.

  Each part included in the extraction unit 11 and the control unit 9 execute the above-described processing, so that a part of a member indicating the center of the vehicle 1 and a linear part extending in the horizontal direction in front of the vehicle 1 from the image of the vehicle 1 Can be extracted as a feature part.

  Next, the process of the calculation part 12 based on the relative position of the characteristic part extracted as mentioned above is demonstrated.

  FIG. 9 is a flowchart illustrating an example of processing performed by the calculation unit. Each step shown in FIG. 9 is an example of the process of step 303 shown in FIG. Each step shown in FIG. 9 is executed by the control unit 9 and the calculation unit 12, respectively.

  In step 303-1, the control unit 9 determines whether or not the license plate extraction unit 4-a has notified that the license plate has been successfully extracted. For example, when the information indicating that the license plate has been successfully detected in step 302-6 described above is recorded as part of the recognition result corresponding to the vehicle 1, the control unit 9 determines that the license plate has been successfully extracted. It is determined that notification has been given (affirmative determination in step 303-1). In this case, the control unit 9 performs a process of examining the relative position between any of the line segments extracted by the line segment extraction unit 213 and the license plate (step 303-2). For example, the control unit 9 includes information indicating the position of the license plate recorded in step 302-6 described above and information indicating the position of the line segment for one of the line segments extracted in step 302-9. Based on this, the relative position between the license plate and the line segment may be examined. For example, the control unit 9 specifies the address of the pixel corresponding to the midpoint of the bottom of the quadrangle corresponding to the license plate in the vehicle image based on the information indicating the position of the license plate. Based on the magnitude of the difference between this address and the address of the pixel corresponding to the midpoint of one of the line segments, the position is located between the license plate and the horizontal straight line portion of the vehicle 1 corresponding to the line segment. It is determined whether or not there is a deviation. In addition, it is desirable that the line segment used when determining the positional deviation is a line segment extracted from a position close to a quadrangle corresponding to the license plate in the image of the vehicle 1.

  When the above-described address difference is equal to or smaller than a predetermined threshold, the control unit 9 determines that the license plate is installed in the center of the vehicle 1 (affirmative determination in step 303-3). In this case, the control unit 9 instructs the calculation unit 12 to use the license plate as a characteristic part indicating the center of the vehicle 1. At this time, the control unit 9 may pass the address of the pixel corresponding to the midpoint of the base of the quadrangle corresponding to the license plate to the calculation unit 12 in the vehicle image.

  Based on the address of the pixel received from the control unit 9 and information indicating one position of the line segment extracted by the line segment extraction unit 213, the calculation unit 12 corresponds to the line segment corresponding to the center position of the license plate. The ratio of the distance between the upper point and both ends of the line segment is calculated (step 303-4). For example, the calculation unit 12 specifies the center position of the line segment extracted from the position close to the license plate portion in the vehicle image based on the pixel address received from the control unit 9. The center of this line segment corresponds to the center of the straight line portion in front of the vehicle 1 corresponding to the line segment. Next, the calculation unit 12 obtains the distance from the identified position to both ends from the difference between the address of the pixel indicating the identified position and the address of each pixel corresponding to both ends of the line segment, and calculates the calculated distance. Calculate the ratio.

  On the other hand, when the position of the midpoint of the base of the quadrangle corresponding to the license plate and the position of the midpoint of the line segment of interest are separated from the distance indicated by the predetermined threshold, the control unit 9 Is determined to be at a position deviating from the center (No at Step 303-3). In this case, the control unit 9 instructs the calculation unit 12 to use the emblem as a characteristic part indicating the center of the vehicle 1. In response to this, the calculation unit 12 sets the center position of the emblem based on the information indicating the position of the emblem portion in the vehicle image and the information indicating one position of the line segment extracted by the line segment extraction unit 213. The ratio of the distance between the corresponding point on the line segment and both ends of the line segment is calculated (step 303-5). In this case, the calculation unit 12 may obtain an address of a pixel indicating the position of the center of the emblem in the vehicle image from the information indicating the position of the emblem portion in the vehicle image obtained by the emblem extraction unit 214. For example, the calculation unit 12 specifies the center position of the line segment extracted from the position close to the emblem portion in the image of the vehicle 1 based on the obtained pixel address. Next, the calculation unit 12 obtains the distance from the identified position to both ends from the difference between the address of the pixel indicating the identified position and the address of each pixel corresponding to both ends of the line segment, and calculates the calculated distance. Calculate the ratio.

  Note that in the extraction unit 11, instead of the emblem, a part representing a seal of the vehicle inspection certificate or the like may be extracted from the vehicle image as a characteristic part indicating the center position of the vehicle 1. In this case, the calculation unit 12 may calculate the above-described ratio based on information indicating the position of the seal on the vehicle inspection certificate. Further, when the license plate extraction is successful, the calculation unit 12 may perform the process of step 303-5 in addition to the process of step 303-4 described above. And the calculation part 12 may obtain | require more probable ratio based on the ratio each obtained by the process of step 303-4, 303-5.

  The ratio obtained in step 303-4 or step 303-5 is passed to the coordinate calculation unit 216 of the generation unit 13.

  For each line segment, the coordinate calculation unit 216 calculates the coordinates of a point that internally divides the line segment according to the above-described ratio. The coordinate calculation unit 216 may obtain an address indicating the coordinates of the pixel corresponding to the internal dividing point based on the address of the pixel corresponding to both ends of each line segment and the ratio calculated by the calculation unit 12. The coordinates of the internal dividing points thus obtained indicate the position of the upper point of the line segment corresponding to the center of each line segment, that is, the center of the straight line portion of the vehicle 1 corresponding to the line segment.

  Next, the equation generation unit 217 generates an equation that connects the internal dividing points corresponding to each line segment based on the coordinates calculated by the coordinate calculation unit 216. For example, when the address of each pixel in the image is represented by a set of a position in the row direction and a position in the column direction, the equation generation unit 217 has a value indicating the position in the row direction included in the address of the inner dividing point. Generate an equation that connects the interior dividing points so that they are connected in ascending or descending order. Further, the equation generation unit 217 may generate an equation for connecting the internal dividing points so that values indicating the position in the column direction included in the addresses of the internal dividing points are connected in ascending order or descending order.

  Thus, in each line segment extracted by the extraction unit 11, an equation indicating a center line passing through a position indicating the center of the straight line portion of the vehicle 1 corresponding to the line segment can be generated.

  Next, a process in which the detection unit 14 detects an asymmetric area from the image of the vehicle 1 based on the center line generated as described above will be described.

  FIG. 10 is a flowchart illustrating an example of processing for detecting an asymmetric region in the image of the vehicle 1. Each step shown in FIG. 10 is an example of the process of step 305 shown in FIG. Each step shown in FIG. 10 is executed by each unit and the control unit 9 included in the detection unit 14 shown in FIG.

  FIG. 11 shows an example of the vehicle recognition database 8. The vehicle recognition database 8 illustrated in FIG. 11 corresponds to a recognition ID (IDentifier) for identifying each recognition target vehicle, a recognition date and time, a recognition result for each component included in the recognition target vehicle, and will be described later. A determination result based on a predetermined condition is held.

  First, the control unit 9 selects a part of the vehicle 1 to be subjected to symmetry verification (step 305-1). The control unit 9 sequentially selects, for example, parts having a symmetrical basic shape such as a bumper and a bonnet from among a large number of parts included in the vehicle 1.

  Next, the partial image extraction unit 218 shown in FIG. 7 extracts a part representing the part selected by the control unit 9 from the vehicle image extracted by the image processing unit 212 (step 305-2). For example, the partial image extraction unit 218 may use the feature information stored in the vehicle feature database 5 for the process of extracting a part representing the selected part from the vehicle image. Further, the partial image extraction unit 218 may extract a part representing the selected part from the vehicle image by using features such as color continuity in the image, for example.

  First, the deforming unit 219 shown in FIG. 7 divides the image of the part representing the part extracted by the partial image extracting unit 218 into two regions using the center line (step 305-3). For example, the deformation unit 219 includes an image of a part representing a selected part on the left side when the vehicle 1 is viewed from the front using an equation of a center line included in the part extracted by the partial image extraction unit 218. Dividing into a left area and a right area included on the right side.

  Next, the deforming unit 219 deforms the left region image and the right region image in accordance with the distance ratio obtained by the calculating unit 12 described above (step 305-4). For example, the deformation unit 219 may perform a deformation process of enlarging one of the divided two regions in a direction perpendicular to the center line using a magnification c corresponding to the ratio of the distances obtained by the calculation unit 12. Good. Based on the ratio a: b obtained by the calculation unit 12, the deformation unit 219 enlarges the image in the left region by a / b times in a direction perpendicular to the center line. By this deformation processing, distortion caused by photographing the vehicle 1 from an oblique direction is corrected.

  Next, the collation unit 220 shown in FIG. 7 collates the features of the left image and the features of the right image after the deformation processing by the deformation unit 219 described above (step 305-5). For example, the collation unit 220 may collate the luminance distribution indicating the color and brightness characteristics of the component under verification for the left image and the right image after the deformation process. In this case, the collation unit 220, for example, information indicating a range where the absolute difference value of the luminance at the corresponding position in the left image and the right image exceeds the threshold, or information indicating the number of pixels included in this range. Or the like may be output as a collation result for the luminance distribution. Further, the collation unit 220 determines the distance between each point on the left outline of the component being verified and the center line based on the left image and the right image after the deformation, and each of the right outline of the component being verified. You may compare the distance of a point and a centerline. Thereby, the collation part 220 can collate about the characteristic of the shape of the right and left of components in verification. In this case, the collation unit 220 represents, for example, information indicating the sum of squared differences or a range in which the difference absolute value exceeds the threshold based on the difference in distance between each point on the left and right contours and the center line. Information or the like may be generated and output as a matching result for the contour.

  As described above, the left image and the right image are deformed for each component, and collation is performed, so that it is high whether there is an asymmetric portion in the shape of the symmetrical component that extends left and right from the symmetry plane in the vehicle 1. It can be detected with accuracy. The collation unit 220 can also detect a feature such as an asymmetric paint on the bonnet by collating luminance distribution and color distribution in an image.

  The symmetry determination unit 221 illustrated in FIG. 7 compares the collation result obtained by the collation unit 220 with a predetermined threshold (step 305-6). For example, the symmetry determination unit 221 may prepare a threshold value to be compared with the matching result for each type of component being verified and for each feature to be checked by the matching unit 220.

  When all the collation results received from the collation unit 220 for the component under verification are equal to or less than the corresponding threshold value (negative determination in step 305-6), the symmetry determination unit 221 determines that the component is normal in the vehicle 1. Is output as a detection result (step 305-7). At this time, the result storage unit 6 corresponds to the recognition ID indicating the vehicle 1 information indicating that the part is normal based on the detection result output in step 305-7 in the vehicle authentication database 8. save. For example, when all the matching results received from the matching unit 220 for the bumper are equal to or less than the corresponding threshold value, the symmetry determining unit 221 outputs a detection result indicating that the bumper is normal. In response to this, the result storage unit 6 shown in FIG. 7 shows that the bumper is normal in the vehicle recognition database 8 as shown in the example of FIG. 11 corresponding to the recognition ID [0001] of the recognition target vehicle. Save something.

  On the other hand, when a difference greater than or equal to the threshold is found in any of the verification results for the part under verification received from the verification unit 220 (positive determination in step 305-6), the symmetry determination unit 221 first performs verification. It is determined whether or not the part is a bumper (step 305-8).

  When the part under verification is other than the bumper (negative determination at step 305-8), the symmetry determination unit 221 determines that the part under verification has an asymmetric part. Then, the symmetry determining unit 221 detects the part under verification as one of the asymmetric features of the vehicle 1 (step 305-9). At this time, the result storage unit 6 stores the asymmetric feature detected in step 305-9 in the vehicle authentication database 8 in correspondence with the recognition ID indicating the vehicle 1. For example, when the sum of squared differences between the points on the left and right contours obtained by the matching unit 220 and the center line is equal to or greater than the corresponding threshold value, the symmetry determining unit 221 has an asymmetric bonnet shape. Information indicating that it is present is output as a detection result. In response to this, the result storage unit 6 informs the vehicle recognition database 8 that the shape of the bonnet is asymmetric as shown in the example of FIG. 11 corresponding to the recognition ID [0002] of the recognition target vehicle. save.

  By the way, when the license plate of the vehicle 1 is installed at a position other than the center of the bumper, the image of the license plate is a right-and-left image in the verification process of the verification unit 220 for verifying the bumper. Distributions may be found as inconsistent parts. In order to cope with such a case, when the verification target part is a bumper (affirmative determination in step 305-8), the symmetry determination unit 221 performs the processing in steps 305-10 to 305-12.

  In step 305-10, the symmetry determination unit 221 detects, for example, a portion in which the luminance distribution or the like is mismatched by the collation unit 220 with respect to the license plate recognition device 4 illustrated in FIG. 7 via the control unit 9. Request license plate extraction processing as a target.

  When the license plate recognition device 4 notifies that the license plate has been successfully extracted (affirmative determination in step 305-11), the symmetry determination unit 221 determines the license plate mounting position based on the asymmetric feature of the vehicle 1. (Step 305-12). Accordingly, the result storage unit 6 has a normal license plate on the left side in the vehicle recognition database 8 as shown in the example of FIG. 11 corresponding to the recognition ID [0003] of the recognition target vehicle. Save the information.

  On the other hand, when it is notified from the license plate recognition device 4 that the extraction of the license plate has failed (No in Step 305-11), the symmetry determination unit 221 uses the bumper as one of the asymmetric features of the vehicle 1. (Step 305-9). In response to this, the result storage unit 6 stores in the vehicle recognition database 8 that there is an abnormality in the bumper, as shown in correspondence with the recognition ID [0002] of the recognition target vehicle in the example of FIG. .

  After the processing of Step 305-7, Step 305-9, or Step 305-12 described above, the control unit 9 determines whether or not the verification for all components has been completed (Step 305-13). When there is an unverified part (negative determination at step 305-13), the control unit 9 returns to step 305-1 and selects a new part as a part to be verified. Then, in accordance with Step 305-2 to Step 305-12, processing for verifying symmetry with respect to the new part is performed.

  In this way, the result of verifying the symmetry of each major part included in the vehicle 1 can be reflected in the vehicle authentication database 8. The vehicle authentication database 8 illustrated in FIG. 11 shows the verification result of the vehicle 1 indicated by each recognition ID for main parts including a license plate, a rearview mirror, a door mirror, a windshield, a bumper, and a bonnet.

  In the example of FIG. 11, as a detection result for the vehicle with the recognition ID [0001], information indicating that an asymmetric feature that is determined to be abnormal for the headlight is detected, and an asymmetric feature is not detected for other components. Information to the effect is shown. On the other hand, information indicating that an asymmetric feature that is determined to be abnormal is detected for all components other than the license plate is shown as a detection result for the vehicle with the recognition ID [0002]. Further, as a detection result for the vehicle having the recognition ID [0003], information indicating that the asymmetric feature has not been detected for all components and information indicating that the license plate is installed on the left side are shown. In addition, as a detection result for the vehicle with the recognition ID [0004], information indicating that an asymmetric feature is detected on the windshield and information indicating that the license plate is not detected are indicated. Yes. Further, as a detection result for the vehicle having the recognition ID [0005], information indicating that a feature due to asymmetric painting is detected for the bonnet is indicated as a feature that is not determined to be abnormal.

  Note that the windshield portion included in the image of the vehicle 1 may be excluded from the object of collation of the luminance distribution by the collation unit 220 in consideration that an object existing in the car is reflected asymmetrically. Conversely, a matching unit for matching the pattern corresponding to the outer shape of the person in the vehicle and the image of the windshield part can be added to the detection unit 14 to detect the number of persons present in the vehicle. It is.

  Whether the asymmetric feature detected by the detection unit 14 is stored in the vehicle recognition database 8 as a feature indicating vehicle abnormality or as a normal range feature is determined by, for example, the result storage unit 6. You may judge by setting conditions for each component.

  Further, a processing method suitable for detecting an asymmetric feature of a component such as a door mirror or a headlight that is separated from the left and right symmetrical positions in the vehicle 1 will be described later.

  Further, the result notifying unit 7 shown in FIG. 7 uses the information stored in the vehicle recognition database 8 as described above, for example, the information on the vehicle detected based on a predetermined condition to the operator's operation terminal ( A process of displaying on a screen (not shown) may be performed. For example, the result notification unit 7 extracts a vehicle that satisfies a condition according to a condition in which detection results of asymmetry for each component stored in the vehicle recognition database 8 are arbitrarily combined for each vehicle to be recognized.

  In the example of FIG. 11, a vehicle detected based on a condition is indicated by putting a circle in the determination column corresponding to a vehicle in which an abnormality is detected for any part in the vehicle recognition database 8.

  The result notification unit 7 can also notify the operator in an easy-to-understand manner by displaying asymmetric features using the vehicle image extracted by the image processing unit 212 of the extraction unit 11.

  FIG. 12 shows a display example of the result of detecting a vehicle having an asymmetric feature. In the example of FIG. 12, portions representing asymmetric features detected by the processing by the detection unit 14 are indicated by reference numerals P1 and P2.

  The result notifying unit 7 detects what asymmetric feature is detected by the detection unit 14 together with the name of the part from which the asymmetric feature is detected, as indicated by the symbols D1 and D2, for each part indicated by the symbols P1 and P2. You may display the message which shows whether it did.

  In the example of FIG. 12, it is shown that “shape mismatch” has been detected by the detection unit 14 by the message indicated by the symbol D1 for the bonnet portion indicated by the symbol P1. Further, regarding the door mirror portion indicated by reference sign P2, the message indicated by reference sign D2 indicates that the door mirror could not be detected by the detection unit 14.

  Next, another embodiment of the detection unit 14 suitable for detecting an asymmetric feature of a component such as a door mirror or a headlight that is separated from the left and right symmetrical positions in the vehicle 1 will be described.

  FIG. 13 shows another embodiment of the vehicle detection device. 13 that are the same as those shown in FIG. 7 are denoted by the same reference numerals and description thereof is omitted.

  The detection unit 14 illustrated in FIG. 13 includes an estimation unit 222, a region image extraction unit 223, a comparison unit 224, and a symmetry determination unit 221. Based on the center line generated by the generation unit 13, the estimation unit 222 cooperates with the control unit 9 and the vehicle feature database 5 to set a pair of left and right components that are arranged at symmetrical positions in the vehicle 1. Are estimated in the image. The region image extraction unit 223 is based on the positions estimated from the image of the vehicle 1 extracted by the image processing unit 212 of the extraction unit 11 corresponding to the pair of parts to be verified by the estimation unit 222. Extract images in the range. The comparison unit 224 compares the image feature of the region extracted for the left part by the region extraction unit 223 with the image feature of the region extracted for the right part. For example, the comparison unit 224 may obtain the similarity between the image feature of the area extracted for the left part and the image feature of the area extracted for the right part. Further, the symmetry determination unit 221 may determine whether the left and right parts have asymmetric characteristics based on whether the similarity obtained by the comparison unit 224 is lower than a predetermined threshold. Good.

  FIG. 14 is a flowchart showing another example of processing for detecting asymmetric features. Of the steps shown in FIG. 14, the same steps as those shown in FIG. 10 are denoted by the same reference numerals, and description thereof is omitted. Each step shown in FIG. 14 is an example of the process of step 305 shown in FIG. Moreover, each step shown in FIG. 14 is performed by each part and the control part 9 which are included in the detection part 14 shown in FIG.

  In step 305-21, for example, among the many parts included in the vehicle 1, the control unit 9 is a part of a part in which parts corresponding to each other such as a headlight and a door mirror are arranged symmetrically apart from each other. Pairs are sequentially selected as parts to be verified. For example, the control unit 9 may perform the processing of step 305-21 using information stored in the vehicle feature database 5. The control unit 9 designates the selected pair of parts and instructs the estimation unit 222 to estimate the position.

  In response to this, the estimation unit 222 estimates the position in the image of the part representing the left and right parts included in the vehicle image based on the information in the vehicle feature database 5 and the ratio obtained by the calculation unit 12 described above. (Step 205-22). The vehicle feature database 5 includes, for example, information indicating the approximate positions of the both ends of the lower contour of the front grille and the left and right headlights corresponding to the line segment indicated by E3 in FIG. Based on this information and the ratio obtained by the calculation unit 12, the estimation unit 222 estimates the positions of the left and right headlight portions included in the vehicle image in the image.

  As described above, based on the positions estimated for the left and right parts, the region image extraction unit 223 extracts a predetermined range of images from the vehicle image (steps 305-23). For example, the region extraction unit 223 may extract an image of a region corresponding to the size of the left and right components with the position estimated by the estimation unit 222 as the center. At this time, the region extraction unit 223 may obtain the sizes of the left and right parts in the image based on the ratio obtained by the calculation unit 12.

  The images of the regions extracted for the left and right parts in this way are passed to the comparison unit 224. The comparison unit 224 performs a process of comparing the respective features of the images of the regions extracted for the left and right parts (step 305-24). For example, the comparison unit 224 extracts features for the luminance distribution of the image of the region extracted for the left part and the luminance distribution of the image of the region extracted for the right part, and obtains the similarity of the luminance distribution. Also good. Further, the comparison unit 224 may obtain the degree of similarity between each other using the average luminance of the regions corresponding to the left and right parts and the colors of the regions as the characteristics of the respective images.

  As described above, the detection unit 14 compares the characteristics of the image around the position estimated for the part pair. As a result, it is possible to determine the left-right symmetry of components grounded away from the symmetry line in the horizontal direction of the front of the vehicle 1 regardless of the distortion of the image caused by photographing from the oblique direction.

  In any case, when the similarity determined by the comparison unit 224 is higher than a predetermined threshold (negative determination at step 305-25), the symmetry determination unit 221 determines that the step 305-7 shown in FIG. Similarly, the fact that the left and right parts under verification are normal is output. On the other hand, when the degree of similarity obtained by the comparison unit 224 is lower than a predetermined threshold (Yes in Step 305-25), asymmetric features are detected for the left and right parts (Step 305-9). Based on the detection result, the result storage unit 6 stores the asymmetric feature detected in step 305-9 in the vehicle authentication database 8 in correspondence with the recognition ID indicating the vehicle 1.

  For example, when the similarity of the luminance distribution of the region corresponding to the left and right headlights obtained by the comparison unit 224 is equal to or less than a predetermined threshold, the symmetry determination unit 221 indicates that the luminance of the headlight is asymmetric. Is output as a detection result. In response to this, the result storage unit 6 turns on only one headlight in the vehicle recognition database 8 as shown in the example of FIG. 11 corresponding to the recognition ID [0001] of the recognition target vehicle. Save that you are. The comparison unit 224 can also compare the average luminance of the areas corresponding to the left and right headlights and output information indicating which one has the lower luminance. If this information is used, the result storage unit 6 clearly shows the part suspected of being abnormal as shown in parentheses in the headlight column of the recognition ID [0001] of the recognition target vehicle in FIG. You can also.

  Further, for example, when one of the door mirrors is damaged, the color similarity obtained by the comparison unit 224 is lower than a predetermined threshold for the region extracted corresponding to the estimated positions of the left and right door mirrors. Become. In this case, the symmetry determination unit 221 outputs information indicating that the door mirror is asymmetric as a detection result. In response to this, the result storage unit 6 stores in the vehicle recognition database 8 that one of the door mirrors cannot be detected, as shown in association with the recognition ID [0002] of the recognition target vehicle in the example of FIG. To do. The comparison unit 224 can also compare the color of the area corresponding to the left and right door mirrors with the color of the body part and output information indicating which area has the same color. If this information is used, the result storage unit 6 clearly shows the parts suspected of being abnormal as shown in parentheses in the door mirror column of the recognition ID [0002] of the vehicle to be recognized in FIG. You can also.

  Note that components such as door mirrors arranged at positions deviating from the center of the vehicle 1 may not appear in the image depending on the shooting direction of the camera 2. In order to deal with such a case, for example, when the ratio obtained by the calculation unit 12 is not included in a predetermined range, the parts to be verified may be limited.

  In FIG. 11, the sign “−” shown in the column of the door mirror of the recognition ID [0005] of the recognition target vehicle indicates that the door mirror is excluded from the symmetry verification target for the vehicle.

  By the way, when the shooting direction of the camera 2 is the front direction of the vehicle 1, the center line generated by the generation unit 13 described above matches the center line of the outer shape of the vehicle image included in the image obtained by the camera 2. To do. In this case, the detection unit 14 can determine whether each component included in the vehicle 1 has an asymmetric feature by simply comparing the vehicle images on the left and right. Note that whether or not the shooting direction of the camera 2 is the front direction of the vehicle 1 is, for example, whether or not the license plate image extracted by the license plate extraction unit 4-a illustrated in FIG. It can be judged based on.

DESCRIPTION OF SYMBOLS 1 ... Vehicle; 1g ... Vehicle image, 2 ... Camera; 3 ... Image input part; 4 ... License plate recognition apparatus; 4-a ... License plate extraction part; 4-b ... Number recognition processing part; DESCRIPTION OF SYMBOLS 5 ... Vehicle characteristic database; 6 ... Result preservation | save part; 7 ... Result notification part; 8 ... Vehicle recognition database; 9 ... Control part; 10 ... Vehicle detection apparatus; 11 ... Extraction part; DESCRIPTION OF SYMBOLS 14 ... Detection part; 21 ... Processor; 22 ... Memory; 23 ... Hard disk drive (HDD); 24 ... Display control part; 25 ... Display device; 26 ... Input device; 27 ... Image input device; Communication control unit; 30 ... network; 31 ... removable disk; 212 ... image processing unit; 213 ... line segment extraction unit; 214 ... emblem extraction unit; 216 ... coordinate calculation unit; Department; 218 ... partial image extracting unit; 219 ... deformation unit; 220 ... matching part; 221 ... symmetry decision section; 222 ... estimator; 223 ... area image extracting unit; 224 ... comparing unit

Claims (5)

An extraction unit that extracts a feature portion including a portion indicating a center position of the front surface of the vehicle and a plurality of line segments indicating a straight line portion extending in a horizontal direction on the front surface of the vehicle from an image obtained by photographing the vehicle;
Based on the relative position of the line segment adjacent to the part indicating the central position and the part indicating the central position, the center of the line segment is obtained, and the distance from the center of the line segment to one end of the line segment; A calculation unit that calculates a ratio of the distance from the center of the line segment to the other end of the line segment;
A generating unit for internally dividing the plurality of line segments based on the ratio, and generating a center line connecting the internal dividing points of the line segments;
In the image of the portion representing the vehicle included in the image obtained by photographing the vehicle, the left side portion and the right side portion of the front surface of the vehicle are compared with the center line as a boundary, thereby being asymmetrical with respect to the center line A detection unit for detecting a region of an image having
A vehicle detection apparatus comprising: The vehicle detection device according to claim 1,
The detector is
A deformation unit that applies deformation based on the ratio to two partial images obtained by dividing a portion of the image corresponding to a member having a symmetrical shape in the vehicle by the center line;
A vehicle detection device comprising: a verification unit that collates the two partial images deformed by the deformation unit with reference to the center line. The vehicle detection device according to claim 1,
The detector is
An estimation unit that estimates the positions of a pair of members arranged symmetrically in the vehicle based on the ratio;
A vehicle detection apparatus comprising: a comparison unit that compares the characteristics of the image with respect to a predetermined region including a position estimated for the pair of members by the estimation unit. Extracting a feature portion including a portion indicating a center position of the front of the vehicle and a plurality of line segments indicating a straight line portion extending in a horizontal direction on the front of the vehicle from an image of the vehicle,
Based on the relative position of the line segment adjacent to the part indicating the central position and the part indicating the central position, the center of the line segment is obtained, and the distance from the center of the line segment to one end of the line segment; Calculate the ratio of the distance from the center of the line segment to the other end of the line segment,
Obtaining an internal dividing point that internally divides the plurality of line segments based on the ratio, and generating a center line connecting the internal dividing points of the line segments;
In the image of the portion representing the vehicle included in the image obtained by photographing the vehicle, the left side portion and the right side portion of the front surface of the vehicle are compared with the center line as a boundary, thereby being asymmetrical with respect to the center line Detect areas of images with
The vehicle detection method characterized by the above-mentioned. Extracting a feature portion including a portion indicating a center position of the front of the vehicle and a plurality of line segments indicating a straight line portion extending in a horizontal direction on the front of the vehicle from an image of the vehicle,
Based on the relative position of the line segment adjacent to the part indicating the central position and the part indicating the central position, the center of the line segment is obtained, and the distance from the center of the line segment to one end of the line segment; Calculate the ratio of the distance from the center of the line segment to the other end of the line segment,
Obtaining an internal dividing point that internally divides the plurality of line segments based on the ratio, and generating a center line connecting the internal dividing points of the line segments;
In the image of the portion representing the vehicle included in the image obtained by photographing the vehicle, the left side portion and the right side portion of the front surface of the vehicle are compared with the center line as a boundary, thereby being asymmetrical with respect to the center line Detect areas of images with
A vehicle detection program that causes a computer to execute processing.

Images