JP2008015771A - Image recognition device, image recognition method, and vehicle control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce computing volume while maintaining recognition accuracy in recognition of a pedestrian. <P>SOLUTION: When a frame of a determination object area is captured from an image P1 shot by a camera, and if the frame of the determination area is captured from a lower side in the image, the capturing spacing is made larger compared with the case when capturing is made from the upper side. Then, the matching levels with the template in the captured determination object area which is captured with comparatively large spacing is compared. For the determination object area of comparatively high matching rate, the determination target area including whole pedestrian image is captured by performing shift matching to the circumference or performing the pattern matching for the second time by enlarging the size of the capturing area. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image recognition apparatus that recognizes a specific object from an input image captured by a camera, and a method thereof, and more particularly, to an image recognition apparatus that recognizes a pedestrian, an image recognition method, and a vehicle control apparatus.

  It is particularly important to avoid collisions with pedestrians while the vehicle is running. Therefore, in recent years, a technique for recognizing pedestrians around the own vehicle using image recognition and radar detection has been devised (see, for example, Patent Document 1).

JP 2002-362302 A

  However, conventionally, in order to find an area that matches the pattern image of a pedestrian, a search is performed by shifting one pixel at a time regardless of the position in the image, which requires a large amount of computation. There is a problem that the processing time required for recognition increases.

  On the other hand, the possibility that a pedestrian image exists in the image and the size of the image vary depending on the position in the image and are not uniform. When recognizing a pedestrian from a running vehicle and using it to avoid danger, it is very important to shorten the time required to recognize the pedestrian, so it is possible to control the matching interval according to the position in the image. There has been a problem of reducing the amount of calculation while improving the efficiency of processing and maintaining the accuracy of pedestrian recognition.

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems, and reduces the amount of calculation while maintaining the recognition accuracy, and achieves a high-speed and high-accuracy image recognition apparatus and image recognition. It is an object to provide a method and a vehicle control device.

  In order to solve the above-described problems and achieve the object, an image recognition device, an image recognition method, and a vehicle control device according to the present invention extract a determination target region from an input image captured by a camera, When the presence of a specific object is recognized by comparing with the reference image, the interval between the extraction positions is controlled based on the position of the determination target region in the input image.

  According to the present invention, an image recognition device, an image recognition method, and a vehicle control device efficiently extract a place where a specific object is likely to exist by extracting determination target regions at different intervals based on a position in a screen. Therefore, it is possible to obtain a high-speed and high-accuracy image recognition apparatus, image recognition method, and vehicle control apparatus while maintaining the recognition accuracy.

  Exemplary embodiments of an image recognition device, an image recognition method, and a vehicle control device according to the present invention will be described below in detail with reference to the accompanying drawings.

  First, an image recognition method according to the present invention will be described with reference to FIG. In the present invention, when a determination target region is cut out from an image captured by the camera, the interval of extraction (extraction) is controlled based on the position of the determination target region in the input image.

  For example, in the image P1 shown in FIG. 1, when the determination area is cut out from the lower side in the image, the cut-out interval is made larger than that from the upper side. This is because the pedestrian image that appears in the lower part of the image is an image of a pedestrian close to the host vehicle, and the closer the distance from the host vehicle to the pedestrian, the larger the size of the pedestrian image in the input image. This is because even if the interval is increased, there is a higher possibility that a part of the pedestrian image will fall within the determination region. (On the other hand, the pedestrian image that appears in the upper part of the image is a pedestrian located far from the host vehicle. Since the image size is small, detection omission may occur unless the cut-out interval of the determination region is reduced.

  If a part of the pedestrian image is included in the determination target area cut out at a relatively wide interval in this way, pattern matching is performed to respond to the ratio of the part included in the determination target area to the entire pedestrian image The degree of matching with the template is obtained.

  Therefore, the determination target area including the entire pedestrian image can be obtained by shifting and matching the determination target area with a relatively high degree of matching, or by performing pattern matching again by increasing the size of the cutout area. Can be cut out.

  In this way, the position of the pedestrian is predicted from the position in the image, and the cutout interval is controlled in accordance with the prediction result (for example, the position of the search area is large for a position close to the host vehicle and the position is far from the host vehicle. In contrast, the pedestrian image can be efficiently obtained by reducing the interval between the search areas.

  As for the size of the search area (determination target area) itself, the closer to the vehicle, the larger the pedestrian image, so the search area below the image is enlarged accordingly, It is preferable to make matching easier by reducing the search area.

  Next, with reference to FIG. 2, a schematic configuration of the in-vehicle image recognition apparatus 10 which is an embodiment of the present invention will be described. As shown in the figure, the image recognition device 10 mounted on the vehicle is connected to a navigation device 30, a camera 31, a radar 33, and a pre-crash ECU 40.

  The navigation device 30 is an in-vehicle device that sets and guides a travel route using the position of the host vehicle specified by communicating with a GPS (Global Positioning System) artificial satellite and map data 30a stored in advance. In addition, the navigation device 30 provides the image recognition device 10 with location information of the host vehicle, surrounding map information, a planned travel route, and the like.

  The camera 31 captures the vicinity of the host vehicle and inputs the captured result to the image recognition apparatus 10. The radar 33 detects an object around the own vehicle and measures a distance to the object and inputs the detected object to the image recognition apparatus 10.

  The pre-crash ECU 40 receives the control of the image recognition device 10 when the image recognition device 10 predicts a collision of the host vehicle, controls the operation of the vehicle by the brake 41 and the engine control device (EFI) 42, and displays the display 43 and the speaker. 44 is an electronic control device that executes notification by 44.

  The display 43 is an output means for notifying the user, that is, the vehicle occupant by display, and the speaker 44 is an output means for notifying by voice. The display 43 and the speaker 44 can be shared by various in-vehicle devices such as the navigation device 30 and an in-vehicle audio device (not shown) in addition to performing output under the control of the pre-crash ECU 40.

  The image recognition apparatus 10 includes a preprocessing unit 11, a vehicle recognition unit 16, a white line recognition unit 17, a pedestrian recognition unit 18, and a collision determination unit 20 therein. Here, the vehicle recognition unit 16, the white line recognition unit 17, the pedestrian recognition unit 18, and the collision determination unit 20 are realized by, for example, a single microcomputer 10a (an arithmetic processing unit including a combination of a CPU, a ROM, and a RAM). Is preferred.

  The preprocessing unit 11 performs processing such as filtering, edge detection, and contour extraction on the image captured by the camera 31, and then outputs the image to the vehicle recognition unit 16, the white line recognition unit 17, and the pedestrian recognition unit 18.

  The vehicle recognition unit 16 performs pattern matching on the image output from the preprocessing unit 11 to recognize the vehicle, and outputs the recognition result to the collision determination unit 20. Further, the white line recognition unit 17 recognizes a white line by performing pattern matching or the like on the image output from the preprocessing unit 11, and outputs the recognition result to the collision determination unit 20.

  The pedestrian recognition unit 18 is a processing unit that recognizes a pedestrian image from the image (input image) output by the preprocessing unit 11, and includes a cutout unit 18a, an interval setting unit 18b, and a recognition processing unit 18c.

  The cutout unit 18a performs a process of cutting out the determination target region from the input image. The interval setting unit 18b is a processing unit that controls the cut-out interval of the determination target region by the cut-out unit 18a. The recognition processing unit 18c performs a process of recognizing the presence of a pedestrian by comparing the extracted image in the determination target region with the template.

  The collision determination unit 20 uses the recognition results obtained by the vehicle recognition unit 16, the white line recognition unit 17, the pedestrian recognition unit 18, the detection result obtained by the radar 33, and the position information output from the navigation device 30, and automatically detects the collision with the pedestrian and other vehicles. Determine the risk of collision with the vehicle.

  Specifically, the collision determination unit 20 determines the probability of a collision with a pedestrian or another vehicle, the collision time, the distance to the collision position, the collision angle, and the like, and the display 43 based on the determination result. Information display instruction, warning sound output instruction to the speaker 44, brake control instruction, EFI control instruction, and the like are output to the pre-crash ECU 40.

  Next, the cutout interval control of the determination target area will be further described. First, as shown in FIG. 3, when a pedestrian image exists below the image P2, pattern matching is performed on the determination target region cut out at a relatively wide interval, and the transition of the matching degree (matching rate) is changed. If acquired, a rough place where a pedestrian may exist can be obtained. In other words, since the search (sampling) interval is large here, the matching rate is also a rough value.

  Therefore, as shown in the image P3 in the figure, the determination target region having a relatively high degree of coincidence is lateral to the periphery (the x-axis direction of the image, horizontal to the ground, and orthogonal to the traveling direction of the host vehicle). (Direction) is cut out with a short interval, pattern matching is performed, and a position where a peak of the matching rate is obtained in the horizontal direction is searched. That is, since the search interval is small here, the matching rate can be seen in detail, and a detailed peak value can be detected.

  After that, as shown in the image P4 in FIG. 4, the pattern matching is performed by performing cutout with a shorter interval in the vertical direction (the y-axis direction of the image, the combined direction of the vertical direction with respect to the ground and the traveling direction of the host vehicle), When the position where the peak of the matching rate is obtained in the direction is searched, the determination target region including the entire pedestrian image can be cut out as shown in the image P5.

  Here, a plurality of templates are used for pedestrian recognition, and it is often determined that the person is a pedestrian if the matching rate with any template is equal to or higher than a predetermined value. In this case, after determining that there is a pedestrian by any of a plurality of templates, the calculation target amount is further reduced by excluding the determination target region from the subsequent clipping (pattern matching by other templates is omitted). be able to.

  In addition, pedestrian recognition is performed every time an image is captured by the camera 31 (for example, every several msec). Therefore, by prioritizing the cutout in the vicinity of the position where the pedestrian is recognized in the previous input image, the pedestrian recognition can be speeded up.

  For example, in the image P7 shown in FIG. 5, a pedestrian is present in the lower right in the image taken last time. Since the shooting interval of the camera 31 is sufficiently shorter than the moving speed of the pedestrian, there is a high possibility that the pedestrian is present at a position close to the previous time in the image P7.

  Therefore, in the segmentation of the determination target region with respect to the image P7, first, the determination target region with a short interval is segmented near the lower right where the pedestrian existed last time, and thereafter, in a normal interval and a normal order (from the upper left in the figure). ) The determination target area is cut out.

  Further, when a pedestrian newly appears in the image, there is a high possibility that the pedestrian will appear from the left end or the right end of the captured image. Therefore, as shown as an image P8 in FIG. 6, when the determination region is cut out from the vicinity of the left and right end portions in the input image, it is desirable to reduce the cut-out interval.

  Similarly, since a pedestrian may suddenly appear from behind the object, as shown as an image P9 in FIG. 7, the object, that is, an object that may block the pedestrian, such as another vehicle, a building, a tree, etc. If it is already recognized, it is desirable to reduce the cut-out interval near the shield. For the recognition of the shielding object, image recognition by the vehicle recognition unit 16 or the output of the radar 33 can be used.

  Next, the processing operation of the entire image recognition apparatus 10 shown in FIG. 2 will be described with reference to FIG. The processing flow shown in the figure is started when a power switch (which may be linked to an ignition switch or the like) is turned on and the camera 31 captures an image, and for each processing of an image frame (for example, every several msec). This process is repeatedly executed.

  First, the image recognition apparatus 10 performs processing such as filtering, edge detection, and contour extraction by the preprocessing unit 11 on an image captured by the camera 31 (step S101). Next, the white line recognition process (step S102) by the white line recognition part 16 and the vehicle recognition process (step S103) by the vehicle recognition part 16 are performed.

  Thereafter, the pedestrian recognition unit 18 performs pedestrian recognition (step S104), the collision determination unit 20 performs collision determination (step S105), and outputs the determination result to the pre-crash ECU 40 (step S106). finish.

  Here, the concrete processing content of the pedestrian recognition process part 18 shown as step S104 is shown in FIG. As shown in the figure, the pedestrian recognition unit 18 first sets a cut-out interval by the interval setting unit 18b (step S201), and cuts out the determination target region by the cut-out processing unit 18a at the set cut-out interval (step S201). S202). As the determination target area, a unique image remaining after the background image difference is set as a candidate area, and an area surrounding the specific image is set as a determination target.

  Next, the recognition processing unit 18c calculates the matching rate with the template (step S203), and searches for the peak value of the matching rate, that is, the value of the matching rate when the pedestrian image is completely in the determination target region. The peak value is compared with a threshold value, and the presence or absence of a pedestrian is determined based on whether or not the threshold value is exceeded (step S204), and the process ends. In recognition, the determination is made based on whether or not the degree of coincidence (%) of the shape exceeds a determination threshold.

  Next, the processing operation of the pre-crash ECU 40 will be described with reference to the flowchart of FIG. The processing flow shown in the figure is repeatedly executed during the operation of the pre-crash ECU 40.

  First, the pre-crash ECU 40 acquires a collision determination result from the image recognition device 10 as collision determination information (step S301). When the risk of collision is high (step S302, Yes), the display 43 and the speaker 44 are used to notify the driver (step S303), and the brake 41 and the EFI 42 are controlled to control the traveling state of the host vehicle. Is controlled (step S304), and the process is terminated.

  As described above, when the image recognition device 10 according to the present embodiment cuts out the determination target region from the image captured by the camera, the image recognition apparatus 10 controls the cut-out interval based on the position of the determination target region in the input image. Therefore, the amount of calculation can be reduced while maintaining the accuracy of pattern matching, and the recognition process can be speeded up.

  In addition to the cutout interval, the size of the determination target area may be controlled according to the position in the image.

  In the present invention, recognition by pattern matching has been described as an example. However, the present invention is not limited to this, and can be applied to any recognition method in which a determination target region is cut out.

  Similarly, in this embodiment, the case where a pedestrian is recognized as a specific object has been described as an example, but the present invention can also be applied to recognition of other objects such as a falling object on a road.

  As described above, the image recognition device, the image recognition method, and the vehicle control device according to the present invention are useful for image recognition in a vehicle and are particularly suitable for reducing the recognition processing load.

It is explanatory drawing explaining the image recognition method in this invention. 1 is a schematic configuration diagram illustrating a schematic configuration of an image recognition apparatus according to an embodiment of the present invention. It is explanatory drawing explaining the peak value search of a matching rate (the 1). It is explanatory drawing explaining the peak value search of a matching rate (the 2). It is explanatory drawing explaining clipping interval control using the last recognition result. It is explanatory drawing explaining the control in the case of shortening the cutting-out space | interval of the right-and-left edge part of a screen. It is explanatory drawing explaining the control in the case of shortening the cut-out space | interval of a shield vicinity. 4 is a flowchart for explaining a processing operation of the image recognition apparatus 10. It is a flowchart explaining the specific process operation | movement of a pedestrian recognition process. 4 is a flowchart illustrating a processing operation of a pre-crash ECU 40.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image recognition apparatus 10a Microcomputer 16 Vehicle recognition part 17 White line recognition part 18 Pedestrian recognition part 18a Cutout part 18b Space | interval setting part 18c Recognition processing part 20 Collision determination part 30 Navigation apparatus 30a Map data 31 Camera 33 Radar 40 Pre-crash ECU
41 Brake 42 EFI
43 Display 44 Speaker

Claims (10)

Extraction means for extracting a determination target region from an input image captured by the camera;
Recognition means for comparing the image of the determination target region with a reference image to recognize the presence of a specific object;
Based on the position in the input image, interval control means for controlling the interval of the extraction position of the determination target region by the extraction means;
An image recognition apparatus comprising:   The interval control means increases the extraction interval when extracting the determination area from the lower side in the image based on the vertical position of the determination target area in the input image as compared with the case of extracting from the upper side. The image recognition apparatus according to claim 1.   The image recognition apparatus according to claim 1, wherein the interval control unit reduces the extraction interval when a determination region is extracted from the vicinity of left and right end portions in the input image.   The said space | interval control means makes small the extraction space | interval of this shielding object vicinity, when the object which may shield the said specific object has been recognized. The image recognition apparatus described in one.   When the recognition unit uses a plurality of reference images, the interval control unit excludes a region determined by any of the plurality of reference images that the specific object is present from subsequent extraction. The image recognition apparatus according to any one of claims 1 to 4.   The interval control means reduces the extraction interval in the vicinity of the position where the specific object is recognized in the previous input image when sequentially recognizing the input images taken in time series. The image recognition apparatus as described in any one of Claims 1-5.   The interval control means causes the extraction with a reduced extraction interval to be performed again for the vicinity of the position determined to have a relatively high degree of coincidence with the reference image by the recognition processing by the recognition means. Item 7. The image recognition device according to any one of Items 1 to 6.   The image recognition apparatus according to claim 1, wherein the specific object is a pedestrian. An extraction step of extracting a determination target region from an input image captured by the camera;
A recognition step of recognizing the presence of a specific object by comparing the image of the determination target region with a reference image;
An interval control step for controlling an interval of extraction positions of the determination target region by the extraction unit based on a position in the input image;
An image recognition method comprising: Extraction means for extracting a determination target region from an input image captured by the camera;
Recognizing means for recognizing the presence of a specific object by comparing the image of the determination target region with a reference image;
Based on the position in the input image, interval control means for controlling the interval of the extraction position of the determination target region by the extraction means;
Control means for executing notification control for the driver and / or traveling control for controlling the traveling state of the vehicle based on the recognition result by the recognition means;
A vehicle control device comprising:

Images