JP2010061375A - Apparatus and program for recognizing object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a program for recognizing an object for preventing erroneous recognition of an object due to a reflected image without having special equipment to improve recognition accuracy of object recognition. <P>SOLUTION: The apparatus extracts a plurality of feature points through edge detection from an image photographed with a photograph device 12, associates a feature point of the latest image with a feature point of an image photographed in the preceding frame if a vehicle travels, if the coordinates of the associated feature points are the same as each other, the apparatus stores the feature points as candidate points of a fixed feature point together with a candidate number count. The apparatus determines candidate points by repeating processing to each image of a predetermined number of frames, adds one to the candidate number count if the coordinates are the same as those of the stored candidate points, and determines the candidate point, as the fixed feature point, of which the candidate number count becomes not smaller than a predetermined value to eliminate the fixed feature point from the feature point extracted from the latest image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object recognition apparatus and program, and more particularly to an object recognition apparatus and program for recognizing an object around a vehicle from an image captured by a camera or the like.

  2. Description of the Related Art Conventionally, a driver driving a vehicle is assisted by recognizing an object from a captured image and displaying a recognition result on a display, for example. In many cases, when using an image captured in this way, an image capturing device such as a camera is mounted in the vehicle, and for example, an image obtained by capturing the front of the vehicle through the windshield is used. In this case, light from the object in the vehicle is reflected by the windshield and incident on the in-vehicle camera, and the object in the vehicle is reflected in the captured image, so-called reflection image causes erroneous recognition. There was a problem.

  Therefore, an illuminating device that illuminates the dashboard inside the windshield and in the vicinity of the camera facing forward is provided to create a differential image of two edge images obtained from images taken when the illuminating device is turned on and off. Then, an average value and a variance value of each pixel value are calculated for a plurality of difference images, and pixels having an average value that is equal to or greater than a predetermined threshold and a variance value that is equal to or less than the predetermined threshold are taken as pixels occupied by the edge of the reflected image A reflection detection method has been proposed in which an edge of a reflection image is removed from a traveling road image by detecting and setting this pixel value to zero (for example, Patent Document 1).

  Also, there has been proposed a reflection preventing method in which a polarizing filter is provided on the front surface of the camera, the light incident on the camera is reflected by the windshield, and the reflection on the captured image is prevented (for example, Patent Document 2). ).

Also, a difference image between an image obtained by projecting a projected image onto a target object existing in the real space and capturing the projected real space, and an estimated reflected image created by correcting the projection source image An image processing apparatus that removes a reflected image (projected image) by extracting the image has been proposed (for example, Patent Document 3).
JP 2002-230563 A JP 2001-94842 A JP 2006-259627 A

  However, the reflection detection method disclosed in Patent Document 1 has a problem that the illumination device is provided as a special equipment for obtaining a difference image, which increases the cost and restricts the mounting conditions. In addition, the reflection prevention method disclosed in Patent Document 2 also requires a polarizing filter, which increases costs and reduces the amount of captured image light. In addition, since the image processing apparatus of Patent Document 3 is based on the premise that the pattern of the reflected image is known, it is possible to cope with a case where it is unclear what pattern is detected as the reflected image. There is a problem that it is not possible.

  The present invention has been proposed to solve the above-described problem, and can prevent the object from being erroneously recognized by the reflected image without providing any special equipment and improve the object recognition accuracy. An object is to provide a recognition device and a program.

  In order to achieve the above object, an object recognition apparatus according to the present invention includes an imaging unit that continuously captures a target region in a predetermined direction around a vehicle and outputs a plurality of images, and each image captured by the imaging unit. A feature point extracting means for extracting a plurality of feature points from the image, and a fixed feature point extracted from each image captured within a predetermined time and having the same coordinates in each image captured within the predetermined time. Fixed feature point removing means for removing from a plurality of feature points extracted from the latest image captured in time, and feature points of the latest image from which fixed feature points have been removed by the fixed feature point removing means And object recognition means for performing object recognition.

  In the object recognition program of the present invention, the computer extracts a plurality of feature points from each image captured by an imaging unit that continuously captures a target area in a predetermined direction around the vehicle and outputs a plurality of images. The feature point extracting means and the latest feature imaged within the predetermined time are fixed feature points extracted from each image captured within the predetermined time and having the same coordinates in each image captured within the predetermined time. Fixed feature point removing means for removing from a plurality of feature points extracted from an image, and object recognition for performing object recognition based on feature points of the latest image from which fixed feature points have been removed by the fixed feature point removing means It is for functioning as a means.

  According to the object recognition device and the program of the present invention, the imaging unit continuously captures a target area in a predetermined direction around the vehicle and outputs a plurality of images, and the feature point extraction unit is captured by the imaging unit. A plurality of feature points are extracted from each image. The feature points extracted here may include feature points extracted based on the reflected image. Since the reflected image is captured at the same location in the image for a long time, the fixed feature point removing means is extracted from each image captured within a predetermined time, and in each image captured within a predetermined time The fixed feature point having the same coordinate is removed from a plurality of feature points extracted from the latest image captured within a predetermined time, and the object recognition unit is the latest in which the fixed feature point is removed by the fixed feature point removing unit. Object recognition is performed based on the feature points of the image.

  In this way, the fixed feature points having the same coordinates in each image captured within a predetermined time are removed as feature points extracted based on the reflected image, and the feature points of the latest image from which the fixed feature points are removed Therefore, the object recognition accuracy can be improved by preventing the erroneous recognition of the object by the reflected image without providing any special equipment.

  In addition, an image captured within the predetermined time can be at least three frames. As described above, by determining the fixed feature point based on the image of three frames or more, the probability of determining the feature point having the same accidental coordinates but not by the reflected image as a fixed feature point is reduced, and the It is possible to remove fixed feature points from the reflected image.

  In addition, the object recognition device of the present invention further includes vehicle travel detection means for detecting whether or not the vehicle is traveling, and the fixed feature point removal means is configured to cause the vehicle to travel by the vehicle travel detection means. If it is detected that the fixed feature point is detected, the fixed feature point can be removed.

  As described above, when the vehicle is traveling, by removing fixed feature points, the feature points extracted from each image captured within a predetermined time are coordinated according to the traveling state of the vehicle. Since the feature point that changes and the fixed feature point are more clearly distinguished, the fixed feature point can be removed more appropriately, and the accuracy of object recognition is improved.

  As described above, according to the object recognition apparatus and program of the present invention, it is possible to prevent the object from being erroneously recognized by the reflected image without providing special equipment, and to improve the accuracy of the object recognition. An effect is obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, the object recognition device 10 according to the present embodiment detects the vehicle speed of the vehicle 1, the imaging device 12 mounted in the vehicle interior so as to capture the range including the target area in front of the vehicle 1. A vehicle speed sensor 14, a control device 16 that executes a processing routine of an object recognition program that recognizes an object based on a captured image output from the imaging device 12, and a display device 18 that displays a processing result in the control device 16. I have.

  The imaging device 12 captures a range including a target area in front of the vehicle 1 frame by frame in response to an imaging instruction from the control device 16 and generates an image signal for each frame, and an analog generated by the imaging unit. An A / D conversion unit that converts an image signal that is a signal into a digital signal, and an image memory that temporarily stores the A / D converted image signal are provided.

  The vehicle speed sensor 14 is provided at a position where the rotational speed of the output shaft of the engine of the vehicle 1 can be detected, and generates a signal corresponding to the vehicle speed of the vehicle 1 and inputs it to the control device 16.

  The control device 16 includes a CPU that controls the entire object recognition device 10, a ROM as a storage medium that stores programs such as an object recognition process and a fixed feature point determination and removal process described later, a RAM that temporarily stores data as a work area, It consists of a microcomputer including an I / O (input / output) port and a bus connecting them. An imaging device 12, a vehicle speed sensor 14, and a display device 18 are connected to the I / O port.

  Next, the processing routine of the object recognition program in the present embodiment will be described with reference to FIG. Here, a case where the recognition target object is a lane (white line) will be described. This program is executed by the CPU when a processing start instruction signal is received from a vehicle control device (not shown), for example, when an ignition switch of the vehicle 1 on which the object recognition device 10 is mounted is turned on.

  In step 100, a vehicle speed detection start signal for starting detection of the vehicle speed v by the vehicle speed sensor 14 is transmitted to the vehicle speed sensor 14. Next, in step 102, 1 is set to the counter value k. Next, in step 104, an imaging instruction signal for imaging an image for one frame is transmitted to the imaging device 12, and then in step 106. Thus, the image I (k) captured by the imaging device 12 is captured. In the captured image, as shown in FIG. 3A, when there is an image (reflection image 32) in which the object 36 in the vehicle interior reflected on the windshield 34 is reflected, An image obtained by capturing the target area and a reflected image 32 are included.

  Next, in step 108, edge detection is performed on the image input from the imaging device 12 using a differential filter or the like, and the edge point of the object included in the image I (k) is determined as the feature point Pi (k) (i = 1, 2, 3,... N: n is extracted as the total number of feature points extracted from the image I (k). An example of extracting feature points from the image shown in FIG. 3A is shown in FIG. The edge point of the lane portion is extracted as the feature point by the edge detection, and the edge point of the reflected image 32 is also extracted as the feature point.

  Next, in step 110, it is determined whether or not a feature point has been extracted from an image of two or more frames by determining whether or not the counter value k is greater than one. If it is greater than 1, go to step 114, otherwise go to step 112. Since k = 1 here, the process proceeds to step 112 where the counter value k is incremented and the process returns to step 104.

  Returning to step 104, an imaging instruction signal is transmitted to the imaging device 12, and in step 106, for example, the next image I (k) as shown in FIG. 3C is captured, and in step 108, for example, FIG. As shown in (D), feature points of the image I (k) are extracted.

  In the next step 110, since the counter value k is larger than 1 and the feature points are extracted from the two frame images, the process proceeds to step 114, and the image I (k) detected by the vehicle speed sensor 14 is displayed. The vehicle speed v (k) signal of the vehicle at the time of imaging is captured.

  Next, in step 116, it is determined whether or not the vehicle is traveling by determining whether or not the captured vehicle speed v (k) is greater than zero. If it is detected that the vehicle is traveling, the routine proceeds to step 118, where fixed feature point determination removal processing is executed. If the vehicle is not running, the process proceeds to step 120 without executing the fixed feature point determination removal process.

  Here, the processing routine of the fixed feature point determination removal program will be described with reference to FIG.

In step 200, the counter value i is set to 1. Next, in step 202, the feature point P _i (k) of the image I (k) that is the latest image and the image I ( k-1) with the feature points P _j (k-1) (j = 1, 2, 3... l: l is the total number of feature points extracted from the image I (k-1)). performed, the feature point _P i (k) in correspondence is the feature point _P j (k-1) determining whether there is. The association of feature points can be made to associate pairs of points having similar brightness distributions in small areas set around the feature points using, for example, values such as SSD (Sum of Squared Differences). . Further, the association may be performed only from the coordinates of the feature points, or the association may be performed based on the coordinates of the feature points and the attribute of the feature quantity such as the edge direction. If there is a feature point to be associated, the process proceeds to step 204; otherwise, the process proceeds to step 214.

In step 204, the coordinates of the associated feature point P _i (k) and feature point P _j (k−1) are calculated, and it is determined whether the coordinates are the same. If the coordinates are the same, it is determined that the feature point P _i (k) is a candidate point L for the fixed feature point, and the process proceeds to step 206. If not, the process proceeds to step 214.

In step 206, with reference to the predetermined storage area candidate point L of the fixing characteristic points are stored, from which one candidate point L _m of coordinates of the same coordinate of the candidate point L you are trying to storage has already been stored whether Determine whether. The predetermined storage area, the candidate count indicating the number of times is determined as the coordinates and the candidate points of the candidate point L _m is stored. If the candidate point L has already been stored, the process proceeds to step 208, one is added to the candidate count of the corresponding candidate points L _m. If the candidate point L has not yet been stored, the process proceeds to step 210, as a new candidate point L _m, stores "1" that coordinates and candidate count. Note that m is a serial number assigned to the candidate points stored in the predetermined area.

  Next, in step 212, it is determined whether the counter value i is equal to n, which is the total number of feature points P (k) of the image I (k), thereby determining all the features of the image I (k). It is determined whether or not it is determined whether or not the point P (k) is a candidate point L. If i = n, the process proceeds to step 216. If i = n is not satisfied, the counter value i is incremented in step 214, the process returns to step 202, and the processes in steps 202 to 212 are repeated. .

Next, at step 216, among the candidate points L _m of a fixed point feature stored in a predetermined area, 1 to the number of candidates counted in step 208 deletes the candidate point L which have not been added. The fact that 1 is not added to the candidate count means that the feature point of the same coordinate has not been extracted in the latest image I (k), and is not a fixed feature point. Note that the newly stored candidate point L _m in step 210, since the number of candidates count 0 1 is the same as the summed, not removed in this step.

Next, in step 218, 1 is set to the counter value m, then, at step 220, by referring to the predetermined region candidate point L _m is stored, the candidate count is a predetermined value of the candidate point L _m By determining whether or not it is the above, it is determined whether or not the image is a feature point of the same coordinate in each image of a predetermined frame or more. Here, the predetermined value is set by the predetermined number of frames of the image, but the number of imaging instructions transmitted within a predetermined time may be calculated and set as the predetermined value. In the case of more than the predetermined value, the routine proceeds to step 222, it determines the candidate point L _m as a fixed point feature. If smaller than the predetermined value, the process proceeds to step 224.

In step 224, by determining whether or not the counter value m is equal to the maximum serial number m _{max of} stored fixed feature point candidate points, for all stored candidate points L _m , It is determined whether or not the candidate count has been determined. If m = m _max, the process proceeds to step 228. If m = m _max is not satisfied, the counter value m is incremented in step 226, the process returns to step 220, and the processing from step 220 to step 224 is performed. repeat.

In step 228, the feature point having the same coordinate as that of the fixed feature point determined in step 222 is removed as the fixed feature point of the image I (k), the process is terminated, and the process returns. For example, when the feature points extracted from the latest image I (k) are shown in FIG. 3D and the feature points extracted from the image I (k−1) are shown in FIG. When the fixed feature points as shown in FIG. 5 are determined through the steps, the fixed feature points shown in FIG. 5 are removed from the plurality of feature points shown in FIG. 6 remains. If there is no candidate point L _m whose candidate count is equal to or greater than the predetermined value in step 220, no fixed feature point is determined, and therefore no fixed feature point is removed.

  Next, in step 120 of FIG. 2, an object is recognized based on the feature points from which the fixed feature points have been removed. Object recognition processing is executed based on the feature point P (k) of the extracted image I (k). Here, when the fixed feature point determination removal process is performed in step 112, object recognition is performed based on the remaining feature points from which the fixed feature points are removed from the plurality of feature points extracted in step 106. Do. For the object recognition process, a known technique such as pattern matching or comparison with a learning model created in advance can be used, and therefore, detailed description thereof is omitted.

  Next, in step 122, based on the recognition result in step 116, for example, processing for changing or enhancing the color of the recognized lane portion is performed on the image captured from the imaging device 12 and displayed. Display on device 18.

  Next, in step 124, for example, it is determined whether or not a processing end instruction signal is input from a vehicle control device (not shown) by turning off the ignition switch of the vehicle. The value k is incremented and the process returns to step 104 and is repeated. When a process end instruction signal is input, this routine ends.

  If the control device 16 that executes this process is described in terms of functional blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. Extracted by a feature point extraction unit 20 that extracts feature points, a vehicle travel detection unit 22 that detects whether the vehicle is traveling based on a signal corresponding to the vehicle speed input from the vehicle speed sensor 14, and a feature point extraction unit 20 A fixed feature point determination unit 24 that determines a fixed feature point from the determined feature points, a fixed feature point removal unit 26 that removes a feature point determined as a fixed feature point by the fixed feature point determination unit 24, and a vehicle is running. In this case, based on the feature point from which the fixed feature point is removed from the feature point extracted by the feature point extracting unit 20, the feature point extracted by the feature point extracting unit 20 when the vehicle is not traveling. Based on And an object recognition unit 28 for recognizing an object, and a display control unit 30 for controlling the recognition result of the object recognition unit 28 to be superimposed on the captured image captured by the imaging device 12 and displaying the image on the display device 18. Can be expressed as

  As described above, according to the object recognition device 10 according to the present embodiment, when the vehicle is running, among the feature points extracted from each image of a predetermined frame or more captured within a predetermined time. Since the feature points having the same coordinates in each image are determined as fixed feature points based on the reflected image and removed from the extracted feature points, the object by the reflected image is not provided with any special equipment. Can be prevented and the accuracy of object recognition can be improved.

  In the present embodiment, the case where the edge point extracted by edge detection from the input image is used as the feature point has been described. However, the shape, contour, color, pattern, etc. of the object are used as the feature point for recognizing the object. For example, a plurality of pixels corresponding to a contour extracted by matching an input image with a predetermined pattern image may be extracted as feature points, or a predetermined color may be extracted. Alternatively, a pixel having a pixel value indicating 特 徴 may be extracted as a feature point.

  In the present embodiment, the case where whether or not the vehicle is running is detected only by the vehicle speed has been described. However, the steering angle detected by the steering angle sensor may also be used for detection. In the present embodiment, the vehicle travel is detected when the vehicle speed is greater than 0. However, the vehicle travel may be detected when the vehicle speed is equal to or higher than a predetermined value.

  In this embodiment, the case where the recognition target object is a lane has been described. However, a road surface, a plant, the sky, a building, or the like may be used as the recognition target object, and a plurality selected from these may be used as the recognition target object. Also good.

  Moreover, although the case where the recognition result is displayed on the display device has been described in the present embodiment, the recognition result may be used as input information of another device such as a pedestrian detection device or a vehicle control device. For example, by inputting the recognition result to the pedestrian detection device and limiting the search range of the pedestrian, it is possible to reduce the calculation cost and the false detection rate in the pedestrian detection device. Further, the recognition result may be input to the vehicle control device and used for control such as automatic tracking.

  Further, the configuration of the object recognition apparatus 10 according to the present embodiment and the flow of the object recognition processing program including the fixed feature point determination and removal processing are merely examples, and can be changed as appropriate without departing from the scope of the present invention. It is.

It is the schematic which shows the structure of the object recognition apparatus which concerns on this Embodiment. It is a flowchart which shows the processing routine of the object recognition program which concerns on this Embodiment. It is a figure which shows an example of the feature point extracted from the image imaged with the imaging device, and the imaged image. It is a flowchart which shows the processing routine of the fixed feature point determination removal program in this Embodiment. It is a figure which shows an example of a fixed feature point. It is a figure which shows an example of the result by which the fixed feature point was removed from the extracted feature point. It is a functional block diagram which shows the functional structure of the object recognition apparatus which concerns on this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object recognition apparatus 12 Imaging device 14 Vehicle speed sensor 16 Control apparatus 18 Display apparatus 20 Feature point extraction part 22 Vehicle travel detection part 24 Fixed feature point determination part 26 Fixed feature point removal part 28 Object recognition part 30 Display control part

Claims (4)

Imaging means for continuously imaging a target area in a predetermined direction around the vehicle and outputting a plurality of images;
Feature point extracting means for extracting a plurality of feature points from each image imaged by the imaging means;
A plurality of fixed feature points extracted from each image captured within a predetermined time and having the same coordinates in each image captured within the predetermined time, extracted from the latest image captured within the predetermined time Fixed feature point removing means for removing from the feature points;
Object recognition means for performing object recognition based on the feature points of the latest image from which fixed feature points have been removed by the fixed feature point removal means;
An object recognition apparatus including:   The object recognition apparatus according to claim 1, wherein an image captured within the predetermined time is at least three frames. Vehicle travel detection means for detecting whether or not the vehicle is traveling;
The object recognition apparatus according to claim 1, wherein the fixed feature point removing unit removes the fixed feature point when the vehicle running detection unit detects that the vehicle is running. Computer
Feature point extracting means for extracting a plurality of feature points from each image captured by an imaging means for continuously capturing a target region in a predetermined direction around the vehicle and outputting a plurality of images;
A plurality of fixed feature points extracted from each image captured within a predetermined time and having the same coordinates in each image captured within the predetermined time, extracted from the latest image captured within the predetermined time Fixed feature point removing means for removing from the feature points;
Object recognition means for performing object recognition based on the feature points of the latest image from which fixed feature points have been removed by the fixed feature point removal means;
Object recognition program to make it function.