JP2016015043A - Object recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object recognition device capable of reducing the risk of losing sight of an object and appropriately tracking the object.SOLUTION: An object recognition device of the present invention detects an object from an image obtained by imaging surroundings of an own vehicle; extracts feature points of the object; compares a plurality of image continuous in time series for an object image, thereby calculating a tracking score that becomes a lower value as changes in moving amounts and moving directions of the extracted feature points are greater; compares the tracking score with a tracking threshold for tracking, thereby tracking the object for which the tracking score is equal to or greater than the tracking threshold; stores object images of the tracked object; calculates a degree of change of the object on the basis of the stored object images; and executes threshold update processing for changing a value the tracking threshold to be lower if the calculated degree of change is equal to or higher than a predetermined value and maintaining the value of the tracking threshold if the degree of change is lower than the predetermined value.

Description

  The present invention relates to an object recognition apparatus.

  As a conventional object recognition apparatus, for example, a technique described in Patent Document 1 has been reported. In the technique described in Patent Document 1, a region in which a moving body is captured in a captured image is tracked regardless of the type of the moving body and the positional relationship between the moving bodies. Specifically, in the technique described in Patent Document 1, a captured image is divided into a plurality of areas, and a trackable area that can track a moving body is detected from the divided areas. In the technique described in Patent Document 1, the amount of movement of the traceable area on the image is detected based on at least two images captured at predetermined time intervals, Based on the amount of movement of the vehicle in time, the position of the tracking target in the trackable area in the real space is calculated. In the technique described in Patent Document 1, it is determined whether the trackable area is a stationary object area or a moving body area based on the calculated position of the tracking target and the own vehicle movement amount.

JP 2004-362334 A

  However, in the conventional object recognition device (Patent Document 1, etc.), the tracking threshold value used when tracking a moving object is a fixed value set in advance, and therefore, a steep object that the tracking threshold value is not assumed. If there is a change in the amount and direction of movement, tracking of the object is likely to fail, and the object that has been recognized as the tracking object may be lost.

  The present invention has been made in view of the above circumstances, and even if there is a sudden change in the moving amount and moving direction with respect to an object recognized as a tracking target, An object of the present invention is to provide an object recognition apparatus that can appropriately track an object by reducing the possibility of being lost.

  An object recognition apparatus according to the present invention includes an image acquisition unit that acquires an image obtained by capturing an image of the periphery of the host vehicle, and an object that detects an object existing around the host vehicle from the image acquired by the image acquisition unit. An object detection unit, a feature point extraction unit that extracts feature points of the object detected by the object detection unit, and a plurality of images that are sequentially acquired by the image acquisition unit and that are sequentially acquired in time series, By comparing the object image set in the image as a partial region including the object detected by the object detection unit, the movement amount and movement direction of the feature point extracted by the feature point extraction unit By comparing the tracking score calculated by the tracking score calculation unit with the tracking threshold for tracking, a tracking score calculation unit that calculates a tracking score that becomes a lower value as the change increases is obtained. An object tracking unit that tracks an object that is equal to or greater than the tracking threshold; an image accumulation unit that accumulates an object image of the object that can be tracked by the object tracking unit; and the image accumulation unit that accumulates the object image. A degree-of-change calculating unit that calculates the degree of change of the object based on the object image, and the tracking threshold value is lowered when the degree of change calculated by the degree-of-change calculating unit is equal to or greater than a predetermined value. And a tracking threshold update unit that executes a threshold update process for maintaining the value of the tracking threshold when the degree of change is less than a predetermined value.

  The object recognition device according to the present invention reduces the possibility of losing an object even when there is a sudden change in the amount and direction of movement of the object recognized as a tracking object. The object can be suitably tracked.

FIG. 1 is a diagram illustrating an example of a configuration of an object recognition apparatus according to an embodiment. FIG. 2 is a diagram illustrating an example of an object from which feature points are extracted. FIG. 3 is a diagram illustrating an example of an object from which feature points are extracted. FIG. 4 is a diagram illustrating a setting example of the tracking threshold. FIG. 5 is a diagram illustrating an example of processing performed by the object recognition apparatus according to the embodiment. FIG. 6 is a diagram illustrating an example of processing of the object recognition device according to the embodiment.

  Embodiments of an object recognition apparatus according to the present invention will be described below in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. In addition, constituent elements in the following embodiments include those that can be easily assumed by those skilled in the art or those that are substantially the same.

[Embodiment]
The configuration of the object according to the present embodiment will be described with reference to FIGS. 1 to 4. FIG. 1 is a block diagram illustrating an example of a configuration of an object recognition apparatus according to an embodiment. 2 and 3 are diagrams illustrating an example of an object from which feature points are extracted. FIG. 4 is a diagram illustrating a setting example of the tracking threshold.

  As shown in FIG. 1, the camera 1 is a device that captures the periphery of the host vehicle and generates image data in which the periphery of the host vehicle is projected. The camera 1 is, for example, a CCD camera or a CMOS camera that can capture an image in the visible light region or the infrared region. The camera 1 is installed at an arbitrary position where the surroundings of the vehicle such as the front, side, and rear of the vehicle can be imaged.

  The object recognition device 2 is configured using, for example, an automobile control computer mounted on a vehicle, and outputs a sound to the camera 1 mounted on the vehicle, a driver in the vehicle interior, and the like. The speaker 3 that controls the vehicle and the actuator 4 that controls the behavior of the vehicle are communicably connected.

  The object recognition device 2 includes an image acquisition unit 2a, an object detection unit 2b, a feature point extraction unit 2c, a tracking score calculation unit 2d, an object tracking unit 2e, an image storage unit 2f, and a degree of change calculation. Unit 2g, tracking threshold update unit 2h, pop-up determination unit 2i, alerting unit 2j, and vehicle control unit 2k.

  The image acquisition unit 2a is an image acquisition unit that acquires an image of the periphery of the host vehicle. The image acquisition unit 2a acquires the image data input from the camera 1 as an image obtained by imaging the periphery of the host vehicle. The image acquisition unit 2a sequentially acquires image data input from the camera 1, thereby acquiring a plurality of images obtained by imaging the periphery of the host vehicle at different times.

  The object detection unit 2b is an object detection unit that detects an object existing around the host vehicle from the image acquired by the image acquisition unit 2a. In the present embodiment, the object means a moving object to be tracked in order to detect pop-out detection described later. Examples of the moving body include pedestrians, bicycles, opponent vehicles, and the like that exist around the host vehicle. As the object detection technique, a technique known in the field such as pattern matching is used. The object detection unit 2b acquires a two-dimensional position (ordinate, abscissa), preferably a three-dimensional position (ordinate, abscissa, height) of the detected object.

  The feature point extraction unit 2c is a feature point extraction unit that extracts feature points of the object detected by the object detection unit 2b. In the present embodiment, the feature point of an object means a pixel region on an image having a feature amount that can clearly distinguish an object to be tracked from a background or an obstacle. The feature amount is an amount indicating features such as color, shape, texture, and the like. As a feature point extraction method, a method known in the field such as SIFT or SURF is used.

  Here, an example of the feature points extracted by the feature point extraction unit 2c will be described with reference to FIG. In FIG. 2, the object is a pedestrian, and the object image is set in the image as a partial area including the pedestrian. In such a case, for example, the feature point extraction unit 2c has a feature amount that can clearly distinguish a pedestrian to be tracked from a background, an obstacle, and the like regarding a pedestrian included in the object image. Five feature points 1 to 5 are extracted as the pixel region. Note that the number of feature points extracted by the feature point extraction unit 2c is not limited to five as shown in FIG. 2, but may be any number as long as an appropriate tracking score can be calculated by the tracking score calculation unit 2d described later. . In the present embodiment, the object image includes the outline of the object detected by the object detection unit 2b and the feature points extracted by the feature point extraction unit 2c, as shown in FIG. In this way, it is only necessary to be a partial region cut out from the image acquired by the image acquisition unit 2a, and the size and shape of the object image are not particularly limited.

  Returning to FIG. 1, the tracking score calculation unit 2d includes a plurality of images that are sequentially acquired by the image acquisition unit 2a in the time series as partial areas including the target detected by the target detection unit 2b. Is a tracking score calculation means for calculating a tracking score that becomes a lower value as the change in the moving amount and moving direction of the feature point extracted by the feature point extracting unit 2c is larger. . In the present embodiment, for example, the image acquired at the first time by the image acquisition unit 2a is set as the first image, and the image acquired at the second time that is later than the first time is the first image. In the case of the second image, the first image and the second image are a plurality of images that move back and forth in time series. More specifically, the first image is an image a predetermined time before the current time, and the second image is the current image. It should be noted that some or all of the plurality of images that move back and forth in time series may be object images stored in the image storage unit 2f described later.

  Specifically, the tracking score calculation unit 2d calculates the tracking score by quantifying the moving amount of the feature point and the changing amount of the moving direction in a plurality of object images that move back and forth in time series. Here, the tracking score calculation unit 2d is less likely to be able to track the feature point of the target object between the plurality of target images that are moved back and forth in time series as the movement amount of the feature point and the change amount of the movement direction are larger. As a result, the tracking score is calculated to be a low value. On the other hand, the tracking score calculation unit 2d is more likely to be able to track a feature point of a target object between a plurality of target images moving back and forth in time series as the movement amount of the feature point and the change amount of the movement direction are smaller. Predict and calculate the tracking score to be a high value. The tracking score calculated by the tracking score calculation unit 2d is used when determining whether or not the target tracking unit 2e described later has tracked the target.

  The object tracking unit 2e is an object tracking unit that tracks an object whose tracking score is equal to or greater than the tracking threshold by comparing the tracking score calculated by the tracking score calculating unit 2d with the tracking threshold for tracking. . Specifically, the object tracking unit 2e determines that an object having a tracking score equal to or higher than the tracking threshold can be tracked, while an object having a tracking score less than the tracking threshold is lost without being tracked. judge. In the present embodiment, an appropriate value determined in advance through experiments or the like is set as the initial value of the tracking threshold. Note that the initial value of the tracking threshold is updated according to the degree of change of the object by a tracking threshold update unit 2h described later.

  Here, referring to FIG. 2 and FIG. 3, as an example, the image shown in FIG. 2 is used as the first image, the image shown in FIG. 3 is used as the second image, and the tracking score calculation unit 2d and the object tracking unit 2e. The process will be described. Similar to FIG. 2, in FIG. 3, an object image is set in the image as a partial region including a pedestrian detected as an object, and five feature points are extracted from the pedestrian by the feature point extraction unit 2 c. It is assumed that points 1 to 5 have been extracted. In such a case, as shown in FIG. 2 and FIG. 3, the tracking score calculation unit 2d is the target in the first image (see FIG. 2) that is sequentially acquired by the image acquisition unit 2a. The object images in the image and the second image (see FIG. 3) are compared.

  Specifically, the tracking score calculation unit 2d is common between the first image and the second image when comparing the object image in the first image with the object image in the second image. Focusing on the same feature point, the amount of movement and the amount of change in the movement direction related to the feature point are digitized to calculate a tracking score. For example, when focusing on the first feature point among the same feature points common between the first image and the second image, the tracking score calculation unit 2d uses the feature point 1 in the first image as a reference. The tracking score is calculated by quantifying the amount of movement up to the feature point 1 and the amount of change in the direction of movement in the second image. Similarly, the tracking score calculation unit 2d calculates a tracking score for each of the same feature points for the feature points 2 to 5 other than the feature point 1. The tracking score calculation unit 2d collects the tracking scores calculated for the feature points 1 to 5, respectively, takes an average value, and compares the average value with the first image and the second image. Calculate as a tracking score. Then, the object tracking unit 2e compares the average value of the tracking score for the feature points 1 to 5 calculated by the tracking score calculation unit 2d with the tracking threshold for tracking. At this time, when the average value of the tracking score is equal to or greater than the tracking threshold, the target tracking unit 2e determines that the tracking target is tracked, while the average tracking score is less than the tracking threshold In the case of, it is determined that the object that was the tracking target is lost without being tracked.

  Returning to FIG. 1, the image accumulating unit 2 f is an image accumulating unit that accumulates the object image of the object tracked by the object tracking unit 2 e. The image accumulating unit 2f associates the object image cut out from the image so as to include the object tracked by the object tracking unit 2e with the time when the image including the object is acquired. A tracking object image database is constructed by storing in a computer memory. The object image accumulated in the image accumulating unit 2f is a plurality of objects that are moved back and forth in time series by the tracking score calculating unit 2d described above when calculating the degree of change of the object by the degree-of-change calculating unit 2g described later. Used when comparing images.

  The degree-of-change calculator 2g is a degree-of-change calculator that calculates the degree of change of the object based on the object image accumulated in the image accumulator 2f. In the present embodiment, the degree-of-change calculation unit 2g compares a plurality of target images that are accumulated in time series in the image storage unit 2f and changes the shape information of the target object between these target images. From this, the degree of change indicating the apparent change in the posture of the object is calculated. The degree-of-change calculation unit 2g predicts that the apparent change of the target object increases as the difference in shape information between the target object images to be compared increases, so that the value of the degree of change of the target object increases. calculate. On the other hand, the degree-of-change calculation unit 2g predicts that the apparent change of the object becomes smaller as the difference in shape information between the object images to be compared becomes smaller, and the value of the degree of change of the object becomes smaller. Calculate as follows. The degree of change of the object calculated by the degree-of-change calculator 2g is used when the tracking threshold is updated according to the degree of change of the object by the tracking threshold update unit 2h described later.

  The tracking threshold update unit 2h changes the tracking threshold value to be lower when the change degree of the object calculated by the change degree calculation part 2g is equal to or greater than a predetermined value, and the change degree of the object is less than the predetermined value. Is a tracking threshold update means for executing a threshold update process for maintaining the value of the tracking threshold. When the change degree of the target is equal to or greater than a predetermined value, the tracking threshold update unit 2h changes the tracking threshold to be lower than the tracking threshold value used when tracking the target in at least the previous frame. The tracking threshold value updating unit 2h changes the tracking threshold value to be lower as the degree of change of the object increases. In the present embodiment, an appropriate value determined in advance through experiments or the like is set as the predetermined value regarding the degree of change of the object.

  Here, with reference to FIG. 4, an example of the tracking threshold updated by the tracking threshold update unit 2h according to the degree of change of the object will be described. In the graph of FIG. 4, the average value of the tracking score calculated for each predetermined frame by the tracking score calculation unit 2d is plotted. In FIG. 4, the initial value of the tracking threshold is used when the object tracking unit 2e compares the tracking score with the tracking score in the first frame and the second frame. An updated tracking threshold is used. In the following, various processes executed from the first frame to the fifth frame in the example of FIG. 4 will be described.

  As shown in FIG. 4, in the first frame, an object image as a partial region including the object detected by the object detection unit 2b is accumulated by the image accumulation unit 2f. In the first frame, the tracking score of the first frame is calculated by the tracking score calculation unit 2d according to the information amount of the feature point extracted by the feature point extraction unit 2c. Then, with the calculated tracking score value of the first frame as a reference, an initial value of the tracking threshold having a value higher than the tracking score value is set.

  Here, in FIG. 4, when the object tracking unit 2e determines whether or not the object can be tracked between the first frame and the second frame, the initial value of the tracking threshold and the tracking of the second frame are performed. Compare the score. Here, since the tracking score of the second frame is equal to or greater than the initial value of the tracking threshold, the object tracking unit 2e determines that the object can be tracked between the first frame and the second frame. The object image that can be tracked in the second frame is accumulated by the image accumulating unit 2f. Further, in the second frame, in order to set the tracking threshold for the third frame, the degree of change of the object is calculated, and threshold update processing is executed on the tracking threshold according to the degree of change. Here, as a result of comparing the object image of the first frame and the object image of the second frame stored in the image storage unit 2f by the degree-of-change calculator 2g, the object image of the first frame and the second frame are compared. It is assumed that the difference in shape information is relatively large and the apparent change of the object is predicted to be relatively large. In this case, since the value of the degree of change of the target object is relatively large, the tracking threshold update unit 2h determines that the calculated degree of change of the target object is equal to or greater than a predetermined value, and performs the second frame. The tracking threshold for the third frame is changed so as to have a value lower than the initial value of the used tracking threshold.

  In FIG. 4, when the object tracking unit 2e determines whether the object can be tracked between the second frame and the third frame, the object is tracked by the tracking threshold update unit 2h in the second frame. The tracking threshold for the third frame set according to the degree of change of the third frame is compared with the tracking score of the third frame. Here, since the tracking score of the third frame is equal to or greater than the tracking threshold for the third frame, the object tracking unit 2e determines that the object can be tracked between the second frame and the third frame. Then, the object image that can be tracked in the third frame is accumulated by the image accumulating unit 2f. In the third frame, in order to set the tracking threshold for the fourth frame, the degree of change of the object is calculated, and the threshold update process is executed on the tracking threshold according to the degree of change. Here, as a result of comparing the object image of the second frame and the object image of the third frame stored in the image storage unit 2f by the degree-of-change calculator 2g, the object images of the second frame and the third frame are compared. It is assumed that the difference in shape information is relatively large and the apparent change of the object is predicted to be relatively large. In this case, since the value of the degree of change of the target object is relatively large, the tracking threshold update unit 2h determines that the calculated degree of change of the target object is equal to or greater than a predetermined value, and performs the third frame. The tracking threshold for the fourth frame is changed to have a lower value than the used tracking threshold.

  In FIG. 4, when the object tracking unit 2e determines whether the object can be tracked between the third frame and the fourth frame, the object is tracked by the tracking threshold update unit 2h in the third frame. The tracking threshold for the fourth frame set according to the degree of change of the fourth frame is compared with the tracking score of the fourth frame. Here, since the tracking score of the fourth frame is equal to or greater than the tracking threshold for the fourth frame, the object tracking unit 2e determines that the object can be tracked between the third frame and the fourth frame. Then, the object image that can be tracked in the fourth frame is accumulated by the image accumulating unit 2f. In the fourth frame, in order to set the tracking threshold for the fifth frame, the degree of change of the object is calculated, and the threshold update process is performed on the tracking threshold according to the degree of change. Here, as a result of comparing the object image of the third frame and the object image of the fourth frame stored in the image storage unit 2f by the degree-of-change calculator 2g, the object image of the third frame and the fourth frame are compared. It is assumed that the difference in shape information is relatively small and the apparent change of the object is predicted to be relatively small. In this case, since the value of the degree of change of the object is a relatively small value, the tracking threshold update unit 2h determines that the calculated degree of change of the object is less than the predetermined value, and in the fourth frame The tracking threshold used is kept unchanged and set as the tracking threshold for the fifth frame.

  In FIG. 4, when the object tracking unit 2e determines whether the object can be tracked between the fourth frame and the fifth frame, the object is detected by the tracking threshold update unit 2h in the fourth frame. The tracking threshold for the fifth frame set in accordance with the degree of change in is compared with the tracking score of the fifth frame. Here, since the tracking score of the fifth frame is less than the tracking threshold for the fifth frame, the object tracking unit 2e determines that the object cannot be tracked between the fourth frame and the fifth frame and is lost. .

  Returning to FIG. 1, the jump determination unit 2 i is a jump determination unit that performs jump detection on the object image tracked by the target tracking unit 2 e and determines whether or not the target has jumped. For example, the popping-out determination unit 2i includes a feature image included in the object image with respect to the object image among a plurality of images (for example, the first image and the second image) that move back and forth in time series. The amount of change of the feature point based on the amount of movement of the point, the direction of movement, etc. is calculated using a method known in the field. Then, when the amount of change calculated for the target image is equal to or greater than a predetermined pop-up detection threshold, the pop-out determination unit 2i determines that there is a target pop-out, and the predetermined pop-up detection threshold If it is less than that, it is determined that there is no object jumping out. In the present embodiment, the pop-up detection threshold value is more than the tracking threshold value so that it can be clearly distinguished from the tracking threshold value for tracking to determine whether or not the object can be tracked. It is assumed that a large value is set. The pop-up detection threshold is set to an appropriate value determined in advance through experiments or the like.

  In the present embodiment, the jumping out of the object means a change of the dangerous object in the traffic environment. Examples of such a change in dangerous objects include a change in which a pedestrian or a bicycle suddenly changes movement and comes out on the road, or a side-by-side vehicle or motorcycle suddenly changes lanes and enters the own lane. A change in which the preceding vehicle suddenly starts a U-turn, a change in which the preceding vehicle suddenly starts a right / left turn to enter a store on the road, and the like, but is not limited thereto.

  The alerting unit 2j alerts the driver of the host vehicle by outputting a voice through the speaker 3 when the target determining unit 2i determines that the target has jumped out. It is an arousal means For example, the alerting unit 2j outputs audio information, a buzzer sound, and the like that prompt the driver to pay attention in a direction in which the object is predicted to fly out.

  The vehicle control unit 2k controls the behavior of the vehicle via the actuator 4 so that the host vehicle avoids the target when the jump determination unit 2i determines that the target jumps out. Vehicle control means for performing control. For example, the vehicle control unit 2k performs steering control, deceleration control, and the like via the actuator 4 so that the host vehicle can avoid the object.

  Subsequently, processing executed in the above-described object recognition device 2 will be described with reference to FIGS. 5 and 6. In the following processing, a pedestrian will be described as an example of an object existing around the vehicle, but the present invention is not limited to this.

  First, an example of a process for setting an initial value of the tracking threshold executed by the object recognition device 2 will be described with reference to FIG.

  As shown in FIG. 5, the image acquisition unit 2a acquires the image data at time t = 0 input from the camera 1 as an image obtained by imaging the periphery of the host vehicle (step ST11).

  And the target object detection part 2b detects a pedestrian as a target object which exists in the circumference | surroundings of the own vehicle from the image acquired by the image acquisition part 2a by step S11 (step ST12).

  And the feature point extraction part 2c extracts the pedestrian's feature point as a target object detected by the target object detection part 2b in step ST12 (step ST13). For example, if an image as shown in FIG. 2 is acquired by the image acquisition unit 2a in step ST11, in step ST13, the feature point extraction unit 2c, as shown in FIG. To extract.

  Then, the image accumulating unit 2f cuts out a partial area including the pedestrian as the object detected by the object detecting unit 2b in step ST12 from the image acquired by the image acquiring unit 2a in step ST11, and extracts the object image. (Step ST14). For example, if an image as shown in FIG. 2 is acquired by the image acquisition unit 2a in step ST11, in step ST14, the image storage unit 2f acquires the object image in the image of FIG. 2 in step ST11. The unit 2a stores the image in association with the time when the image is acquired.

  Then, the object recognition device 2 calculates a tracking score at time t = 0 according to the information amount of the feature points extracted by the feature point extraction unit 2c in step ST13, and uses the calculated tracking score value as a reference. Then, an initial value of the tracking threshold having a value higher than the value of the tracking score is set (step ST15). Thereafter, this process is terminated.

  Next, an example of the object tracking process and the tracking threshold update process executed by the object recognition device 2 will be described with reference to FIG. The process shown in FIG. 6 is repeatedly executed every time images are sequentially acquired by the image acquisition unit 2a after the tracking threshold initial value setting process shown in FIG. 5 is completed.

  As shown in FIG. 6, the image acquisition unit 2a acquires the image data of the next frame input from the camera 1 as an image obtained by imaging the periphery of the host vehicle (step ST21). For example, assuming that the first image is acquired at the first time, in step ST21, the image acquisition unit 2a selects the second image acquired at the second time which is a time after the first time. And obtained as image data of the next frame.

  And the target object detection part 2b detects a pedestrian as a target object which exists in the circumference | surroundings of the own vehicle from the image acquired by the image acquisition part 2a by step ST21 (step ST22).

  And the feature point extraction part 2c extracts the pedestrian's feature point as a target object detected by the target object detection part 2b in step ST22 (step ST23). For example, if an image as shown in FIG. 3 is acquired by the image acquisition unit 2a in step ST22, in step ST23, the feature point extraction unit 2c, as shown in FIG. To extract.

  Then, the tracking score calculation unit 2d reads the feature points regarding the previous object stored in the image storage unit 2f, and calculates the tracking score (step ST24). Specifically, in step ST24, the tracking score calculation unit 2d saves the object stored at a time earlier than the time when the image of the next frame was acquired in step ST21 from the object image accumulated in the image accumulation unit 2f. An object image is searched and feature points included in the object image are read out. Then, the tracking score calculation unit 2d compares the feature points relating to the previous object stored in the image storage unit 2f with the feature points extracted by the feature point extraction unit 2c in step ST23, and the feature points are compared. A tracking score that is lower as the change in the amount and direction of movement of the point is larger is calculated.

  For example, if the image shown in FIG. 2 is the first image and the image shown in FIG. 3 is the second image, in step ST24, the tracking score calculation unit 2d stores FIG. 2 accumulated in step ST14 of FIG. The feature points 1 to 5 in the object image of FIG. 3 and the feature points 1 to 5 in the object image of FIG. 3 extracted in step ST23 are compared for each identical feature point, and a tracking score for each feature point is calculated. . Then, the tracking score calculation unit 2d outputs the calculated average value of the tracking score for each feature point as a final tracking score used in the next process.

  Then, the object tracking unit 2e uses the tracking score calculated by the tracking score calculation unit 2d in step ST24 and the tracking threshold value for tracking set in step ST15 of FIG. 5 (for example, as shown in FIG. 4). (Initial value of the tracking threshold value) is compared with each other to determine whether or not the object detected by the object detection unit 2b in step ST22 is the same object as the previous time based on the tracking threshold value ( Step ST25). Specifically, in step ST25, when the tracking score is equal to or higher than the tracking threshold (step ST25: Yes), the object tracking unit 2e predicts that the object is the same as the previous object and recognizes it as the tracking object. It is determined that the target object has been tracked. In this case, the process proceeds to the next step ST26. On the other hand, in step ST25, when the tracking score is less than the tracking threshold (step ST25: No), the target tracking unit 2e predicts that the target is not the same target as the previous time and recognizes it as the tracking target It is determined that the object was lost because it could not be tracked. In this case, this process ends.

  If it is predicted in step ST25 that the object is the same as the previous object (step ST25: Yes), the image accumulating unit 2f determines a partial region including a pedestrian as a tracked object in step ST21. Then, it is cut out from the image acquired by the image acquisition unit 2a and accumulated as the object image (step ST26). For example, if an image as shown in FIG. 3 is acquired by the image acquisition unit 2a in step ST21, in step ST26, the image storage unit 2f acquires an object image in the image of FIG. 3 in step ST21. The unit 2a stores the image in association with the time when the image is acquired.

  Then, the degree-of-change calculator 2g calculates the degree of change of the object based on the object image stored in the image storage unit 2f (step ST27). For example, if the target image stored in the image storage unit 2f is the target image in FIG. 2 and the target image in FIG. 3, the degree-of-change calculation unit 2g in step ST27 determines the target image in the target image in FIG. The shape information of the object is compared with the shape information of the object in the object image of FIG. 3, and from the change of the shape information of the object between these object images, the appearance of the object is apparent Calculate the degree of change indicating the change in. The degree-of-change calculation unit 2g predicts that the apparent change of the target object increases as the difference in shape information between the target object images to be compared increases, so that the value of the degree of change of the target object increases. calculate. On the other hand, the degree-of-change calculation unit 2g predicts that the apparent change of the object becomes smaller as the difference in shape information between the object images to be compared becomes smaller, and the value of the degree of change of the object becomes smaller. Calculate as follows.

  Then, the tracking threshold value update unit 2h changes the tracking threshold value to be lower when the degree of change of the object calculated by the degree of change calculation unit 2g in step ST27 is a predetermined value or more, and changes the object. If the degree is less than the predetermined value, threshold update processing for maintaining the value of the tracking threshold is executed (step ST28). As shown in FIG. 4 described above, the tracking threshold update unit 2h compares the tracking threshold value used when tracking the object in at least the previous frame when the degree of change of the object is a predetermined value or more. Change to lower. The tracking threshold value updating unit 2h changes the tracking threshold value to be lower as the degree of change of the object increases.

  The tracking threshold set in step ST28 is used when performing the object tracking process in step ST25 as a tracking threshold for the next frame and thereafter. For example, in step ST25 of the next frame, the object tracking unit 2e detects the tracking score calculated based on the feature point of the image of the next frame by the tracking score calculation unit 2d in step ST24, and in step ST28 in the previous frame. In step ST22 based on the tracking threshold by comparing with a tracking threshold for tracking that has been changed or maintained (for example, a tracking threshold updated according to the degree of change of the object as shown in FIG. 4). It is determined whether or not the object detected by the object detection unit 2b is the same object as the previous object.

  As described above, according to the object recognition device 2 according to the present embodiment, the tracking threshold for tracking an object, which was a fixed value in the conventional technology, is appropriately set according to the degree of change of the object for each frame. It can be updated to be a correct value. Thereby, according to the target object recognition apparatus 2 according to the present embodiment, if there is a sudden change in the moving amount and moving direction of the target object that the tracking threshold value having a preset fixed value is not assumed, It is possible to reduce the occurrence of a situation in which tracking of the user is likely to fail. As a result, according to the object recognition device 2 according to the present embodiment, the object is detected even when there is a sudden change in the movement amount and the movement direction with respect to the object recognized as the tracking object. The object can be suitably tracked by reducing the possibility of being lost.

  In the present embodiment, after the process of step ST28, the jump determination unit 2i performs a jump detection on the target image tracked by the target tracking unit 2e in step ST25, and the target jumps out. You may determine the presence or absence of. In addition, after the jump determination process, the alerting unit 2j outputs a sound through the speaker 3 when the jump determination unit 2i determines that the target has jumped out, and thereby the host vehicle The driver may be alerted. In addition, after the jump determination processing, the vehicle control unit 2k controls the behavior of the vehicle via the actuator 4 when the jump determination unit 2i determines that the target has jumped out. Vehicle control may be performed so that the host vehicle avoids the object.

DESCRIPTION OF SYMBOLS 1 Camera 2 Object recognition apparatus 2a Image acquisition part 2b Object detection part 2c Feature point extraction part 2d Tracking score calculation part 2e Object tracking part 2f Image storage part 2g Change degree calculation part 2h Tracking threshold update part 2i Pop-out determination part 2j Alerting part 2k Vehicle control part 3 Speaker 4 Actuator

Claims (1)

An image acquisition unit for acquiring an image obtained by imaging the periphery of the host vehicle;
An object detection unit for detecting an object existing around the host vehicle from the image acquired by the image acquisition unit;
A feature point extraction unit for extracting feature points of the object detected by the object detection unit;
Comparing a plurality of images that are sequentially acquired by the image acquisition unit and that move back and forth in time series with respect to an object image set in the image as a partial region including the object detected by the object detection unit. Then, a tracking score calculation unit that calculates a tracking score that becomes a lower value as the movement amount and movement direction of the feature point extracted by the feature point extraction unit is larger, and
By comparing the tracking score calculated by the tracking score calculation unit with a tracking threshold for tracking, an object tracking unit that tracks an object whose tracking score is equal to or higher than the tracking threshold;
An image accumulating unit for accumulating an object image of the object that can be tracked by the object tracking unit;
A degree-of-change calculator that calculates the degree of change of the object based on the object image stored in the image storage unit;
When the degree of change calculated by the degree-of-change calculator is equal to or greater than a predetermined value, the tracking threshold value is changed to be low, and when the degree of change is less than a predetermined value, the value of the tracking threshold is changed. A tracking threshold update unit that executes a threshold update process to maintain;
An object recognition apparatus comprising:

Images