JP2013030129A - Moving object detection device, moving object detection system, computer program, and moving object detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving object detection device, a moving object detection system, a computer program, and a moving object detection method that can automatically gather a non-vehicle pattern without any person's help and also detect a moving object such as a vehicle with high precision using the gathered non-vehicle pattern.SOLUTION: A negative example feature quantity calculation part 108 calculates a feature quantity based upon pixel values in a corresponding area of a moving object candidate area specified on a picked-up image of time (t) on a picked-up image of time (t+Ts) which is a predetermined time Ts after time (t). A negative example similarity calculation part 109 calculates similarity between a feature quantity of the moving body candidate area and a feature quantity of the corresponding area. A negative example registration part 110 registers the moving object candidate area as a non-vehicle pattern when the calculated similarity is equal to or larger than a predetermined threshold S.

Description

  The present invention provides a moving body detection device that detects a moving body based on a captured image obtained by imaging with an imaging device, a moving body detection system including the moving body detection device, and the moving body detection device. The present invention relates to a computer program and a moving object detection method.

  A system in which a roadside infrastructure device and an in-vehicle device exchange information by wired or wireless road-to-vehicle communication means and each device realizes a function that cannot be realized independently has been studied. In particular, there is a study on a road-vehicle cooperative safe driving support system that suppresses an accident by providing detection information that detects a vehicle or a pedestrian present in a blind spot from the vehicle driver to the vehicle side from the infrastructure side. It is being advanced.

  As a main technique for realizing such a system, a sensor that detects vehicle position information, vehicle type information such as an automobile, a two-wheeled vehicle, or a pedestrian, or a moving speed or a moving direction of the vehicle or the like is necessary. Considering the wide measurement range, product life, cost or performance balance, image processing type sensors are promising. In vehicle detection, which is the main function of this image processing type sensor (hereinafter referred to as “vehicle sensor”), high-precision vehicle detection must be realized at any measurement point where an environmental change exists.

  For example, as a method of realizing highly accurate vehicle detection, a vehicle is detected as an object from a road image, and through vehicle / non-vehicle pattern sample learning represented by a support vector machine (SVM) or boosting (Boosting). A vehicle tracking device that uses a pattern recognition technique for generating a criterion (boundary criterion) for a vehicle / non-vehicle pattern is disclosed (see Patent Document 1).

JP 2010-92248 A

  However, assuming that the vehicle sensors are installed at various points, it is extremely difficult to prepare in advance all the various non-vehicle patterns that can occur at each point. Therefore, a method is conceivable in which non-vehicle patterns that may occur at each point are collected for each installation point of the vehicle sensor, and the determination criteria of the vehicle sensor are appropriately corrected or improved based on the collected non-vehicle patterns. However, it is not realistic to cause the operator to correct the determination criteria of the vehicle sensor for each vehicle sensor installed in each place because it requires a great deal of labor and a huge cost.

  The present invention has been made in view of such circumstances, and automatically collects non-vehicle patterns (moving body determination information) without human intervention and moves a vehicle or the like using the collected non-vehicle patterns. It is an object to provide a moving body detection apparatus capable of detecting a body with high accuracy, a moving body detection system including the moving body detection apparatus, a computer program for realizing the moving body detection apparatus, and a moving body detection method. And

  The moving body detection device according to the first aspect of the present invention detects a moving body based on moving body determination information registered in advance to determine the moving body and a captured image obtained by imaging with the imaging device. In the detection device, a specifying unit that specifies a moving object candidate area that is a pixel area indicating a moving object candidate based on a first captured image obtained at the first imaging time point, and a feature based on a pixel value of the moving object candidate area A first feature amount calculating means for calculating an amount; and a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second imaging time point after a first predetermined time has elapsed from the first imaging time point. A second feature quantity calculating means for calculating a feature quantity based on the similarity, a similarity calculating means for calculating a similarity between the feature quantities calculated by the first feature quantity calculating means and the second feature quantity calculating means, and the similarity calculating means. The similarity calculated in step 1 is a predetermined first threshold value. If it is above, characterized in that it comprises a registration means for registering the feature amount of the moving object candidate area as a moving object determination information.

  In the moving body detection apparatus according to a second aspect of the present invention, in the first aspect, the second feature amount calculating means is obtained between the second imaging time point and a third imaging time point after a second predetermined time has elapsed. The feature amount of each corresponding region on the plurality of captured images obtained is calculated, and the similarity calculation unit calculates the similarity between the feature amount of the moving object candidate region and the feature amount of each corresponding region. When the number of corresponding regions whose similarity calculated by the similarity calculation unit is equal to or greater than the first threshold is equal to or greater than a predetermined value, the registration unit calculates the feature amount of the moving object candidate region. It is configured to be registered as moving body determination information.

  The moving body detection device according to a third aspect of the present invention is the moving body detection device according to the first aspect or the second aspect, wherein the imaging point of the moving body candidate region specified based on a captured image obtained at an arbitrary imaging point by the specifying unit. A tracking unit that tracks a position on a captured image obtained at a subsequent imaging time point, and when the tracking unit performs tracking for a time difference greater than or equal to a time difference between the second imaging time point and the first imaging time point, the movement And a discarding unit that discards the feature amount of the body candidate region.

  The mobile object detection apparatus according to a fourth aspect of the present invention is the mobile object detection device according to any one of the first aspect to the third aspect, wherein the specifying unit specifies one or a plurality of mobile object candidate regions, The registration unit registers, for each moving object candidate area, the feature amount of the moving object candidate area as moving object determination information in association with an area having a position and size on a captured image equivalent to the moving object candidate area. It is comprised by these.

  The moving body detection apparatus according to a fifth aspect of the present invention is the mobile body detection device according to the fourth aspect, wherein the specifying means specifies a moving body candidate region based on a captured image obtained at an arbitrary imaging time point, The first feature quantity calculating means calculates the feature quantity of the moving object candidate area, and the similarity calculating means calculates the feature quantity of the moving object candidate area and the area associated with the moving object candidate area. If the moving object determination information having a calculated similarity equal to or greater than a predetermined second threshold exists, the moving object candidate region is calculated. It has a detection means which detects as a non-moving body, It is characterized by the above-mentioned.

  The mobile body detection device according to a sixth aspect of the present invention is the mobile body detection device according to the fifth aspect, wherein the detection means includes the mobile body candidate when there is no mobile body determination information whose calculated similarity is equal to or greater than a predetermined second threshold. The region is configured to be detected as a moving body.

  A mobile body detection system according to a seventh aspect of the present invention includes the mobile body detection apparatus according to the first aspect described above, and an imaging device that outputs a captured image to the mobile body detection device. .

  A computer program according to an eighth aspect of the present invention causes a computer to detect a moving body based on moving body determination information registered in advance to determine a moving body and a captured image obtained by imaging with an imaging apparatus. In the computer program, the step of identifying a moving object candidate area which is a pixel area indicating a moving object candidate based on the first captured image obtained at the first imaging time, and the pixel value of the moving object candidate area And a feature based on a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second imaging time point after the first predetermined time has elapsed from the first imaging time point. A step of calculating a quantity; a step of calculating a similarity of the calculated feature quantity; and the calculated similarity is equal to or greater than a predetermined first threshold, the moving object candidate region Characterized in that and a step of registering the feature amount as the moving object determination information.

  The moving body detection method according to the ninth aspect of the present invention is a moving body that detects a moving body based on moving body determination information registered in advance to determine a moving body and a captured image obtained by imaging with an imaging device. In the moving object detection method by the detection device, a step of specifying a moving object candidate area that is a pixel area indicating a moving object candidate based on a first captured image obtained at the first imaging time point; and A step of calculating a feature amount based on a pixel value; and a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second captured time after a first predetermined time has elapsed from the first captured time. A step of calculating a feature amount based on the step, a step of calculating a similarity of the calculated feature amount, and the calculated similarity is equal to or greater than a predetermined first threshold, Register as judgment information A step that, characterized in that it comprises the steps of detecting a moving object using a moving object determination information registered.

  In the first aspect, the eighth aspect, and the ninth aspect, a moving object candidate area that is a pixel area indicating a moving object candidate is specified based on the first captured image obtained at the first imaging time point. The moving body is, for example, a vehicle such as an automobile or a two-wheeled vehicle. The moving object candidate area may be specified by using a pattern recognition technique or based on information such as a change in a pixel in the image, for example, an edge point. A feature amount based on the pixel value of the identified moving object candidate region is calculated. For example, the feature amount can be expressed as a 256-dimensional vector by converting the moving object candidate region into a predetermined size (for example, a 16 × 16 pixel block). Further, the feature amount is not limited to the pixel value, and an HOG (Histograms of Oriented Gradients) feature amount based on the gradient of each pixel in the pixel block may be used.

  And the feature-value based on the pixel value of the corresponding | compatible area | region of a moving body candidate area | region in the 2nd captured image acquired at the 2nd imaging time point after 1st predetermined time progress from a 1st imaging time point is calculated. The feature amount of the corresponding region can be obtained by the same method as the feature amount of the moving object candidate region. The first predetermined time is, for example, a time longer than a time that is normally considered to be a stop time when the vehicle stops due to a signal or the like, and can be, for example, about 120 seconds to 150 seconds. The similarity between the feature amount of the moving object candidate region and the feature amount of the corresponding region is calculated. For example, when the feature quantity is a 256-dimensional vector, the similarity can be obtained by the magnitude of the Euclidean distance in the 256-dimensional Euclidean space of the points represented by both vectors (the smaller the similarity, the greater the similarity). It is also possible to use a cosine similarity that is a cosine (cos θ) of an angle θ formed by both vectors.

  When the calculated similarity is equal to or greater than a predetermined first threshold, the feature amount of the moving object candidate region is registered as moving object determination information (for example, a non-vehicle pattern). That is, it is obtained by imaging at the second imaging time point after the elapse of a feature amount of the moving object candidate region on the first captured image obtained by imaging at the first imaging time point and a time considered to be the stop time of the vehicle. Since the feature amount of the corresponding region on the second captured image is similar, the moving object candidate region is registered not as a vehicle image but as a non-vehicle pattern. It should be noted that the feature quantity, coordinates (x, y), size (height and width), etc. of the moving object candidate area are registered as the non-vehicle pattern (moving object determination information). By the above-described automatic collection process, it is possible to automatically collect non-vehicle patterns without manual intervention by simply repeating imaging. And a moving body (for example, vehicle etc.) can be detected with high precision by detecting a moving body using registered moving body judging information (non-vehicle pattern).

  In the second mode, the feature amounts of the corresponding areas on the plurality of captured images obtained from the second imaging time point to the third imaging time point after the second predetermined time (for example, about 30 seconds) has elapsed. And the similarity between the feature amount of the moving object candidate region and the feature amount of each corresponding region is calculated. Then, when the number of corresponding areas in which the calculated similarity is equal to or greater than the first threshold is equal to or greater than a predetermined value (for example, 2), the feature amount of the moving object candidate area is used as the moving object determination information (for example, non-vehicle pattern). sign up. The predetermined value is the number of times that a vehicle having a feature value similar to the feature value of the moving object candidate region is accidentally present at the same position as the moving object candidate region between the second imaging time point and the third imaging time point (for example, 1). As a result, it is possible to prevent accidental registration of an actual vehicle feature quantity existing at the same position as the moving object candidate region as a non-vehicle pattern between the second imaging time point and the third imaging time point. Can often collect non-vehicle patterns.

  In the third aspect, the position on the captured image obtained at the imaging time after the imaging time of the moving object candidate region specified based on the captured image obtained at the arbitrary imaging time is tracked. For the tracking process, for example, a tracking technique such as pattern matching or particle filter can be used. When tracking is performed for a time difference between the second imaging time point and the first imaging time point or more, the feature amount of the moving object candidate region is discarded. That is, when the moving object candidate area is tracked for a time difference between the second imaging time point and the first imaging time point or more, the moving object candidate area is an image of an actual vehicle. Since it can be determined, it is discarded without being registered as a non-vehicle pattern. Accordingly, it is possible to prevent the original characteristic amount of the vehicle from being erroneously registered as a non-vehicle pattern, and to improve the accuracy of the collected non-vehicle pattern.

  In the fourth aspect, for each moving object candidate area, the feature amount of the moving object candidate area is registered as moving object determination information in association with an area having a position and size on the captured image equivalent to the moving object candidate area. To do. That is, when registering the feature amount or the like of the moving object candidate area as the moving object determination information (non-vehicle pattern), it is registered for each moving object candidate area. That is, the determination of the vehicle / non-vehicle when performing the moving object detection is performed only in the registered area (the coordinates and size of the moving object candidate area). As typical non-vehicle patterns, shadows of buildings, trees, etc. exist locally in almost the same area, so non-vehicles can be determined without determining whether or not the entire captured image is a non-vehicle pattern. By determining only the area registered as a pattern, sufficient vehicle / non-vehicle determination can be made, and processing time can be shortened compared to the case where the entire captured image is processed, thereby realizing high-speed processing. Can do. Further, even if the vehicle / non-vehicle determination is wrong (even if an internal error occurs), the region that has been registered as the non-vehicle pattern has an effect, so that robustness against the internal error can be obtained.

  In the fifth aspect, the moving object candidate area is identified based on the captured image obtained at an arbitrary imaging time point, and the feature amount of the identified moving object candidate area is calculated. The similarity between the feature amount of the moving object candidate region and the feature amount of the moving object determination information (for example, non-vehicle pattern) registered in the region associated with the moving object candidate region is calculated, and the calculated similarity is predetermined. If there is moving object determination information (non-vehicle pattern) that is equal to or greater than the second threshold value, the moving object candidate area is detected as a non-moving object. By using the moving body determination information (non-vehicle pattern) obtained by the automatic collection process, it is possible to prevent a non-moving body from being erroneously detected as a moving body and to detect the moving body with high accuracy. .

  In the sixth aspect, when there is no moving object determination information whose calculated similarity is equal to or greater than the predetermined second threshold, the moving object candidate region is detected as a moving object. By using the moving body determination information (non-vehicle pattern) obtained by the automatic collection process, the moving body can be detected with high accuracy.

  In the seventh aspect, the non-vehicle pattern (moving body determination information) is automatically collected without human intervention, and the moving body such as a vehicle is detected with high accuracy using the collected non-vehicle pattern. It is possible to realize a moving body detection system capable of

  According to the present invention, it is possible to automatically collect non-vehicle patterns (moving body determination information) without human intervention and to detect a moving body such as a vehicle with high accuracy using the collected non-vehicle patterns. .

It is a block diagram which shows an example of a structure of the mobile body detection system of this Embodiment. It is a schematic diagram which shows an example of a mobile body candidate area | region. It is a time chart which illustrates the relationship between a vehicle detection process and a vehicle tracking process. It is explanatory drawing which shows the relationship between a mobile body candidate area | region and a corresponding | compatible area | region. It is explanatory drawing which shows an example of the registration method of a non-vehicle pattern. It is explanatory drawing which shows the other example of the registration method of a non-vehicle pattern. It is explanatory drawing which shows an example of the registered non-vehicle pattern. It is a flowchart which shows the process sequence of the automatic collection of the negative example by the vehicle detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the automatic learning result of the vehicle detection apparatus of this Embodiment. It is explanatory drawing which shows the other example of the automatic learning result of the vehicle detection apparatus of this Embodiment.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments thereof. FIG. 1 is a block diagram showing an example of the configuration of the moving object detection system of the present embodiment. The moving body detection system includes a vehicle detection device 100, an imaging device 200, and the like as a moving body detection device. In the example of FIG. 1, one imaging device 200 is illustrated, but a plurality of imaging devices 200 may be provided. That is, the imaging device 200 can be installed at each of a plurality of points on the road, and a captured image captured by the imaging device 200 can be transmitted (output) to the vehicle detection device 100. Moreover, in this Embodiment, although a moving body is vehicles, such as a motor vehicle or a two-wheeled vehicle, for example, moving bodies, such as a pedestrian, can also be included.

  The vehicle detection device 100 includes an image input unit 101, a vehicle detection unit 102, a vehicle tracking unit 112, a negative example collection unit 107, a negative example storage unit 111, and the like. In the present embodiment, a negative example is a false detection pattern detected by determining that a non-vehicle such as a road surface is a vehicle based on a captured image obtained by imaging with the imaging device 200. Say. Further, the vehicle pattern includes information such as a feature amount of an area registered by determining that the vehicle is a vehicle, coordinates (x, y) on the captured image of the area, and the size (height, width) of the area. . Further, the non-vehicle pattern is information such as a feature amount of a region registered by determining that the vehicle is a non-vehicle, coordinates (x, y) on the captured image of the region, and the size (height, width) of the region. including.

  The vehicle detection unit 102 includes a vehicle candidate area specifying unit 103, a feature amount calculation unit 104, a vehicle similarity calculation unit 105, a vehicle determination unit 106, and the like. The negative example collection unit 107 includes a negative example feature amount calculation unit 108, a negative example similarity calculation unit 109, a negative example registration unit 110, and the like.

  The vehicle detection apparatus 100 according to the present embodiment automatically recognizes that the vehicle detection unit 102 has erroneously detected a vehicle as a non-vehicle pattern such as a road surface and collects it as a negative example. Automatically avoiding a situation where a certain pattern is mistakenly collected as a non-vehicle pattern, correcting the vehicle determination standard of the vehicle detection unit 102 based on the collected negative example, and improving the determination accuracy, and the vehicle detection unit When the vehicle detection process at 102 and the vehicle tracking process at the vehicle tracking unit 112 are performed in parallel, an automatic learning function that satisfies maintaining the real-time property of the entire process is realized. Note that the automatic learning function is a function that reduces errors in determination of vehicle detection by automatically correcting the determination criterion of vehicle detection without involving human judgment.

  The image input unit 101 includes an image memory (not shown) and the like, and stores the captured image (image data) input from the imaging apparatus 200 in the image memory in synchronization with each frame. One frame is, for example, 100 ms, but is not limited to this. The captured image stored in the image memory is read by the vehicle detection unit 102.

  The vehicle candidate area specifying unit 103 has a function as specifying means for specifying the moving object candidate area based on the captured image. For the moving object candidate region, it is possible to employ appropriate processing that is normally performed in image processing. For example, an edge point in a captured image is detected, and an edge grouping process for grouping adjacent edge points in an edge image composed of the detected edge points is performed, and a rectangular area surrounding the edge group is set as a moving object candidate area. Can be identified. Note that a plurality of moving object candidate regions may be specified in a captured image obtained at an arbitrary imaging time point.

  The feature amount calculation unit 104 has a function as a feature amount calculation unit that calculates the feature amount of the moving object candidate region. Regardless of the size of the moving object candidate area on the captured image, the feature value calculating unit 104 resizes the moving object candidate area to a predetermined size (for example, 16 × 16 pixels) and sets 256 as the feature value. Calculate the dimension vector. The feature amount is a 256-dimensional vector having 256 pixel values of 16 × 16 pixels. Note that the feature amount is not limited to the pixel value, and other feature amounts may be used. For example, the gradient of each pixel within 16 × 16 pixels can be obtained, and HOG (Histograms of Oriented Gradients) feature quantities having a gradient direction quantization (for example, 9 directions) and frequency distribution can be used. . Note that the size of the pixel block when calculating the feature amount is not limited to 16 × 16 pixels.

  In the vehicle detection process, the vehicle similarity calculation unit 105 calculates the similarity between the feature amount of the identified moving object candidate region (vehicle candidate region) and the negative example feature amount stored (registered) in the negative example storage unit 111. Is calculated. For example, when the feature quantity is a 256-dimensional vector, the similarity can be obtained by the magnitude of the Euclidean distance in the 256-dimensional Euclidean space of the points represented by both vectors (the smaller the similarity, the greater the similarity). It is also possible to use a cosine similarity that is a cosine (cos θ) of an angle θ formed by both vectors.

  If the similarity between the feature amount of the moving object candidate region and the feature amount of the negative example (a pattern in which a non-vehicle is erroneously detected as a vehicle) is equal to or greater than a predetermined threshold, the vehicle determination unit 106 determines that the moving object candidate region is It is determined that the vehicle is a non-vehicle, and if the similarity is less than a predetermined threshold value, the vehicle is determined.

  The vehicle detection unit 102 outputs the moving object candidate area detected as a vehicle to the vehicle tracking unit 112 as a pattern determined as “vehicle”. In addition, the vehicle detection unit 102 outputs the pattern detected as the vehicle to the negative example collection unit 107 as “false detection pattern candidate” (negative example candidate). In this case, the vehicle detection unit 102 outputs to the negative example collection unit 107 the region ID of the moving object candidate region as the “false detection pattern candidate”, the region coordinates (x, y), the size of the region (high , Width), and the feature amount of the moving object candidate area. Hereinafter, the “false detection pattern candidate” output from the vehicle detection unit 102 to the negative example collection unit 107 is referred to as a pattern A.

  FIG. 2 is a schematic diagram illustrating an example of a moving object candidate region. A moving object candidate area determined as a vehicle on a captured image captured at an arbitrary imaging time point is an area ID, upper left coordinates (x, y) of the area, and the size of the area as a pattern A (false detection pattern candidate). The height (height h, width w) and feature quantity (for example, a 256-dimensional vector composed of 256 pixel values) are specified.

  The vehicle tracking unit 112 serves as a tracking unit that tracks the position of the moving object candidate area specified based on the captured image obtained at an arbitrary imaging time point on the captured image obtained at the subsequent imaging time point. It has a function. More specifically, the vehicle tracking unit 112 tracks the moving object candidate area detected as a vehicle on the captured image at an arbitrary time point by the vehicle detection unit 102 on the captured image captured after the arbitrary time point. . For the tracking process, for example, a tracking technique such as pattern matching or particle filter can be used.

  FIG. 3 is a time chart illustrating the relationship between the vehicle detection process and the vehicle tracking process. As shown in FIG. 3, when a vehicle is detected at time t (frame at time t), a moving object candidate region (vehicle region) representing the vehicle is displayed on a captured image at time (frame) after time t. Track with. If another vehicle is detected at time t + 1 (frame at time t + 1), a moving object candidate region (vehicle region) representing the vehicle is tracked on a captured image at time (frame) after time t + 1. . The same applies thereafter. The tracking process is continued until the detected vehicle does not exist on the captured image.

  The negative example feature value calculation unit 108 has a function as a feature value calculation unit that calculates a feature value based on the pixel value of the corresponding region of the moving object candidate region. In the present embodiment, the corresponding area is an area equivalent to the moving object candidate area. For example, if there is a moving object candidate area (pattern A) detected as a vehicle on the captured image at time t (first imaging time point), the equivalent corresponding area is from time t (first imaging time point). The position and size on the captured image (on the second captured image) obtained at the time (t + Ts) after the predetermined time Ts (second captured time point) is the same area as the moving object candidate area. The corresponding region is the same region (region having the same coordinates and size) as the moving object candidate region (pattern A), but it is not necessarily the same region. For example, all or part of the moving object candidate region It may be a nearby area, a larger area than the moving object candidate area, or a smaller area. The regions need not be the same if accuracy sufficient to calculate the similarity is obtained.

  The predetermined time Ts is, for example, a time longer than a time that is normally considered to be a stop time when the vehicle stops due to a signal or the like, and can be, for example, about 120 seconds to 150 seconds.

  FIG. 4 is an explanatory diagram showing the relationship between the moving object candidate area and the corresponding area. As shown in FIG. 4, when a moving object candidate region (pattern A) having a region ID = k, coordinates (x, y), height h, and width w is specified at time t, the corresponding region is, for example, This is an area having the same coordinates (x, y), height h, and width w as the moving object candidate area on the captured image at time t + Ts.

  More specifically, if there is a moving object candidate area (pattern A) detected as a vehicle on the captured image at time t (first imaging time point), the negative example feature value calculation unit 108 determines that the time t ( The position and size on the captured image (on the second captured image) obtained at the time (t + Ts) after the first predetermined time Ts has elapsed from the first captured time point (on the second captured image) are equivalent to the moving object candidate region. The feature amount based on the pixel value of the corresponding region is calculated.

  The negative example similarity calculation unit 109 has a function as a similarity calculation unit that calculates the similarity of the above-described feature amount. For example, when the feature quantity is a 256-dimensional vector, the similarity can be obtained by the magnitude of the Euclidean distance in the 256-dimensional Euclidean space of the points represented by both vectors (the smaller the similarity, the greater the similarity). It is also possible to use a cosine similarity that is a cosine (cos θ) of an angle θ formed by both vectors.

  When the similarity calculated by the negative example similarity calculation unit 109 is equal to or greater than a predetermined threshold S (first threshold), the negative example registration unit 110 sets the feature amount of the moving object candidate region (pattern A) as a negative example, that is, It is registered as a non-vehicle pattern (moving body determination information). That is, the feature amount of the moving object candidate region on the captured image obtained by imaging at time t (first imaging time point) and time t + Ts (second imaging time point) after elapse of a time considered to be the stop time of the vehicle. Since the feature amount of the corresponding area on the captured image obtained by imaging in (1) is similar, the moving object candidate area is registered as a non-vehicle pattern, not an image of the vehicle.

  It should be noted that the negative example, ie, the non-vehicle pattern (moving body determination information) is registered with the feature amount, coordinates (x, y), size (height and width), etc. of the moving body candidate region (pattern A). is there. By the above-described automatic collection process, it is possible to automatically collect non-vehicle patterns without manual intervention by simply repeating imaging. And a moving body (for example, vehicle etc.) can be detected with high precision by detecting a moving body using registered moving body judging information (non-vehicle pattern).

  Moreover, registration of a non-vehicle pattern (negative example) can also be performed with the following method. FIG. 5 is an explanatory diagram showing an example of a non-vehicle pattern registration method. As shown in FIG. 5, a plurality of times (Tu−Ts) obtained from time t + Ts (second imaging time point) to time t + Tu (third imaging time point) after elapse of a predetermined time (for example, about 30 seconds). The feature amount of each corresponding region on the captured image is calculated, and the similarity between the feature amount of the pattern A (moving body candidate region) and the feature amount of each corresponding region when the vehicle is detected at time t is calculated. . When the number of corresponding areas whose calculated similarity is equal to or greater than the threshold S is equal to or greater than a predetermined value C (for example, 2), the feature amount of the pattern A (moving object candidate area) is set as a non-vehicle pattern (negative example). sign up.

  The predetermined value C is the same as that of the moving object candidate area when the vehicle has a characteristic amount similar to that of the moving object candidate area between time t + Ts (second imaging time point) and time t + Tu (third imaging time point). It is a numerical value that exceeds the number of times (for example, once) that is considered to exist at the position. As a result, it is possible to prevent accidentally registering the actual vehicle feature quantity existing at the same position as the moving object candidate region as a non-vehicle pattern between time t + Ts and time t + Tu, and more accurately. Can be collected.

  When the moving object candidate area (pattern A) is identified and detected as a vehicle at time t, tracking processing of the vehicle detected from time t is started. With reference to the information of the tracking process, the accuracy of the vehicle detection process is enhanced, and the collection timing of the non-vehicle pattern (negative example) is controlled to suppress the collection of an erroneous negative example.

  For example, when tracking is performed for a time difference Ts or more between time t + Ts (second imaging time) and time t (first imaging time), the feature amount of the moving object candidate region is discarded. That is, when the moving object candidate area is tracked for a time difference Ts or more between time t + Ts and time t, it is determined that the moving object candidate area is an image of an actual vehicle. Can be discarded without registering as a non-vehicle pattern. Accordingly, it is possible to prevent the original characteristic amount of the vehicle from being erroneously registered as a non-vehicle pattern, and to improve the accuracy of the collected non-vehicle pattern.

  FIG. 6 is an explanatory diagram showing another example of a non-vehicle pattern registration method. As shown in FIG. 6, when a pattern A (moving object candidate region) when a vehicle is detected at time t (first imaging time point) is specified, a plurality of time zones after a predetermined time from time t are set. To do. For example, the similarity between the feature quantity of the corresponding area on one or a plurality of captured images taken in each time zone and the feature quantity of the pattern A, such as the time from time t to time t + T1, and the time from time t + T1 to time t + T2. When there is a pattern whose degree is equal to or greater than the threshold value S, the pattern A is registered as a non-vehicle pattern (negative example). The time zone can be set to about 30 seconds, for example.

  Since it is divided into a plurality of time zones, and only the pattern similar to the pattern A is registered as a non-vehicle pattern in each time zone, even if there is a vehicle that stops in the same area as the pattern A by chance, Since it rarely exists over time, it is possible to prevent a pattern representing a vehicle from being erroneously registered as a non-vehicle.

  When a plurality of moving object candidate areas (pattern A) are specified, the negative example registration unit 110 associates the position and size on the captured image with an area equivalent to the moving object candidate area for each moving object candidate area. The moving object candidate area (pattern A) is registered in the negative example storage unit 111 as a non-vehicle pattern (negative example). That is, when registering the feature amount or the like of pattern A as a non-vehicle pattern (negative example), it is registered for each area of pattern A.

  FIG. 7 is an explanatory diagram showing an example of a registered non-vehicle pattern. As shown in FIG. 7, the non-vehicle pattern (negative example) is registered for each region. For example, in the non-vehicle pattern with ID 1, the upper left coordinate of the region is (x1, y1), the height of the region is h1, the width is w1, and the feature amount (for example, 256-dimensional vector) is v1. The non-vehicle pattern whose ID is 1 is determined to be similar to the feature amount of the pattern A (moving body candidate region) whose coordinates are (x1, y1), the height of the region is h1, and the width is w1. The same applies to other non-vehicle patterns.

  By registering a non-vehicle pattern (negative example) for each area of pattern A, the vehicle / non-vehicle determination when detecting a vehicle based on the captured image is performed using the registered area (the coordinates of the area of the non-vehicle pattern, (Size) only, and it is not necessary to perform determination processing over the entire captured image. As typical non-vehicle patterns, shadows of buildings, trees, etc. exist locally in almost the same area, so non-vehicles can be determined without determining whether or not the entire captured image is a non-vehicle pattern. By determining only the area registered as a pattern, sufficient vehicle / non-vehicle determination can be made, and processing time can be shortened compared to the case where the entire captured image is processed, thereby realizing high-speed processing. Can do. Further, even if the vehicle / non-vehicle determination is wrong (even if an internal error occurs), the region that has been registered as the non-vehicle pattern has an effect, so that robustness against the internal error can be obtained.

  The above-described algorithm is intended to collect, as a main target, erroneous detection patterns that frequently occur in a local region in an image. If the pattern frequently occurs in a specific local area, it is highly likely that the same pattern frequently occurs in the same area at subsequent times. On the other hand, the fact that the same vehicle continues to stop for a long time in the area where the vehicle travels and that the same vehicle appears multiple times in the same area at the same timing, except for abnormal situations such as when a traffic accident occurs Almost impossible. Therefore, by setting a value greater than the time that can be considered as a normal vehicle stop time to Ts, it is possible to collect frequent detection patterns while avoiding erroneous collection of vehicles as erroneous detection patterns.

  Further, as the lower limit of the number of occurrences of similar patterns, the predetermined value C is accidentally positioned in the same region at the timing when the traveling vehicle of the same type as the erroneous detection pattern candidate (pattern A) determines the collection condition between time t + Ts and time t + Tu. By setting it as a value that is greater than or equal to the number of times considered to be, the probability of erroneous collection of a non-vehicle pattern (negative example) can be kept extremely low.

  Normalized correlation of 256-dimensional pixel values was used for similarity, time Ts was 1200 frames (1 frame is 100 ms), time Tu was 1500 frames, similarity threshold S was 0.99, and predetermined value C was 2. In the experiment, a false detection pattern (negative example) could be collected without error for various test scenes by the negative example automatic collection method of the present embodiment.

  Although it is assumed that one or more erroneous detection pattern candidates (pattern A) are actually prepared at a time, the amount of calculation of the collection process itself depends on the number of erroneous detection pattern candidates secured. Since most of the similarities relating to 256-dimensional vectors are the upper limit of the number of false detection pattern candidates to be secured at a time, it is easy to ensure real-time performance of the entire process.

  FIG. 8 is a flowchart showing a processing procedure of negative automatic collection by the vehicle detection device 100 of the present embodiment. In the following description of the flowchart, the vehicle detection device 100 is simplified and referred to as the device 100 for convenience. The apparatus 100 acquires a captured image (S11), and detects the pattern A as a vehicle on the captured image at time t (S12). The pattern A is the moving object candidate area itself when it is determined that the moving object candidate area is a vehicle.

  The apparatus 100 stores the detected pattern A as an erroneously detected pattern candidate (negative example candidate), the area ID, the coordinates of the area, the size, and the feature amount of the pattern A (S13). The apparatus 100 calculates the feature amount of the corresponding area that is the same area as the pattern A in the captured image from time t + Ts to time t + Tu (S14).

  The apparatus 100 determines whether or not a pattern (corresponding region) whose similarity to the feature amount of the pattern A is greater than or equal to the threshold value S (corresponding region) occurs a predetermined number C or more (S15). YES), it is determined whether or not the tracking process of the pattern A (the pattern determined to be a vehicle at the time t) continues from the time t to the time t + Ts (S16).

  When the tracking process is not continued (NO in S16), the pattern A is registered as a negative example (non-vehicle pattern) (S17), and the process ends. If the predetermined number of times C or more has not occurred in step S15 (NO in S15), or if the tracking process is continuing (YES in S16), the apparatus 100 discards the pattern S as a negative example (S18), The process ends. Note that the processing shown in FIG. 8 can be repeatedly executed at an appropriate cycle.

  The vehicle detection apparatus 100 according to the present embodiment can also be realized using a general-purpose computer that includes a CPU, a RAM, and the like. That is, as shown in FIG. 8, a computer program that defines each processing procedure is recorded on a computer program recording medium such as a CD, DVD, USB memory, etc., and the computer program is loaded into a RAM provided in the computer. By executing the computer program on the CPU, the vehicle detection device 100 can be realized on the computer.

  A vehicle / non-vehicle determination criterion in the vehicle detection unit 102 using a negative example (non-vehicle pattern) that is automatically collected at the automatic collection timing (period) by the negative example collection unit 107 and registered in the negative example storage unit 111. To correct.

  The correction of the determination criterion of the vehicle detection unit 102 is performed by converting a moving object candidate region having a feature amount having a large similarity with the feature amount of the negative example (non-vehicle pattern) collected and registered by the automatic collection method described above into a non-vehicle pattern. The process for determining that there is a vehicle can be realized in the form of being executed in parallel with the vehicle detection by the vehicle detection unit 102. Then, only when it is determined that the vehicle is both, the determination result “vehicle” is output as the final detection result, and the process proceeds to the tracking process phase. Otherwise, the determination result “non-vehicle” is output. However, the tracking process is not executed.

  The vehicle detection unit 102 performs subsequent vehicle detection using the negative example (non-vehicle pattern) automatically collected by the negative example collection unit 107 at the automatic collection timing (period) and registered in the negative example storage unit 111. .

  That is, the vehicle candidate area specifying unit 103 specifies the moving object candidate area based on the captured image obtained at an arbitrary imaging time point. The feature amount calculation unit 104 calculates the feature amount of the identified moving object candidate region. The vehicle similarity calculation unit 105 calculates the similarity between the feature amount of the moving object candidate region and the feature amount of one or more non-vehicle patterns (negative examples) registered in the region associated with the moving object candidate region. To do. In this case, the negative example used is, for example, a non-vehicle pattern automatically collected by the negative example collection unit 107.

  When there is a non-vehicle pattern in which the calculated similarity is equal to or greater than a predetermined threshold, the vehicle determination unit 106 detects the moving object candidate region as a non-vehicle. By using the moving body determination information (non-vehicle pattern) obtained by the automatic collection process, it is possible to prevent a non-moving body from being erroneously detected as a moving body and to detect the moving body with high accuracy. .

  Moreover, the vehicle determination part 106 detects the said mobile body candidate area | region as a vehicle, when the non-vehicle pattern whose calculated similarity is more than a predetermined threshold value does not exist. By using the moving body determination information (non-vehicle pattern) obtained by the automatic collection process, the moving body can be detected with high accuracy. When there are a plurality of moving object candidate areas, the same processing is repeated for each moving object candidate area.

  In the above algorithm, a method is used in which a negative example (non-vehicle pattern) is registered for each region, and vehicle / non-vehicle determination for vehicle detection processing is performed only in the registered region. Non-vehicle patterns that occur frequently in the same area (for example, shadows of buildings that do not change for a long time, changes in a short time due to the influence of wind, etc., but shadows of leaves with limited variations if attention is paid to each area Etc.) can be sufficiently obtained.

  Also, in terms of the amount of calculation required for processing, it is sufficient to perform similarity calculation for a single captured image at most as many as the number of registered negative examples. Compared to a method that uses all of the patterns, the amount of calculation is much smaller. The real-time property of the process can be maintained with a simple response such as setting an upper limit on the number of negative registrations. Furthermore, by adopting a method of registering negative examples for each area, even if the vehicle is mistakenly collected and registered as a false detection pattern at the collection stage, the effect is only on the registered area Therefore, robustness against internal errors can be obtained as a whole system.

  FIG. 9 is an explanatory diagram illustrating an example of an automatic learning result of the vehicle detection device 100 according to the present embodiment. The example of FIG. 9 shows a case where no vehicle is present on the road. FIG. 9A shows a captured image obtained by imaging a road. FIG. 9B shows an initial vehicle detection result and shows a state before the automatic collection algorithm is executed. FIG. 9C shows a vehicle detection result in a case where a negative example is automatically collected by the vehicle detection device 100 of the present embodiment and the determination criteria for vehicles and non-vehicles are corrected.

  As can be seen by comparing FIG. 9B and FIG. 9C, in FIG. 9B, erroneous detection patterns that are erroneously detected as vehicles are frequently generated before automatic learning. On the other hand, as shown in FIG. 9C, the erroneous detection pattern is erased by the automatic learning function, and the generation of the erroneous detection pattern is suppressed.

  FIG. 10 is an explanatory diagram illustrating another example of the automatic learning result of the vehicle detection device 100 according to the present embodiment. The example of FIG. 10 shows a case where a vehicle is present on the road. FIG. 10A shows a captured image obtained by imaging a road. FIG. 10B shows an initial vehicle detection result and shows a state before the automatic collection algorithm is executed. FIG. 10C shows a vehicle detection result in a case where a negative example automatic collection by the vehicle detection apparatus 100 of the present embodiment is performed to correct a vehicle / non-vehicle determination criterion.

  As can be seen by comparing FIG. 10B and FIG. 10C, in FIG. 10B, erroneous detection patterns that are erroneously detected as vehicles are frequently generated before automatic learning. On the other hand, as shown in FIG. 10C, the erroneous detection pattern is erased by the automatic learning function, and the generation of the erroneous detection pattern is suppressed.

  According to this embodiment, non-vehicle patterns (moving body determination information) are automatically collected without human intervention, and a moving body such as a vehicle is detected with high accuracy using the collected non-vehicle patterns. It is possible to realize a moving body detection system capable of

  Conventionally, in a recognition technique based on learning of a sample, the appropriateness of a learning sample naturally affects the recognition accuracy. Among them, the majority of applications that achieve high accuracy of recognition require human intervention. That is, a sample that is determined to be appropriate by a person is used for learning. This also indicates the difficulty in automating the determination of the appropriateness of the learning sample.

  In the present embodiment, it is possible to automate the collection of large amounts of data and the creation of learning samples that had to be performed manually in the past. In addition, it is possible to realize a vehicle sensor having high-accuracy vehicle detection / tracking performance that appropriately corresponds to the characteristics of each point. Thereby, the vehicle sensing system which can provide a driver | operator with the high-accuracy information for assisting safe driving | operating can be spread widely at realistic cost. In addition, since the tracking processing executed based on the detection result tends to have a large amount of calculation, the reduction in the frequency of occurrence of false detection that can be realized by the present embodiment directly leads to a reduction in the amount of calculation. Problems related to processing time, such as delays in information transmission to the driver due to delays, can be reduced.

  In the above-described embodiment, the feature quantity calculation unit 104 and the negative example feature quantity calculation unit 108 can be combined into one. In addition, the vehicle similarity calculation unit 105 and the negative example similarity calculation unit 109 can be combined into one.

  The disclosed embodiments are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 100 Vehicle detection apparatus 101 Image input part 102 Vehicle detection part 103 Vehicle candidate area | region specific part 104 Feature-value calculation part 105 Vehicle similarity calculation part 106 Vehicle determination part 107 Negative example collection part 108 Negative example feature-value calculation part 109 Negative example similarity Calculation unit 110 Negative example registration unit 111 Negative example storage unit 112 Vehicle tracking unit 200 Imaging device

Claims (9)

In a moving body detection apparatus that detects a moving body based on moving body determination information registered in advance to determine a moving body and a captured image obtained by imaging with an imaging apparatus,
A specifying unit that specifies a moving object candidate area that is a pixel area indicating a moving object candidate based on a first captured image obtained at the first imaging time point;
First feature amount calculating means for calculating a feature amount based on a pixel value of the moving object candidate region;
Second feature amount calculation means for calculating a feature amount based on a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second imaging time point after the first predetermined time has elapsed from the first imaging time point. When,
Similarity calculation means for calculating the similarity of the feature quantities calculated by the first feature quantity calculation means and the second feature quantity calculation means;
A moving object detection comprising: a registration means for registering the feature amount of the moving object candidate area as moving object determination information when the similarity calculated by the similarity calculating means is equal to or greater than a predetermined first threshold value. apparatus. The second feature amount calculating means includes:
A feature amount of each corresponding region on the plurality of captured images obtained between the second imaging time point and the third imaging time point after the elapse of the second predetermined time is calculated;
The similarity calculation means includes:
The similarity between the feature amount of the moving object candidate region and the feature amount of each corresponding region is calculated,
The registration means includes
When the number of corresponding areas whose similarity calculated by the similarity calculation means is equal to or greater than the first threshold is greater than or equal to a predetermined value, the feature amount of the moving object candidate area is registered as moving object determination information. The moving body detection device according to claim 1, wherein A tracking means for tracking a position on a captured image obtained at an imaging time after the imaging time of the moving object candidate region identified based on a captured image obtained at an arbitrary imaging time by the identifying means;
And a discarding unit for discarding the feature amount of the moving object candidate region when the tracking unit performs tracking for a time difference greater than or equal to the time difference between the second imaging time point and the first imaging time point. The moving body detection apparatus of Claim 1 or Claim 2. The specifying means is:
One or more moving object candidate areas are identified,
The registration means includes
For each moving object candidate area, the feature amount of the moving object candidate area is registered as moving object determination information in association with an area having a position and size on a captured image equivalent to the moving object candidate area. The moving body detection apparatus according to claim 1, wherein the moving body detection apparatus is provided. The specifying means is:
The mobile object candidate area is specified based on the captured image obtained at an arbitrary imaging time point,
The first feature amount calculating means includes:
The feature amount of the moving object candidate region is calculated,
The similarity calculation means includes:
The similarity between the feature amount of the moving object candidate region and the feature amount of the moving object determination information registered in the region associated with the moving object candidate region is calculated,
5. The movement according to claim 4, further comprising detection means for detecting the moving object candidate region as a non-moving object when there is moving object determination information having a calculated similarity equal to or greater than a predetermined second threshold value. Body detection device. The detection means includes
The mobile object candidate area is configured to be detected as a mobile object when there is no mobile object determination information whose calculated similarity is equal to or greater than a predetermined second threshold value. Moving body detection device.   A mobile body detection system comprising: the mobile body detection apparatus according to claim 1; and an imaging apparatus that outputs a captured image to the mobile body detection apparatus. In a computer program for causing a computer to detect a moving body based on moving body determination information registered in advance to determine a moving body and a captured image obtained by imaging with an imaging device,
On the computer,
Identifying a moving object candidate area which is a pixel area indicating a moving object candidate based on a first captured image obtained at the first imaging time point;
Calculating a feature amount based on a pixel value of the moving object candidate region;
Calculating a feature amount based on a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second captured time after a first predetermined time has elapsed from the first captured time;
Calculating the similarity of the calculated feature quantity;
When the calculated similarity is greater than or equal to a predetermined first threshold value, the computer program is configured to execute a step of registering the feature amount of the moving object candidate region as moving object determination information. In the moving body detection method by the moving body detection device that detects the moving body based on the moving body determination information registered in advance to determine the moving body and the captured image obtained by imaging with the imaging device,
Identifying a moving object candidate area which is a pixel area indicating a moving object candidate based on a first captured image obtained at the first imaging time point;
Calculating a feature amount based on a pixel value of the moving object candidate region;
Calculating a feature amount based on a pixel value of a corresponding region of the moving object candidate region in a second captured image obtained at a second captured time after a first predetermined time has elapsed from the first captured time;
Calculating the similarity of the calculated feature quantity;
When the calculated similarity is equal to or greater than a predetermined first threshold, registering the feature amount of the moving object candidate region as moving object determination information; and
Detecting a moving object using registered moving object determination information. A moving object detection method comprising:

Images