JP2007257133A - Object detection system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection system for accurately discriminating a pedestrian from a vehicle even when an environmental temperature rises and reaches a temperature which is equivalent to the temperature of a pedestrian. <P>SOLUTION: This object detection system is provided with: a far infrared camera 11 for acquiring a thermal image 50 in the periphery of an own vehicle; a featured point extraction part 31 for extracting a featured point 6 on the basis of luminance change with peripheral pixels 61 from the thermal image; a featured point temperature detection part 32 for detecting the temperature of a portion corresponding to the featured point 6 from the thermal image; an image dividing part 33 for dividing the thermal image into predetermined regions; an object region extraction part 35 for extracting a region where the featured point having the detected predetermined temperature is highly likely to appear in each of the divided image regions as an object candidate region; a directional edge extraction part 36 for extracting a horizontal edge 9 from the thermal image; and a vehicle discrimination part 37 for discriminating a vehicle on the basis of the interconnection of the horizontal edge and the object candidate region. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an object detection system that detects an object such as a pedestrian or a vehicle traveling forward.

  Conventionally, Patent Literature 1 is known as a technique for discriminating between a pedestrian and a vehicle in an image captured by a camera.

  In Patent Document 1, first, a candidate area for a pedestrian is extracted by extracting an area having a predetermined temperature or higher by binarization from a thermal image obtained from a far-infrared camera.

Next, when the upper part of the heat source detected as the extracted candidate area of the pedestrian is searched and a horizontal edge that seems to be a part of the vehicle is detected in a certain area, the heat source is a component of the vehicle. Differentiate between pedestrians and vehicles as part of it.
JP 2004-348645 A

  However, in the method of Patent Document 1, when a region having a certain temperature or higher is cut out from a far-infrared image by binarization, a candidate region for a pedestrian or a vehicle has to be cut out.

  In contrast, in summer when the ambient temperature rises, the ambient temperature may be equivalent to that of the pedestrian, so it is difficult to determine the threshold for binarization and it is difficult to cut out the pedestrian area. There was a case.

  Therefore, an object of the present invention is to provide an object detection system capable of accurately distinguishing between a pedestrian and a vehicle even when the environmental temperature rises to a temperature equivalent to that of a pedestrian.

  In order to achieve the above object, the present invention is obtained by a thermal image acquisition unit that acquires a thermal image around a host vehicle and an image acquisition unit that acquires the thermal image acquisition unit or at least an image capable of edge detection. Feature point extraction means for extracting a feature point from an image based on a luminance change with surrounding pixels, and a feature point temperature for detecting a temperature of a part corresponding to the feature point extracted by the feature point extraction means from the thermal image A feature point having a predetermined temperature detected by the feature point temperature detection unit in each of the image regions divided by the detection unit, an image division unit that divides the thermal image into predetermined regions, and the image division unit. An object region extraction unit that extracts a region that appears frequently as an object candidate region, and has a predetermined direction from the thermal image or the image acquired by the image acquisition unit An object detection system comprising: direction edge extraction means for extracting a direction edge; and vehicle determination means for determining a vehicle in the thermal image based on a connection relationship between the direction edge and the object candidate region. To do.

  The object detection system of the present invention configured as described above uses object candidate regions and direction edges that are extracted by capturing local changes in the image, so that the environmental temperature is equivalent to that of pedestrians in summer, etc. In this case, the pedestrian and the vehicle traveling ahead can be accurately determined.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an object detection system 10 according to the present embodiment.

  First, in terms of configuration, the object detection system 10 according to the present embodiment includes an image acquisition unit 1 that acquires an image such as a thermal image around the own vehicle, acquired image data, calculation result data, and the like. It is mainly composed of a storage unit 2 that stores data and a calculation unit 3 that performs calculations for detecting an object.

  The image acquisition unit 1 includes at least a far-infrared camera 11 (IR camera) as thermal image acquisition means for acquiring a thermal image. Here, the thermal image of the present embodiment refers to a far infrared image captured by the far infrared camera 11. The far-infrared wavelength band targeted by the far-infrared camera 11 is 8 to 14 μm.

  As shown in FIG. 2, the image acquisition unit 1 is installed in a front grill portion or the like in front of the host vehicle 4, and mainly images the front space of the host vehicle 4.

  Further, the thermal image 50 (see FIG. 5) obtained by capturing with the far-infrared camera 11 is digitized and stored in the image memory 21 of the storage unit 2. The image memory 21 is configured by a storage device such as a RAM (Random access memory).

  Since this thermal image 50 is output by changing the luminance or color of the pixel according to the temperature of the imaged subject, the temperature of the pixel is calculated by detecting the luminance value and color type of each pixel. can do.

  In addition, the storage unit 2 includes a storage device such as a RAM that temporarily stores the results of calculations performed by the calculation unit 3 as a memory unit 22.

  On the other hand, the calculation unit 3 is provided with an internal CPU (Central Processing Unit) that executes various programs stored in a storage device such as a ROM (Read Only Memory). Processing is performed as point extraction means, feature point temperature detection means, image division means, division area score conversion means, object area extraction means, direction edge extraction means, and vehicle discrimination means.

  In the feature point extracting unit 31 as the feature point extracting means, as shown in FIG. 6, the feature point 6 is extracted from the thermal image 50 as shown in FIG. 5 stored in the image memory 21 based on the luminance change with the surrounding pixels. , ... are extracted. That is, as shown in FIGS. 7A and 7B, the pixels 61 whose luminance difference with the surrounding pixels 61,... Is not less than a predetermined value are extracted as feature points 6A and 6B.

  More specifically, the feature point extraction unit 31 extracts feature points 6 by using a method generally called a corner detector as an image processing method. For example, Jianbo Shi, Carlo Tomasi, (“Good Features to Track”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94) pp.593-600), etc., a method for extracting pixels suitable for tracking Is disclosed.

  In this method, the pixel 61 at the corner that is the corner of the contour of the imaged subject is extracted as the feature point 6. In addition, the feature point 6 is not extracted so much in an image obtained by imaging a space in which the temperature distribution in the thermal image 50 is uniform, and is large in an image obtained by imaging a space in which the temperature distribution changes greatly. Extracted.

  Further, a subject whose surface configuration is not uniform because the surface orthogonal to the imaging direction is not flat, the radiation direction of far infrared rays emitted from the subject is not uniform with respect to the imaging surface. In this case, the change in the luminance value becomes large and the feature point 6 is easily extracted.

  The feature point temperature detection unit 32 as feature point temperature detection means detects the temperature of the pixel 61 corresponding to the feature point 6 extracted by the feature point extraction unit 31 from the thermal image 50.

  That is, since the thermal image 50 is output by changing the brightness or color of the image according to the temperature, the brightness and color of the extracted feature points 6 are detected, so that the temperature of each feature point 6 is determined. Can be calculated.

  Further, in the image dividing unit 33 as an image dividing unit, the thermal image 50 acquired by the far-infrared camera 11 is divided into a lattice shape as shown in FIG. Are formed, or strip-like regions 81,... Are formed by being divided into vertical strips as shown in FIG.

  Then, in the divided region point scoring unit 34 as the divided region point scoring means, the feature point 6 is selected based on the temperature of the feature point 6 detected by the feature point temperature detecting unit 32, and the mesh divided by the image dividing unit 33. .. Or the strip-shaped regions 81..., The frequency of the feature points 6 selected for each region is calculated to score each region.

  For example, only the feature points 6 having a temperature higher than that of the pedestrian 52, which is 40 ° C. or higher, are selected from the feature points 6 as points to be scored, or only the feature points 6 having a medium temperature that is about the pedestrian 52. Can be selected for scoring.

  In addition, the object area extracting unit 35 as the object area extracting unit extracts an area higher than a predetermined score among the areas scored by the divided area scoring unit 34 as an object candidate area.

  For example, a region where a high score is obtained by scoring only the high-temperature feature point 6 is set as a high-temperature region 7 as an object candidate region, and a region where a high score is obtained by scoring only the feature point 6 at medium temperature The intermediate temperature region 8 is set as a candidate region.

  On the other hand, the direction edge extracting unit 36 serving as a direction edge extracting unit extracts an edge having a predetermined direction from the thermal image 50 as a direction edge. For example, the thermal image 50 is filtered by a predetermined differential filter, and the result is binarized to extract direction edges in which the same value continues in one direction such as the horizontal direction or the vertical direction.

  FIG. 10 is a diagram in which the horizontal edge 9 extending in the horizontal direction as the direction edge is extracted from the thermal image 50 and displayed.

  Further, in the vehicle determination unit 37 as a vehicle determination unit, the thermal image 50 is determined based on the connection relationship between the direction edge extracted by the direction edge extraction unit 36 and the object candidate region extracted by the object region extraction unit 35. The vehicle is determined. This vehicle discrimination method will be described in detail later.

  Next, the process flow of the object detection system 10 of the present embodiment will be described with reference to the flowchart of FIG.

  First, in step S1, the far-infrared camera 11 mounted on the host vehicle 4 captures the front space around the host vehicle 4 and acquires a thermal image 50 as shown in FIG. This thermal image 50 is acquired as, for example, a gray scale image. In this figure, other vehicles 51 in front of the host vehicle 4, pedestrians 52, 52 walking on the sidewalk, street lamps 53,.

  Thus, the pixels 61,... Of the thermal image 50 captured as a gray scale image are displayed with multi-level luminance values as shown in the partial enlarged view of FIG. 7, and feature point extraction is performed in step S2. 6 extracts pixels 61,... Whose luminance difference from the surrounding pixels is a predetermined value or more as feature points 6,.

  For the feature points 6,... Extracted in step S2, the feature point temperature detection unit 32 detects the temperature based on the thermal image 50 (step S3).

  Furthermore, in step S4, the thermal image 50 is divided into a grid pattern as shown in FIG. Further, strip-like regions 81,... Divided into vertical bands as shown in FIG.

  In step S5, the object region extraction unit 35 performs scoring on the mesh-like regions 71,... Divided in step S4, and the object candidate region where the object is highly likely to exist in the region where the score is high. Extract as

  This scoring is performed only for the feature points 6... Of 40 degrees or more, which is clearly higher in temperature than the pedestrian, for the mesh-like regions 71.

  For example, the muffler portion of the other vehicle 51 traveling in front of the host vehicle 4 is extracted as the high temperature regions 7A and 7B because it is at a high temperature during traveling, and the street lamp is also used as the high temperature region 7C at night. Extracted.

  Further, the strip-shaped regions 81,... Shown in FIG. 9 are scored for feature points 6,.

  For example, pedestrians 52 and 52 walking around the host vehicle 4 are extracted as intermediate temperature regions 8C and 8D, and tire portions of other vehicles 51 are also extracted as intermediate temperature regions 8A and 8B.

  Here, the intermediate temperature regions 8A to 8D shown in FIG. 9 are obtained by partially extracting a portion having a high score from the strip shape, not the entire strip-shaped region 81.

  On the other hand, in step S6, the horizontal edge 9,... As shown in FIG.

  That is, if the object ahead is the other vehicle 51, many horizontal edges 9A-9C, such as a bumper part, a trunk lid part, and a roof part, can be extracted. Further, the horizontal edges 9A to 9C extracted from the other vehicle 51 are suitable as an index for determining connectivity of each region described later.

  Note that the direction edge stably extracted from the vehicle includes a vertical edge, and therefore, the vertical edge (not shown) in the vertical direction can be extracted together with the horizontal edge 9 to perform vehicle discrimination.

  In step S <b> 7, the vehicle determination unit 37 determines the other vehicle 51 as the vehicle region 510 from the thermal image 50.

  That is, the extraction results of the high temperature regions 7A to 7C (FIG. 8) and the intermediate temperature regions 8A to 8D (FIG. 9) extracted in step S5, and the horizontal edges 9A to 9C (FIG. 10) extracted in step S6. A plurality of high temperature regions 7A and 7B, medium temperature regions 8A and 8B, and horizontal edges 9A to 9C that are recognized as regions related to one vehicle based on the overlap between the extracted results and the distance between them. An area including the area is determined as a vehicle area 510 as shown in FIG.

  The high temperature regions 7A and 7B included in the vehicle region 510 are highly likely to be muffler portions of the other vehicle 51, and the intermediate temperature regions 8A and 8B are highly likely to be warm tire portions, and the side edges 9A to 9C is included in the vehicle region 510 because there is a high possibility that it is a roof portion, a trunk lid portion, and a bumper portion.

  Up to this point, the processing for one thermal image 50 captured at a certain time point has been described. However, in the object detection system 10 according to the present embodiment, the thermal images 50,. Therefore, the flow of processing when the vehicle region 510 or the like is determined in the thermal image 50 (frame) at the previous time will be described with reference to the flowchart of FIG.

  First, in step S11, it is determined whether an area outside the vehicle other than the vehicle such as the pedestrian 52 is extracted in the immediately preceding frame, or whether the area outside the vehicle extracted in the previous frame is being tracked.

  If the vehicle outside region is extracted or tracked, tracking is continued as the vehicle outside region as it is (step S12), and it is determined whether or not the vehicle region 510 is a region that is not determined as the vehicle outside region. The process proceeds to step S13.

  In step S13, it is determined whether or not the vehicle region 510 is based on the high temperature region 7, the intermediate temperature region 8, and the lateral edge 9 extracted in the same manner as in step S7.

  Further, the positions in the respective images of the high temperature region 7 and the medium temperature region 8 grouped as the vehicle region 510 are stored, and tracking is performed in time series. Further, the area where the vehicle area 510 has been extracted in the immediately preceding frame is stored as tracking information from the immediately preceding frame.

  Also, by adding area information to the tracking information, for example, when the area of each area suddenly increases or decreases, such as when the area of the area is doubled or halved, the previous frame It can also be determined to lose the continuity of tracking from.

  In step S14, the positional relationship between the lateral edge 9 and the object candidate regions (the high temperature region 7 and the intermediate temperature region 8) located in the vehicle region 510 of each tracked region is determined.

  For example, when the vehicle area 510A as shown in FIG. 12 is determined, the distance D1 between the high temperature area 7D from which the muffler part and the like are extracted and the intermediate temperature areas 8E and 8F from which the tire parts and the like are extracted is used as an index. Make a decision.

  The distance D1 and the width D2 of the entire vehicle area 510A vary depending on the distance between the host vehicle 4 and the other vehicle 51 in time series. In other words, if the distances D1 and D2 are extracted as the constituent parts of the other vehicle 51 as described above, the fluctuation ratios show the same tendency for both the distances D1 and D2.

  Therefore, in step S15, when the difference in the variation ratio per frame is, for example, 5% or more (processing speed is 10 fps (frame per second)), the intermediate temperature region 8E is a vehicle such as a pedestrian in step S16. The area is excluded from the vehicle area 510A by the area dividing means as an outside area. At this time, the position and size of the vehicle area 510A shown in FIG. 12 are maintained without being changed.

  In addition to this, even when the positions of the high temperature region 7D and the intermediate temperature region 8E fluctuate by 5% or more in the vehicle region 510A, the intermediate temperature region 8E is separated by the region separation means. Also good.

  Furthermore, as a simple discrimination method, the intermediate temperature region 8E may be separated as another region when the high temperature region 7D and the intermediate temperature region 8E are physically crossed.

  In step S17, the intermediate temperature region 8E thus separated is set as a tracking target as a region outside the vehicle, and is subsequently tracked in time series.

  On the other hand, a case will be described in which there is a region that is extracted in advance as a pedestrian region 520 and a vehicle region 510 is separately extracted at the same time as shown in FIG. FIG. 13 shows the outline of the three frames arranged in time series.

  Even in such a case, each area is tracked in time series, and when two areas of the pedestrian area 520 and the vehicle area 510 intersect at a certain time, it can be determined as a different area.

  In addition, in the tracking in the present embodiment, it is highly likely that an area that is continuously determined in time series is an object area that exists stably and reliably.

  For this reason, when tracking is performed, a certainty factor is given to each region, and as the region that seems to be the same over a long time (multiple frames) is extracted, the certainty factor score increases. By increasing the number, even if a region is not extracted only in a frame at a certain time for some reason, it is possible to continue to extract stably without losing sight of the tracking target.

  As described above, the object detection system 10 according to the present embodiment uses the feature points 6 and the lateral edges 9 that capture local changes in the image. Therefore, even in summer when the environmental temperature is equivalent to that of the pedestrian 52. The pedestrian 52 and the other vehicle 51 traveling ahead can be accurately discriminated.

  Further, the temperature of the extracted feature point 6 is detected, and if the feature point 6 has a high temperature, it can be easily determined that the object is other than the pedestrian 52.

  Furthermore, a high temperature object candidate region such as a muffler portion of another vehicle 51 traveling in front of the host vehicle 4 and a region adjacent to the muffler portion having a temperature similar to that of the pedestrian 52 or an object candidate region such as a tire portion. It becomes possible to distinguish the other vehicle 51 and the pedestrian 52 easily by grouping.

  In addition, if the method is to divide the acquired thermal image 50 into grid-like mesh areas 71,..., Calculation required for calculating an area that can be simply divided into areas and that has a high possibility of appearance of an object. Cost can be reduced. This also makes it possible to reduce the system cost.

  Further, by using the strip-shaped regions 81 in the vertical direction, it is possible to easily extract regions of the tire portion and the tail lamp portion that are part of the other vehicle 51, and other areas around the host vehicle 4. The detection performance of the vehicle 51 can be improved.

  Also, by dividing the temperature of the feature points 6,... And scoring the areas, extraction of the tire part and tail lamp part that are often imaged at medium temperatures and imaging at high temperatures are often performed. The muffler part can be extracted separately, and the other vehicle 51 can be extracted by being divided into constituent parts.

  This makes it possible to use a simple vehicle model that uses components such as a muffler part and a tire part when discriminating the vehicle, thereby reducing the calculation cost.

  Further, when distinguishing the feature points 6... From the high temperature feature point 6 and the medium temperature feature point 6, the respective divided regions are formed into separate shapes as a mesh region 71 and a strip region 81. Thus, it is possible to improve the performance of detecting the tire portions existing on both sides of the other vehicle 51 as the object candidate region, and at the same time, the detection performance of the muffler portion of the other vehicle 51 can be improved. The performance can be improved.

  The use of the horizontal edge 9 as the direction edge means that a feature amount that is included in the thermal image 50 obtained by imaging the other vehicle 51 and that is easy to extract is used for the recognition process. Such a horizontal edge 9 is often extracted so as to cross the entire vehicle region 510.

  For this reason, if the region belongs to the same vehicle, the horizontal edge 9 is detected so as to penetrate or touch the plurality of object candidate regions extracted by the feature point 6, so that the vehicle can be determined by determining the connection relationship between the regions. The region 510 can be easily extracted, and the calculation cost can be reduced.

  In addition, when the scored object candidate regions (high temperature region 7 and intermediate temperature region 8) are tracked in time series, the score rapidly decreases in a certain time frame (thermal image 50) for some reason. However, stable detection is possible without losing sight of the extracted area.

  Furthermore, by tracking a plurality of regions such as the high temperature region 7 and the intermediate temperature region 8 existing inside the vehicle region 510 detected and grouped as one other vehicle 51 in time series, Even when another object that is not the vehicle 51 is included in the vehicle area 510, if the positional relationship of each area within the vehicle area 510 changes, the area is immediately divided by the area dividing means. It becomes possible to separate. For this reason, an object other than the vehicle can be accurately detected while being distinguished from the vehicle.

  In addition, the other vehicle 51 can be detected with high accuracy by using the arrangement ratio in the vehicle region 510 and the positional relationship between the intermediate temperature region 8 and the high temperature region 7 to separate the vehicle outside region. .

  Furthermore, even when another region overlaps the vehicle region 510 recognized at a certain time after a predetermined time has elapsed, it is possible to recognize each object without confusion, and the other vehicle 51 is detected with high accuracy. be able to.

  In addition, since it is possible to detect the other vehicle 51 with a single far-infrared camera 11, the distance is calculated by a triangulation method with two cameras configured to measure the distance between objects. Compared with the method, the system cost can be reduced and the calculation load can be reduced.

  Hereinafter, Example 1 of the above-described embodiment will be described with reference to FIG. The description of the same or equivalent parts as those described in the above embodiment will be given the same reference numerals.

  In Example 1, the feature points 6 are scored by a method different from that of the above embodiment.

  In the scoring of the first embodiment, the strength points described later are extracted together when extracting the feature points 6,..., The feature points 6,. Use.

  Thereby, even in the environment where the air temperature and the target temperature are close to each other, the feature point 6 can be stably extracted even when the temperature distribution in the thermal image 50 is nearly uniform and the temperature inflection tends to be small.

  On the other hand, even when the temperature distribution in the thermal image 50 is widened and the temperature suppression is increased, only the feature points 6 having high strength are extracted, so that excessive extraction of the feature points 6 can be suppressed. It becomes possible.

  Here, the intensity of the feature point 6 will be described in detail. When the feature points 6C to 6E as shown in FIGS. 14A to 14C are considered, the feature point 6C which is the central pixel in FIG. Is a feature point 6C having a large luminance difference from the surrounding pixels as compared to the feature points 6D and 6E which are the central pixels in FIGS. 14B and 14C.

  As a result of such comparison, the relative strengths of the feature points 6C to 6E are classified as feature points having a strong feature point 6C, feature points having a moderate strength feature point 6D, and feature points having a low strength feature point 6E. can do.

  As another method, the luminance values of the feature points 6,... Are classified into an arbitrary number as the intensity, and a score is given to each rank, and the divided mesh area 71,. ,..., The sum of the number of feature points 6,.

  For example, if 300 feature points 6,... Are extracted from the thermal image 50, they are included in the upper third group in descending order of the luminance value of the pixel 61 corresponding to the feature points 6,. 100 points are given 3 points, 100 points included in the middle third group are given 2 points, and 100 points contained in the lower third group are given 1 point.

  Note that the distribution of the weighting of the points may be changed according to characteristics such as the temperature of the object to be extracted, and the ratio included in each group may be changed.

  Thus, using the weighted points according to the intensity of the feature points 6,..., The total for each divided image region (mesh region 71,... Or strip region 81,...). By obtaining the score, it is possible to determine that the region having a large number of feature points 6... But having a low total score is not an object candidate region.

  As another method for extracting object candidate regions by scoring other divided regions, for example, the high temperature region and the medium temperature region are extracted from the mesh region 71 without using the strip region 81. Also good.

  In this case, when scoring each mesh-like region 71,..., A score corresponding to the high temperature feature point 6,... And a score corresponding to the medium temperature feature point 6,. Two kinds of points are given.

  Then, the high temperature region is extracted by the same method as described above, and when extracting the intermediate temperature region where there are many intermediate temperature feature points 6,... After extracting as object candidate areas, each mesh-like area 71,... Is checked in the vertical direction, and when a plurality of object candidate areas are extracted in the same column, the area appearing at the top By connecting the lowest appearing region as the upper limit and the lower limit, the same effect as the extraction by the strip region 81 can be obtained.

  On the other hand, only the strip-shaped area 81 is used as a divided area for scoring, and for the high temperature feature point 6, the same object candidate area extraction method as that for the medium temperature feature point 6 is used. It is also possible to simulate the extraction of the object candidate area by the mesh area 71.

  In addition, it is not necessary to score the feature points 6,... Of the entire image when scoring the divided areas. For example, street lamps appearing at the top of the image may be extracted as feature points 6 at high temperatures. In order to deal with the noise component that can be generated in this way, the upper third of the image may be excluded from the processing target.

  Other configurations and functions and effects are substantially the same as those in the above-described embodiment, and thus description thereof is omitted.

  Hereinafter, Example 2 of the above-described embodiment will be described with reference to FIG. The description of the same or equivalent parts as those described in the above embodiment will be given the same reference numerals.

  In Example 2, a method for extracting the vehicle area 510 will be described based on a thermal image different from that of the above embodiment.

  In FIG. 15A, the high temperature region 7E and the intermediate temperature regions 8G and 8H are separated, but these regions are connected by the horizontal edges 9D, 9E, 9F, and 9G, and therefore include the connected regions. The area to be extracted is extracted as one vehicle area 510B.

  In FIG. 15B, the intermediate temperature region 8 is not extracted. However, for example, if the other vehicle 51 has just started running, the tire portion may not be warmed, and only the high temperature region 7F and the lateral edges 9H and 9I. May be extracted.

  Even in such a case, when the region set to include the high temperature region 7F and the horizontal edges 9H and 9I has a predetermined area or more in the image so that it can be extracted as the vehicle region 510C, the high temperature region / edge Extraction is performed by determining that the vehicle region is 510C by the connection determining means.

  Further, in the second embodiment and the above-described embodiment, the thermal image 50 in which the other vehicle 51 is captured in the same direction as the host vehicle 4 has been described. However, the other vehicle 51 is captured in the thermal image 50 in the horizontal direction. Even in this case, the muffler part and the axle part at the lower part of the vehicle are extracted as the high temperature region 7, so that the other vehicle 51 can be accurately identified.

  Other configurations and functions and effects are substantially the same as those of the above-described embodiment or other examples, and thus description thereof is omitted.

  The best embodiment and examples of the present invention have been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and the design does not depart from the gist of the present invention. Such modifications are included in the present invention.

  For example, in the above-described embodiment, the case where only the far-infrared camera 11 is used as the thermal image acquisition unit of the image acquisition unit 1 is described. However, the present invention is not limited to this, and visible light that acquires a visible light image is used. A camera (not shown) can be used in conjunction with the far-infrared camera 11.

  In this case, from the image captured by the visible light camera, the feature point extraction unit 31 extracts the feature point 6 using the luminance difference, or the direction edge extraction unit 36 extracts the direction edge such as the horizontal edge 9. It is used as image data when

It is a block diagram explaining the schematic structure of the object detection system of embodiment of this invention. (A) is a side view explaining the arrangement position of the image acquisition part of the own vehicle, (b) is a top view explaining the arrangement position of the image acquisition part of the own vehicle. It is a flowchart explaining the flow of a process of the object detection system of embodiment of this invention. It is a flowchart explaining the flow of the process of the vehicle discrimination | determination of embodiment of this invention. It is the figure which showed the outline of the thermal image around the own vehicle. It is the figure which displayed the extracted feature point. It is explanatory drawing which expanded the pixel periphery in order to demonstrate the extraction method of a feature point. It is the figure which showed the high temperature area | region extracted as a mesh-like area | region and an object candidate area | region. It is the figure which showed the medium temperature area | region extracted as a strip-shaped area | region and an object candidate area | region. It is the figure which displayed the extracted horizontal edge. It is the figure which displayed the vehicle area | region containing a high temperature area | region, an intermediate temperature area | region, and a horizontal edge. It is explanatory drawing explaining the positional relationship of the some area | region included in the inside of a vehicle area | region. It is explanatory drawing explaining the state which a pedestrian area | region and a vehicle area cross in time series. It is explanatory drawing which expanded the pixel periphery in order to demonstrate the intensity | strength of a feature point. It is explanatory drawing explaining the discrimination method of a vehicle area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection system 1 Image acquisition part 11 Far-infrared camera (thermal image acquisition means)
2 storage unit 3 operation unit 31 feature point extraction unit (feature point extraction means)
32 feature point temperature detector (feature point temperature detection means)
33 Image segmentation unit (image segmentation means)
34 Divided area point scoring unit (divided area scoring means)
35 Object region extraction unit (object region extraction means)
36 Direction edge extraction unit (direction edge extraction means)
37 Vehicle discrimination section (vehicle discrimination means)
4 Own vehicle 50 Thermal image 51 Other vehicle (vehicle)
6 Feature point 7 High temperature area (object candidate area)
71 Mesh area (image area)
8 Medium temperature region (object candidate region)
81 Strip area (image area)
9 Horizontal edge (direction edge)
510 Distance between vehicle areas D1, D2 (positional relationship)

Claims (15)

Thermal image acquisition means for acquiring a thermal image around the host vehicle;
A feature point extracting means for extracting a feature point from an image obtained by the thermal image obtaining means or an image obtaining means for obtaining an image capable of at least edge detection based on a luminance change with a peripheral pixel;
Feature point temperature detection means for detecting a temperature of a part corresponding to the feature point extracted by the feature point extraction means from the thermal image;
Image dividing means for dividing the thermal image into predetermined regions;
Object region extraction means for extracting, as an object candidate region, an area where a feature point having a predetermined temperature detected by the feature point temperature detection means appears in each of the image areas divided by the image dividing means. When,
Direction edge extraction means for extracting a direction edge having a predetermined direction from the thermal image or the image acquired by the image acquisition means;
An object detection system comprising vehicle discriminating means for discriminating a vehicle in the thermal image based on a connection relationship between the direction edge and the object candidate region.   The object detection system according to claim 1, wherein the image acquisition unit is a unit that acquires a visible light image.   The vehicle determination means sets a region including the direction edge and the object candidate region as a vehicle region when the direction edge and the object candidate region have a connection relationship. The object detection system described in 1.   The object detection system according to any one of claims 1 to 3, wherein when the image is divided by the image dividing unit, the thermal image is divided in a grid pattern.   5. The object detection system according to claim 1, wherein when the image is divided by the image dividing unit, the thermal image is divided into vertical bands. 6.   6. The frequency is calculated separately for a high-temperature feature point and an intermediate-temperature feature point according to the temperature detected by the feature point temperature detection means. Object detection system. The image dividing means generates two types of divided images in a vertical band shape and a lattice shape from one thermal image,
4. The feature point of medium temperature is used in the strip-shaped region divided into strips in the vertical direction, and the feature point of high temperature is used in the mesh-like region divided into grids. The object detection system according to any one of claims.   The object detection system according to claim 1, wherein the direction edge extraction unit extracts a horizontal edge in a horizontal direction.   9. The vehicle determination unit includes a high temperature part / edge connection determination unit that determines a vehicle region when there is connectivity between the mesh-like region extracted as a high temperature region and the lateral edge. The object detection system described in 1.   The object detection system according to claim 9, wherein the vehicle determination unit determines whether or not there is connectivity between the strip-shaped region extracted as the intermediate temperature region, the horizontal edge, and the high temperature region.   Confidence level of the area by comparing the points calculated by the divided area scoring means for scoring the frequency of appearance of the feature points with respect to the object candidate area extracted by the object area extracting means in time series The object detection system according to claim 1, wherein the object detection system is calculated.   When the positional relationship between the high temperature region and the intermediate temperature region changes in time series with respect to the positional relationship between the plurality of object candidate regions included as one vehicle region by the vehicle determination unit, the intermediate temperature region is The object detection system according to any one of claims 6 to 11, further comprising a region separation unit that determines that the vehicle regions are not in the same vehicle region.   The region separating means determines that the vehicle is not in the same vehicle region when the ratio of the arrangement position of the intermediate temperature region in the vehicle region changes by a predetermined ratio or more when compared in time series. The object detection system according to claim 12, wherein the system is an object detection system.   13. The region separation means determines that the medium temperature region is not the same vehicle region and separates when the high temperature region and the medium temperature region intersect when they are compared in time series. Or the object detection system of 13.   When the vehicle area extracted by the vehicle determination means crosses when another intermediate temperature area is compared in time series, the intermediate temperature area is determined not to be the same vehicle area and is separated. The object detection system according to any one of claims 12 to 14.

Images