JP5090321B2 - Object detection device - Google Patents

Description

  The present invention relates to an object detection device that detects a predetermined object from an image by template matching.

  In order to reduce the number of casualties due to traffic accidents, the development of preventive safety systems that prevent accidents in advance is underway. A preventive safety system is a system that operates in a situation where there is a high possibility of an accident.For example, when there is a possibility of collision with an obstacle in front of the host vehicle, a warning is given to the driver to alert the driver. Pre-crash safety systems have been put into practical use that reduce the damage to passengers by automatic braking when inevitable situations occur.

  As a method for detecting obstacles such as pedestrians in the above system, for example, a method of imaging the front of the vehicle with a camera and detecting a pattern similar to a previously stored obstacle template from the captured images There is.

  For example, Patent Document 1 describes a method of detecting a pedestrian near the end of a road from an image obtained by imaging the front of the host vehicle. According to this method, when template matching with a pedestrian template is performed in the vicinity of the road edge detected by the road detection means, and a correlation value (hereinafter referred to as a matching rate) indicating the degree of matching with the template is equal to or greater than a threshold value By determining that there is a pedestrian, it is possible to efficiently detect a pedestrian that requires attention from the driver.

JP-A-2005-316607

  However, since template matching only looks at the degree of matching in the entire template, not the local distribution, there is a problem that it is difficult to determine a threshold value that achieves both improvement in detection rate and elimination of false detection. .

  For example, when detecting a pedestrian, if the threshold value is lowered in order to improve the detection performance of the pedestrian, a pattern that is only partially similar to the template is likely to be erroneously detected. On the other hand, if the threshold value is increased in order to reduce false detection, a pedestrian with a low height and a low matching rate, such as a child, cannot be detected.

  The present invention has been made paying attention to the above problems, and an object of the present invention is to provide an object detection device that achieves both improvement in detection rate and elimination of erroneous detection.

  The object detection device of the present invention, which is a means for solving the above problems, divides a detection processing area selected in an image into a plurality of partial areas, and calculates a matching rate between each partial area and a template. Based on each matching rate, it is determined whether or not an object exists in the image.

  According to the present invention, the detection processing area selected in the image is divided into a plurality of partial areas, the matching ratio between each partial area and the template is calculated, and the presence of an object is determined based on each matching ratio. Therefore, it is possible to reduce erroneous detection of patterns that are only partially similar to the template.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram of an object detection apparatus 1000 in the present embodiment.

  The object detection device 1000 is for detecting a preset object from within an image. In the present embodiment, the object detection device 1000 is incorporated in a camera device mounted on an automobile and includes an image obtained by imaging the surroundings of the own vehicle. It is comprised so that the pedestrian 11 may be detected.

  As shown in FIG. 1, the object detection device 1000 includes an image acquisition device 1100, a storage device 1200, and a processing circuit 1300. The image acquisition apparatus 1100 includes a monocular camera mounted at a position where the front of the host vehicle can be imaged, and a digital value signal as a front image obtained by capturing the front of the host vehicle using an image sensor such as a CCD or CMOS. It has the function to convert to.

  The digital value signal may be directly written as image information IMGSRC [x] [y] on the RAM in the processing circuit 1300 as a digital value, or converted into an analog signal and converted again into a digital signal by the processing circuit 1300. The image IMGSRC [x] [y] may be written on the RAM. Note that the image IMGSRC [x] [y] is a two-dimensional array, and x and y indicate the coordinates of the image, respectively.

  The storage device 1200 is configured by a hard disk or a ROM, for example, and stores predetermined data and can be accessed from the processing circuit 1300. The storage device 1200 stores information about a template for detecting a part of an object and a determination threshold value.

  In the present embodiment, a first template 1210 for detecting the upper part (including the head and shoulders) of the pedestrian 11 and a second template for detecting the middle part (including the waist) of the pedestrian 11. 1220, a third template 1230 for detecting the lower part (including the leg) of the pedestrian 11, a first determination threshold value 1211 corresponding to the first template 1210, and a second template corresponding to the second template 1220. A determination threshold 1221 and a third determination threshold 1231 corresponding to the third template 1230 are stored.

  The processing circuit 1300 is configured by a computer having a CPU, a memory, an I / O, and the like, and a predetermined process is programmed to execute the process repeatedly at a predetermined cycle. The processing circuit 1300 includes a detection processing region selection unit 1311, a region division unit 1321, a matching rate calculation unit 1331, an object presence determination unit 1341, an object position detection unit 1310, and a low beam determination unit 1340.

  The object position detection unit 1310 acquires a detection signal from a radar (not shown) that detects an object around the host vehicle such as a millimeter wave radar or a laser radar mounted on the host vehicle, and exists in front of the host vehicle. An object position 21 of the object 10 is detected. For example, as shown in FIG. 3, the object position (relative distance PY [b], lateral position PX [b], lateral width WD [b]) of an object 10 such as a pedestrian 11 or a road sign 12 around the own vehicle is obtained from the radar. get. Here, [b] is an ID number when a plurality of objects 10 are detected.

  The position information of these objects 10 may be acquired by directly inputting a radar signal to the object detection apparatus 1000, or may be acquired by performing communication with a radar using a LAN (Local Area Network). good.

  The detection processing area selection unit 1311 selects a detection processing area W [i] for detecting the object 10 in the image 20, and image information (IMGSRC [x] [y]) from the image acquisition apparatus 1100. And the detection processing area W [i] based on the object information from the object position detection unit 1310.

  Then, the coordinate information of the selected detection processing area W [i] (the horizontal start point coordinate SX [i], the horizontal end point coordinate EX [i], the vertical start point coordinate SY [i], the vertical end point) The coordinates EY [i]) are output to the area dividing unit 1321. Here, [i] is an ID number when a plurality of detection processing areas W are selected.

  If the object position detection unit 1310 is not provided, coordinate information (a horizontal start point coordinate SX [i], a horizontal end point coordinate EX [i], a vertical direction) is searched so as to search all positions of pedestrians to be detected. Direction start point coordinates SY [i], vertical end point coordinates EY [i]). For example, the vehicle may be set to search at intervals of 1 meter from a predetermined distance to a predetermined distance from the front end of the own vehicle and within a predetermined width from the center of the vehicle. You may set so that it may search at intervals of 1 meter within the range of a predetermined width from the center of the vehicle at the distance reached in seconds.

  The area dividing unit 1321 divides the detection processing area W into a plurality of partial areas, and performs a process of dividing according to the template. For example, as shown in FIGS. 6 and 7, the coordinate information (SX [i], EX [i], SY [i], EY [i]) of the detection processing area W, the first template 1210, the second Based on the information of the template 1220 and the third template 1230, the detection processing area W [i] is divided into three partial areas in the vertical direction (vertical direction), and the first template 1210 is detected in the detection processing area W [i]. The first partial area w1 [i] corresponding to the position of the second partial area w2 [i] corresponding to the position of the second template 1220, and the third partial area corresponding to the position of the third template 1230 Each coordinate information of w3 [i] is calculated.

  Then, the coordinate information (SX1 [i], EX1 [i], SY1 [i], EY1 [i]) of the first partial region w1 [i], the coordinate information (SX2) of the second partial region w2 [i]. [I], EX2 [i], SY2 [i], EY2 [i]), the coordinate information (SX3 [i], EX3 [i], SY3 [i], EY3 [) of the third partial area w3 [i]. i]) is output to the matching rate calculator 1331.

  The matching rate calculation unit 1331 calculates the matching rates between these partial regions W1 [i] to W3 [i] and the corresponding templates 1210 to 1230, respectively. Here, the coordinate information (SX1 [i], EX1 [i], SY1 [i], EY1 [i]) of the first partial area w1 [i] and the coordinate information ( SX2 [i], EX2 [i], SY2 [i], EY2 [i]), coordinate information of the third partial area w3 [i] (SX3 [i], EX3 [i], SY3 [i], EY3 [I]), image information IMGSRC [x] [y], information on the first template 1210, the second template 1220, and the third template 1230 are input.

  Then, a first matching rate (MATCH1 [i]) that is a matching rate between the first partial region w1 [i] and the first template 1210, a second partial region w2 [i], and the second template 1220. The second matching rate (MATCH2 [i]) that is the matching rate between the third partial area w3 [i] and the third matching rate (MATCH3 [i]) that is the matching rate between the third template 1230 Are calculated and output to the object presence determination unit 1341.

  The object presence determination unit 1341 performs processing to determine whether an object exists based on the calculation result of the matching rate calculation unit 1331. In the present embodiment, the first matching rate MATCH1 [i], the second matching rate MATCH2 [i], the third matching rate MATCH3 [i], the first threshold value 1211, the second threshold value 1221, The presence / absence of the object 10 is determined based on the third threshold value 1231 and the determination result LBST of the low beam determination unit 1340.

  The low beam determination unit 1340 determines whether or not the vehicle is traveling in a low beam state (the headlamp is lit and facing downward). Then, “1” is substituted for the low beam state, and “0” is substituted for the determination result LBST otherwise, and is output to the object presence determination unit 1341. The determination of the low beam may be performed by directly inputting a headlight lighting switch and a high beam ON / OFF switch signal to the processing circuit 1300, or using a LAN (Local Area Network). You may determine by performing communication.

  Alternatively, the determination may be made using the density of the image acquired by the image acquisition device 1100. For example, in the low beam state, the inside of the low beam is bright and the outside of the range appears dark on the image 30, so the difference between the average luminance value of the lower half of the entire camera image and the average luminance value of the upper half of the camera image is Whether the beam is in the low beam state can be determined depending on whether the value is equal to or greater than a certain threshold value.

  Next, the first template 1210, the second template 1220, and the third template 1230 of the storage device 1200 will be described with reference to FIG. FIG. 2 is a schematic diagram illustrating an example of the first template 1210, the second template 1220, and the third template 1230.

  In this example, a template 1201 for detecting the whole body of the pedestrian 11 is divided into three in the vertical direction, and a first template 1210 for detecting the upper part (head and shoulders) 11A of the pedestrian 11. The second template 1220 for detecting the middle part (waist part) 11B of the pedestrian 11 and the third template 1230 for detecting the lower part 11C (leg part) of the pedestrian 11 are used.

  The first template 1210 is a template in the reference width PW and the reference height PH of the pedestrian 11 as information indicating where the upper part 11A, the middle part 11B, and the lower part 11C of the pedestrian 11 correspond. Information of relative positions (PSX1, PSY1, PEX1, and PEY1) for calculating the matching rate with 1210.

  Similarly, the second template 1220 includes relative positions (PSX2, PSY2, PEX2, PEY2) for calculating the matching rate with the template 1220 within the reference width PW and the reference height PH of the pedestrian 11. Similarly, the third template 1230 includes relative positions (PSX3, PSY3, PEX3, PEY3) for calculating the matching rate with the template 1230 within the reference width PW and the reference height PH of the pedestrian 11.

  Next, the contents of processing in the detection processing region selection unit 1311 will be described with reference to FIGS. 3 is a schematic diagram illustrating an example of setting an object position, FIG. 4 is a schematic diagram illustrating an example of setting a detection processing area, and FIG. 5 is a schematic diagram illustrating another example of setting a detection processing area. .

  The detection processing area selection unit 1311 selects a detection processing area W [i] for detecting the pedestrian 11 in the image IMGSRC [x] [y], and starts and ends the SX [i] and end points of the x coordinate (lateral direction). EX [i], start point SY [i] and end point EY [i] on the y coordinate (vertical direction) are obtained. When there are a plurality of detection processing areas W [i], SX [i], EX [i], SY [i], and EY [i] are arranged and the coordinates of all detection processing areas W [i] are output.

  The detection processing region selection unit 1311 may or may not use the object position detection unit 1310. First, the case where the detection processing area W [i] is set using the object position detection unit 1310 will be described in detail with reference to FIGS. 3 and 4.

  As shown in FIG. 3, the detection processing region selection unit 1311 calculates the image on the image 20 from the relative distance PY [b], the horizontal position PX [b], and the horizontal width WD [b] of the object 10 detected by the object position detection unit 1310. The object position 21 of the object 10 at is calculated. The object position 21 of the object 10 is indicated by a rectangular area surrounding the periphery of the object 10, and the coordinate positions (start point BSX [b], end point BEX [b], end coordinate BEX [b], y coordinate (vertical direction) of the four corners of the object 10 are shown. ) Start point BSY [b], end point BEY [b]).

  A camera geometric parameter that associates the coordinates on the image 20 with the positional relationship in the real world is calculated in advance by a method such as camera calibration, and the height of the object 11 is assumed in advance, for example, 180 [cm]. Thus, the position on the image 20 can be uniquely determined.

  Further, the position of the object 10 detected by the object position detection unit 1310 on the image 20 and the image of the same object 10 shown in the image 20 due to an error in mounting the image acquisition apparatus 1100, a communication delay with the radar, or the like. There may be a difference in position on 20.

  Accordingly, the corrected object position 22 (BSX2 [b], BEX2 [b]) from the object position 21 (BSX [b], BEX [b], BSY [b], BEY [b]) on the image 20 is corrected. , BSY2 [b], BEY2 [b]). The correction of the object position 21 is performed by enlarging or moving the area of the object position 21 by a predetermined amount.

  Next, as shown in FIG. 4, the detection processing area W [] is scanned so that the corrected object position 22 (BSX2 [b], BEX2 [b], BSY2 [b], BEY2 [b]) is raster scanned. i] (SX [i], EX [i], SY [i], EY [i]). In the present embodiment, the object position 22 (repeating the operation of moving the dot position from top to bottom one dot at a time and the movement of moving one dot to the right after moving to the lower end with the upper left end of the object position 22 as the origin is repeated. A plurality of detection processing areas W [i] are set so as to search the BSX2 [b], BEX2 [b], BSY2 [b], BEY2 [b]) without omission.

  The size of the detection processing area W [i] (the estimated pedestrian width SW and the estimated pedestrian height SH) is determined by the relative distance PY [b] from the object 10 detected by the object position detection unit 1310. Further, the assumed pedestrian width SW and the assumed pedestrian height SH are not necessarily the same as the above-described pedestrian reference width PW and pedestrian reference height PH, but a relationship of PW: PH = SW: SH is necessary. It is said.

  FIG. 3 shows an example in which object positions 21 and 21 of two objects 10 (pedestrian 11 and road sign 12) and object positions 22 and 22 obtained by correcting the object positions 21 and 21 are detected. Yes. In FIG. 3, the object position 21 (BSX [b], BEX [b], BSY [b], BEY [b]) on the image 20 of the object 10 is indicated by a solid line, and the corrected object position 22 (BSX2 [BSX [ b], BEX2 [b], BSY2 [b], BEY2 [b]) are indicated by broken lines. In the present embodiment, the corrected object position 22 has a certain margin on the top, bottom, left, and right of the object position 21.

  For example, in the example shown in FIG. 3, the object position 21 (BSX [0], BEX [0], BSY [0], BEY [0]) of the road sign 12 and the corrected object position 22 (BSX2 [0], BEX2 [0], BSY2 [0], BEY2 [0]) are calculated, and the object position 22 of the pedestrian 11 (BSX [1], BEX [1], BSY [1], BEY [1]) Then, the corrected object position 22 (BSX2 [1], BEX2 [1], BSY2 [1], BEY2 [1]) is calculated.

  FIG. 4 shows detection processing areas W [i] (SX [i], EX [i], SY within the object position 22 (BSX2 [b], BEX2 [b], BSY2 [b], BEY2 [b]). A state in which [i], EY [i]) is set is shown.

  For example, the first detection processing area W [0] (SX [0], SY [0], EX [0], EY [0]) in the object position 22 of the road sign 12 is set at the upper left in the object position 22. The last detection processing area W [i] (SX [i], SY [i], EX [i], EY [i]) is set at the lower right in the object position 22. The first detection processing area W [i + 1] (SX [i + 1], SY [i + 1], EX [i + 1], EY [i + 1]) in the object position 22 of the pedestrian 11 is set at the upper left in the object position 22. The last detection processing area W (not shown) is set at the lower right in the object position 22.

  Next, a case where the detection processing area W is set without using the object position detection unit 1310 will be described with reference to FIG. The selection method of the detection processing area W when the object position detection unit 1310 is not used is, for example, a method of setting the search for the entire image 20 while changing the size of the detection processing area W, a specific position, a specific There is a method of setting the detection processing area W limited to only the size of. In the case where the detection processing area W is set to be limited to a specific position, there is a method in which the detection process area W is set to be limited to the distance that the host vehicle travels T seconds later using the host vehicle speed, for example.

  FIG. 5 is an example in the case of searching for a position where the host vehicle has advanced 1.0 seconds later using the host vehicle speed. The detection processing area W is obtained from the calculated position and camera geometric parameter of the size of the detection processing area W (the estimated pedestrian width SW and the estimated pedestrian SH) and the y coordinate, and is detected while being shifted by one pixel in the horizontal direction on the image 20. Area W is set. The estimated pedestrian width SW and the estimated pedestrian height SH need not be the same as the above-described pedestrian reference width PW and pedestrian reference height PH, but a relationship of PW: PH = SW: SH is necessary. It is said.

  This is a method for determining the size of the detection processing region W and the y-coordinate position. By calculating in advance a camera geometric parameter that associates the coordinates on the image 20 with the positional relationship in the real world by a method such as camera calibration, It is possible to calculate backward from the distance at which the pedestrian 11 is desired to be detected.

  In the case of the example shown in FIG. 5, if the horizontal size (image horizontal size) of the image 20 is IMGSIZE, the image start points SX and SY and the image end points EX and EY are respectively (image horizontal size IMGSIZEEX-pedestrian width). SW) pieces of data are entered. The position of the detection processing area W is from the left end (SX [0], SY [0], EX [0], EY [0]) of the image 20 to the right end (SX [IMGSIZEX-W], SY [IMGSIZEX-W]). , EX [IMGSIZEX-W], EY [IMGSIZEX-W]).

  Next, the contents of processing by the area dividing unit 1321 will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of dividing a certain detection processing area W [i] into a plurality of partial areas w [i], and FIG. 7 is a diagram illustrating another example of the division.

  The processing area dividing unit 1321 has a pedestrian assumed width SW [i] (= EX [i] in the detection processing area W (horizontal direction SX [i], EX [i], vertical direction SY [i], EY [i]). ] -SX [i]), estimated pedestrian height SH [i] (= EY [i] -SY [i]), pedestrian reference width PW, pedestrian reference height PH of the first template 1210, and From the relative area positions (PSX1, PSY1, PEX1, PEY1), the following partial expressions (1) to (4) are used to determine the first partial areas w1 [i] (SX1 [i], EX1 [i], SY1 [I], EY1 [i]) are calculated.

SX1 [i] = SX [i] + (PSX1) (SW [i] / PW) (1)
SY1 [i] = SY [i] + (PSY1) (SW [i] / PW) (2)
EX1 [i] = SX [i] + (PEX1) (SW [i] / PW) (3)
EY1 [i] = SY [i] + (PEY1) (SW [i] / PW) (4)
Then, the second partial region w2 [i] (SX2 [i], EX2 [i], SY2 [i], EY2 [i]) and the third partial region w3 [i] (SX3 [i], EX3 [I], SY3 [i], EY3 [i]) are similarly calculated.

  In the example shown in FIG. 6, the template 1201 corresponding to the whole body of the pedestrian 11 is formed by dividing the template 1201 into upper, middle, and lower parts, and the first template 1210 corresponds to the upper part 11 </ b> A of the pedestrian 11. The second template 1220 corresponds to the middle part 11B of the pedestrian 11 and the third template 1230 corresponds to the lower part 11C of the pedestrian 11. When these three templates are combined, a pedestrian template 1201 with no gap is formed. In this case, all the detection processing areas W [i] are classified into one of the first partial area w1 [i], the second partial area w2 [i], and the third partial area w3 [i].

  FIG. 7 shows an example in which a template for viewing only a specific part such as the head or leg of the pedestrian 11 is used. In this case, the first template 1210 is a template having only the head, the second template 1220 is a waist, and the third template 1230 is only a leg. Even if the three templates 1210 to 1230 are combined, a gap is left. In this case, an area that is not classified as any of the first area w1 [i], the second area w2 [i], and the third area w3 [i] occurs in the detection processing area W [i]. .

  Next, the matching rate calculation unit 1331 of the processing circuit 1300 will be described with reference to FIG. FIG. 8 is a flowchart for explaining the processing contents of the matching rate calculation unit 1331. First, in step S301, an image in the first region w1 (SX1 [i], EX1 [i], SY1 [i], EY1 [i]) is cut out from the image IMGSRC [x] [y]. The image IMMGC1 [i] [x] [y] of the area w1 is generated.

  Similarly, in step S302, an image in the second region w2 (SX2 [i], EX2 [i], SY2 [i], EY2 [i]) is cut out from the image IMGSRC [x] [y], The image IMGMC2 [i] [x] [y] of the second region w2 is generated.

  In step S303, an image in the third region w3 (SX3 [i], EX3 [i], SY3 [i], EY3 [i]) is cut out from the image IMGSRC [x] [y], The image IMMGC3 [i] [x] [y] of the area w3 is generated.

  Next, in step S304, size adjustment is performed so that the image IMGMC1 [i] [x] [y] in the first region w1 and the first template 1210 have the same size. The size adjustment includes the pedestrian assumed width SW [i] and the pedestrian assumed height SH [i] used for the calculation of the first region w1, the pedestrian reference width PW of the first template 1220, and the pedestrian reference height. This is done using the PH.

  The image size is PW / SW [i] times in the horizontal direction and PH in the vertical direction so that the image IMGMC1 [i] [x] [y] in the first region has the same size as the first template 1220. / SH [i] times may be enlarged / reduced, or conversely, the template so that the first template 1220 has the same size as the image IMMGC1 [i] [x] [y] in the first region. May be enlarged / reduced by SW [i] / PW times in the horizontal direction and SH [i] / PH times in the vertical direction.

Similarly, in step S305, size adjustment is performed so that the image IMMGMC2 [i] [x] [y] in the second region w2 and the second template 1220 have the same size.
In step S306, size adjustment is performed so that the image IMMGMC3 [i] [x] [y] in the third region w3 and the third template 1230 have the same size.

  In step S307, the first matching that is the matching rate between the size normalized image IMMGC1 [i] [x] [y] of the first region w1 whose size has been adjusted and the first template 1210. The rate MATCH1 [i] is calculated. A method for calculating the matching rate will be described later.

  Similarly, in step S308, the second matching rate MATCH2 [between the size normalized image IMMGC2 [i] [x] [y] of the second region w2 whose size has been adjusted and the second template 1220. i], and in step S309, the third matching between the size normalized image IMMGC3 [i] [x] [y] of the third region w3 whose size has been adjusted and the third template 1230 is performed. The rate MATCH3 [i] is calculated.

  Next, the method for calculating the matching rate in steps S307 to S309 will be described in detail below with reference to FIGS. First, a method for detecting the pedestrian 11 using an image pattern will be described. As a method for detecting the pedestrian 11, a plurality of templates representing a pedestrian pattern are prepared, and a degree of coincidence is obtained by performing a difference accumulation calculation or a normalized correlation calculation, or a classifier such as a neural network There is a method of performing pattern recognition using.

  Whichever method is used, a source database is required as an index for determining whether or not the person is a pedestrian 11 in advance. Various pedestrian 11 patterns are stored as a database, a representative template is created therefrom, or a discriminator is generated. There are various clothes, postures, and pedestrians in the actual environment, and the conditions such as lighting and weather are different, so it is necessary to prepare a large amount of database and reduce misjudgment. .

  At this time, in the former method of preparing a template, if the omission of determination is prevented, the number of templates becomes enormous, which is not realistic. Therefore, in the present embodiment, a determination method using the latter discriminator is adopted. The size of the discriminator does not depend on the size of the source database. A database for generating a classifier is called teacher data.

  A method for determining whether or not the person is a pedestrian 11 using the classifier in the present embodiment will be described with reference to FIGS.

  First, the first matching rate calculation method will be described. FIG. 9 is a block diagram showing a first matching rate calculation method in the present embodiment. Here, in step S304 of FIG. 8, the image IMGMT1 [i] [x] [y] of the first region is enlarged PW / SW [i] times in the horizontal direction and PH / SH [i] times in the vertical direction. A case where the image data is reduced and converted into the first size normalized image IMGMT1N [i] [x] [y] will be described.

  In the present embodiment, the first matching rate MATCH1 [i] is calculated using the first discriminator 41. The first discriminator 41 calculates a matching rate based on the direction and intensity of the light / dark gradient in a plurality of small regions TP1R [n] installed at predetermined positions in the image.

  First, when the input image IMGMT1N [i] [x] [y] is input, filtering processing is performed by a filtering processing unit 411 such as a Sobel filter, for example, and an edge intensity image IMMGMAG1 [x indicating the strength of the grayscale gradient. ] [Y] and an edge direction image IMGDIR1 [x] [y] representing the direction of the light and shade gradient are output. Then, the local edge strength calculation unit 412 calculates the local edge strength TP1M [n] using the calculated strength and direction of the gray gradient in the small region TP1R [n] installed at a predetermined location of each image. .

  Specifically, referring to the value of the edge direction image IMGDIR1 [x] [y] included in the small region TP1R [n], the direction is the same as a predetermined direction defined for each small region TP1R [n]. If there are, the values of the edge intensity images IMGMAG1 [x] [y] at the corresponding positions are summed. In the present embodiment, since 20 small regions TP1R [n] are installed, 20 values of local edge strengths TP1M [0] to TP1M [19] are obtained.

  Next, the local edge determination unit 413 performs threshold processing on each local edge strength with the corresponding thresholds TP1TH [0] to TP1TH [19], and converts them into binary values of “1” or “0”. Then, the weighting unit 414 multiplies the weights TP1W [0] to TP1W [19] and outputs the result to the totaling unit 415. The summation unit 415 sums the 20 values input from the weighting unit 414 and outputs the sum as the first matching rate MATCH [i].

  Note that the first template 1210 in the present embodiment is the number, position, size, and shading of the small regions TP1R [n] for calculating the local edge strengths TP1M [0] to TP1M [19] in the classifier 41. The gradient direction, local edge strength thresholds TP1TH [0] to TP1TH [19] in the local edge determination unit 413, weights TP1W [0] to TP1W [19] in the weighting unit 414, and these parameters are determined by the discriminator 41. The teacher data is used to output “1” when the input image IMMGMC1N [i] [x] [y] is a pedestrian and “0” when it is not a pedestrian. Adjusted. The adjustment using the test data may use a machine learning means such as AdaBoost or may be performed manually.

  For example, the procedure for determining the first template 1210 using AdaBoost from teacher data of NPD pedestrians 11 and teacher data of NBG non-pedestrians is as follows. Hereinafter, a small region having information on the position, the size, and the direction of the gray gradient necessary for calculating the local edge is represented as cTP1R [m]. Here, m is the ID number of the local edge.

  First, a plurality of (for example, 1 million) small regions cTP1R [m] having different positions, sizes, and gradient directions to be determined are prepared, and local edge strength values are calculated from all teacher data, and threshold values cTP1TH [ m]. The threshold cTP1TH [m] selects a value that can best classify the pedestrian 11 teacher data and the non-pedestrian teacher data.

  Next, a weight of wPD [nPD] = 1 / 2NPD is given to each teacher data of the pedestrian 11, and a weight of wBG [nBG] = 1 / 2NBG is given to each teacher data of the non-pedestrian. Here, nPD is an ID number of pedestrian teacher data, and nBG is an ID number of non-pedestrian teacher data.

  Then, it is assumed that k = 1, and the following process is repeated. First, the weights are normalized so that the sum of the weights of all the pedestrian / non-pedestrian teacher data is 1. Next, the false detection rate cER [m] of each local edge is calculated. The false detection rate cER [m] is a result of threshold processing of the local edge strength cTP1M [m] [nPD] of the teacher data of the pedestrian 11 with the threshold cTP1TH [m] in the small region TP1R [m]. Or the result of threshold processing of the local edge strength cTP1M [m] [nBG] of the non-pedestrian teacher data with the threshold cTP1TH [m] becomes the pedestrian 11, that is, the result of the threshold processing is actually This is the total weight of different teacher data.

  After calculating the false detection rate cER [m] of all the local edges, the local edge ID Mmin that minimizes the false detection rate is selected, and the small region TP1R [k] = cTP1R [mMin] and the threshold value TP1TH [k] = cTP1TH [mMin].

  Next, the weight of each teacher data is updated. In the update, in the small region TP1R [k], the local edge strength TP1M [k] [nPD] of the teacher data of the pedestrian 11 is thresholded with the threshold TP1TH [k], and the result is the pedestrian 11 or non- The result of thresholding the local edge strength TP1M [k] [nBG] of the pedestrian teacher data with the threshold TP1TH [k] is a non-pedestrian, that is, the threshold processing result is the weight of the correct teacher data. Multiply by the coefficient BT [k] = cER [mMin] / (1−cER [mMin]).

  Repeat until k = k + 1 and k reaches a preset value (for example, 20). This is a discriminator 41 in which the small area TP1R [k] and the threshold value TP1TH [k] obtained after the end of the iterative process are automatically adjusted by AdaBoost. The weight TP1W [k] = 1 / BT [k].

  Next, a second matching rate calculation method will be described. FIG. 10 is a block diagram showing a second matching rate calculation method in the present embodiment. Here, in step S305 shown in FIG. 8, the image IMGMT2 [i] [x] [y] in the second region w2 is multiplied by PW / SW [i] in the horizontal direction and PH / SH [i] in the vertical direction. , And is converted into a second size normalized image IMGMT2N [i] [x] [y]. In the present embodiment, the second matching rate MATCH2 [i] is calculated using the second discriminator 42.

  Similar to the first discriminator 41, the second discriminator 42 includes a filtering processing unit 421, a local edge strength calculation unit 422, a local edge determination unit 423, a weighting unit 424, and a summation unit 425. The second classifier 42 is a classifier of the middle part 11B of the pedestrian 11, and the processing content is the same as that of the first classifier 41, and thus the description thereof is omitted.

  Furthermore, a third matching rate calculation method will be described. FIG. 11 is a block diagram showing a third matching rate calculation method in the present embodiment. Here, in step S306 shown in FIG. 8, the image IMGMT3 [i] [x] [y] in the third region w3 is multiplied by PW / SW [i] in the horizontal direction and PH / SH [i] in the vertical direction. , And is converted into a third size normalized image IMGMT3N [i] [x] [y]. In the present embodiment, the third matching rate MATCH3 [i] is calculated using the third discriminator 43.

  Similar to the first discriminator 41 and the second discriminator 42, the third discriminator 43 includes a filtering processing unit 431, a local edge strength calculation unit 432, a local edge determination unit 433, a weighting unit 434, and a summation unit 435. Have The third discriminator 43 is a discriminator of the lower part 11C of the pedestrian 11, and the processing content is the same as that of the first discriminator 41, and thus the description is omitted.

  The object presence determination unit 1341 compares the first matching rate MATCH1 [i] calculated by the matching rate calculation unit 1331 with the first threshold value 1211 in the storage device 1200, and the second matching rate MATCH2 [i]. Comparison with the second threshold 1221 and comparison with the third matching rate MATCH3 [i] and the third threshold 1231 are performed.

  The first matching rate MATCH1 [i] is equal to or greater than the first threshold value 1211, the second matching rate MATCH2 [i] is equal to or greater than the second threshold value 1221, and the third matching rate MATCH3 [i]. ] Is the third threshold value 1231 or more, it is determined that the pedestrian 11 exists, and when at least one matching rate falls below the threshold value, it is determined that the pedestrian 11 does not exist.

  According to the above method, since the degree of matching can be seen in a local distribution, a pattern similar to only the upper body of the pedestrian or a pattern similar to only the lower body of the pedestrian It is possible to eliminate erroneous detection as.

  Next, a method for detecting a pedestrian 13 having a low height, such as a child, for example, using this apparatus will be described with reference to FIG. FIG. 12 is a diagram illustrating a method for calculating a matching rate from a short stature pedestrian.

  The object presence determination unit 1341 uses the detection processing region selection unit 1311 to search the detection processing region W in the vertical direction without changing the size of the detection processing region W (the assumed pedestrian width SW and the assumed pedestrian height SH). With this setting, it is possible to detect the short stature pedestrian 13 which is another object having a different vertical length from the pedestrian 11 in the image.

  Specifically, the first matching rate MATCH1 [i], the second matching rate MATCH2 [i], and the third matching rate MATCH3 [i] are stored as an array, and at the same time as the above determination, The pedestrian 13 having a short height is detected using the first matching rate MATCH1 [i], the second matching rate MATCH2 [j], and the third matching rate MATCH3 [k] of the partial regions having different positions.

  FIGS. 12B to 12D show the detection processing area W [i-2c] (SX [i-2c], SY [i-2c], EX [i-2c], EY [ic]). , Detection processing area W [i-c] (SX [ic], SY [ic], EX [ic], EY [ic]]), detection processing area W [i] (SX [ i], SY [i], EX [i], EY [i]), the matching rates of the partial areas are shown.

  As shown in FIG. 12, in the vicinity of a short stature pedestrian 13 whose height is lower than the pedestrian 11 (see FIG. 3) targeted by the first template 1210, the second template 1220, and the third template 1230. Calculate the matching rate.

  As a result, when capturing the short stature pedestrian 13 under the detection processing area W detection processing area W [i-2c], as shown in FIG. 12B, the matching rate MATCH3 [ i-2c] is a high value. When the short pedestrian 13 is captured in the center of the detection processing area W [ic], the matching rate MATCH2 [ic] with the second template 1220 is as shown in FIG. High value. When the short pedestrian 13 is captured on the detection processing area W [i], the matching rate MATCH1 [i] with the first template 1210 becomes a high value as shown in FIG. 12 (d). .

  Accordingly, the detection processing area W is set so as to search in the vertical direction from the top to the bottom on the image without changing the size of the detection processing area W (the estimated pedestrian width SW and the assumed pedestrian height SH). In this case, the height of the pedestrian (adult) 11 assumed to be detected by the first template 1210, the second template 1220, and the third template 1230 is PHE [m], and the short stature pedestrian (children to be detected) ) Let 13's height be CHE [m]. Then, c = (PHE−CHE) × SH / (2 × PHE) is calculated, and in the detection processing area W [i] (SX [i], SY [i], EX [i], EY [i]) First matching rate MATCH1 [i], detection processing area W [ic] (SX [ic], SY [ic], EX [ic], EY [ic]) Second matching rate MATCH2 [ic] and detection processing area W [i-2c] (SX [i-2c], SY [i-2c], EX [i-2c], EY [i-2c]) By using the third matching rate MATCH3 [i-2c] in FIG. 3 to determine the presence of an object, it is possible to detect the presence of the short stature pedestrian 13.

  Therefore, it is possible to detect the short stature pedestrian (child) 13 whose height is lower than the pedestrian (adult) 11 assumed by the prepared template without preparing a template dedicated to the short stature pedestrian. . Similarly, a tall pedestrian having a height higher than that of the pedestrian (adult) assumed by the template can be detected without preparing a special template.

  Using the low beam determination result LBST, the object presence determination unit 1341 can change the determination threshold for the matching rate between when the pedestrian is detected in a low beam state where the surrounding is dark and when it is not.

  A method for changing the threshold for determining the matching rate when the headlight of the host vehicle is a low beam will be described with reference to FIG. FIG. 13A shows an example in which the matching rate of the pedestrian 11 in the image 20 taken in the daytime is calculated, and FIG. 13B shows the pedestrian 11 in the image 20 taken in the low beam state at night. This is an example of calculating the matching rate.

  As shown in FIG. 13 (b), when traveling in the low beam state at night, the pedestrian 11 in front of the host vehicle is projected brightly only at the feet (legs) existing in the irradiation range H. Accordingly, when the matching rate is calculated with the image 20 as it is, the partial regions w1 and w2 that are the upper partial regions are matched with the matching rate MATCH1 [i] between the partial region w1 and the first template 1210 and the partial regions w2 and the second region w2. The matching rate MATCH2 [i] with the second template 1220 becomes low.

  Therefore, in such a case, the object presence determination unit 1341 uses the low beam determination result LBST from the low beam determination unit 1340, for example, sets the coefficient “0.5” to the first threshold (upper determination threshold) 1211, the first The second threshold value (upper determination threshold value) 1221 is multiplied by a coefficient “0.7” to change the first threshold value 1211 and the second threshold value 1221 to values lower than those in the non-low beam state. Then, the third threshold value (lower determination threshold value) 1231 is multiplied by a coefficient “1.2”, and the third threshold value 1231 is changed to a higher value than that in the non-low beam state to perform the object presence determination.

  Further, the object presence determination unit 1341 may change the threshold of the matching rate according to the position information of the object 10 measured using the object position detection unit 1310. As an example, a method for changing the threshold of the matching rate according to whether or not the object 10 is moving in the horizontal direction will be described with reference to FIG.

  FIG. 14 shows an example of the pedestrian 11 moving in the lateral direction. As described in the detection processing region selection unit 1311, when the pedestrian 11 is moving laterally, the image position 20 on the image 20 acquired by the object position detection unit 1310 due to communication delay or attachment error in the object detection device 1000. The object positions (BSX [b], BSY [b], BEX [b], BEY [b]) are shifted from the position of the pedestrian 11 on the image 20.

  In particular, when the object 10 is moving in the horizontal direction, it is not included in the range for setting the detection processing area W (BSX2 [b], BSY2 [b], BEX2 [b], BEY2 [b]). Can happen.

  In this case, if the first template 1210 is a template for the right half of the pedestrian 11 and the second template 1220 is a template for the left half of the pedestrian 11, the right half of the pedestrian 11 moves to the right. Since 11R does not enter the detection processing area W, the first matching rate MATCH1 [i] relating to the template 1210 of the right half is small, and when moving to the left, the left half does not enter the detection processing area W, so the left half The second matching rate MATCH2 [i] related to the template 1220 is reduced.

  Therefore, when the lateral movement speed VX [b] is calculated from the lateral position PX [b] of the object 10 obtained from the object position detection unit 1310, and the absolute value exceeds a predetermined speed threshold value, the lateral movement is performed. If it is determined that it is walking to the right side from the sign of VX [b], it is multiplied by a coefficient that decreases the first threshold value 1211 related to the first template 1210 that is the right side template, and If it is determined that the object is walking, the object is obtained by multiplying the first threshold value 1211 and the second threshold value 1221 by multiplying by a coefficient that reduces the second threshold value 1221 related to the second template 1220 that is the left template. Perform existence determination. The coefficient may be a constant value such as 0.8, for example, using a function such that the lateral speed is 1.0 from 0 to the threshold of the lateral speed, and the value approaches 0 as the speed increases. Also good.

  According to the object detection apparatus 1000 described above, the detection processing area W for detecting the pedestrian 11 is divided in the vertical direction and is divided into three partial areas w1, w2, and w3. Then, the first template 1210 for detecting the upper part 11A of the pedestrian 11, the second template 1220 for detecting the middle part 11B of the pedestrian 11, and the third template 1230 for detecting the leg part 11C of the pedestrian 11 are used. Thus, the matching rates MATCH1, MATCH2, and MATCH3 with the partial regions w1, w2, and w3 are calculated. And it is judged whether each matching rate MATCH1, MATCH2, MATCH3 is larger than each determination threshold value 1211, 1221, 1231, and each matching rate MATCH1, MATCH2, MATCH3 is larger than each determination threshold value 1211, 1221, 1231. When it is determined that it is a pedestrian 11. Accordingly, it is possible to eliminate erroneous detection for a pattern that is similar only to the upper body of the pedestrian 11 or a pattern that is similar to only the lower body of the pedestrian 11.

  Moreover, according to the object detection apparatus 1000, when detecting the pedestrian 11, the matching rate MATCH1 with the first template 1210 calculated at different positions, the matching rate MATCH2 with the second template 1220, and the third The matching rate MATCH3 with the template 1230 can be variously combined.

  For example, the detection processing area W is set to be searched in the vertical direction, and the first matching rate MATCH1 [i], the second matching rate MATCH2 [i], and the third matching rate MATCH3 [i] are stored as an array. In addition, the first matching rate MATCH1 [i], the second matching rate MATCH2 [j], and the third matching rate MATCH3 [k] calculated at different vertical positions can be combined. Therefore, a child whose height is lower than that of an adult can be detected easily and appropriately only by using an adult template without preparing a template dedicated to the child.

  Further, the first threshold value 1211, the second threshold value 1221, and the third threshold value 1231 are multiplied by a coefficient according to the result of the low beam determination unit 1340, and the determination threshold value is changed in the low beam state and other than the low beam state. Even when only the legs of the pedestrian 11 are visible brightly, the pedestrian 11 can be detected appropriately.

  Further, the first template 1210 is used as a template for detecting the right half of the pedestrian 11 and the second template 1220 is used as a template for detecting the left half of the pedestrian 11, and the object 10 is detected using information from the object position detection unit 1310. Is determined to be laterally moving, the first threshold value 1211 and the second threshold value 1221 are multiplied by a coefficient, and the determination threshold value is changed between when the lateral movement is not performed and when the lateral movement is not performed. Thus, even if the position detected by the object position detection unit 1310 is delayed from the position on the image 20, the pedestrian 11 can be detected appropriately.

  The classifiers 41, 42, and 43 used for detecting the pedestrian 11 are not limited to the methods taken up in this embodiment. Template matching using normalized correlation, a neural network classifier, a support vector machine classifier, a Bayes classifier, or the like may be used. Further, the number of templates is not limited to three in the present embodiment, and may be two templates or four or more templates. Further, as described in the object presence determination unit 1341, the processing circuit 1300 can be used in a form that does not use the object position detection unit 1310 or the low beam determination unit 1340.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, the case where the pedestrian 11 is used as the detection target object 10 has been described as an example, but other objects such as a preceding vehicle may be used. In the above-described embodiment, the case where the detection processing area W is decomposed into the three partial areas w1, w2, and w3 to detect the pedestrian 11 is described as an example. For example, it may be two, or four or more.

The block diagram of the object detection apparatus in this Embodiment. The schematic diagram which shows a 1st template-a 3rd template. The schematic diagram which shows the example which sets an object position. The schematic diagram which shows the example which sets a process area | region. The schematic diagram which shows the other example which sets a process area | region. The schematic diagram which shows the example which divides | segments a process area | region into several partial area | regions. The schematic diagram which shows the other example which divides | segments a process area | region into several partial area | regions. The flowchart which shows the processing content of a matching rate calculation part. The block diagram which shows the determination method of the pedestrian using a 1st template. The block diagram which shows the pedestrian determination method using a 2nd template. The block diagram which shows the determination method of the pedestrian using a 3rd template. The figure explaining the method of determining a short stature pedestrian. The figure explaining the example which detects a pedestrian in the state of a low beam. The figure explaining the example which detects the pedestrian who is moving to the horizontal direction ahead of the own vehicle.

Explanation of symbols

1000 Object detection device 1100 Image acquisition device 1200 Storage device 1210 First template 1211 First threshold 1220 Second template 1221 Second threshold 1230 Third template 1221 Third threshold 1300 Processing circuit 1310 Object position detection unit 1311 Processing region selection unit 1321 Region division unit 1331 Matching rate calculation unit 1340 Low beam determination unit 1341 Object presence determination unit

Claims (5)

An object detection device for detecting a preset object from within an image,
A detection processing region selection unit for selecting a detection processing region for detecting the object in the image;
A region dividing unit that divides the detection processing region selected by the detection processing region selection unit into a plurality of partial regions;
A matching rate calculation unit that calculates a matching rate with each of the partial regions using a template that is set for each of the plurality of partial regions and detects a part of the object;
An object presence determination unit that determines whether or not the object is present in the image based on each matching rate calculated by the matching rate calculation unit;
The detection processing area selection unit selects a plurality of detection processing areas in the vertical direction in the image,
The region dividing unit divides the detection processing region in the vertical direction into the plurality of partial regions,
The object presence determination unit uses a matching ratio of partial areas whose positions are different from each other in the vertical direction in the plurality of detection processing areas, so that another object having a vertical length different from the object is included in the image. An object detection device that determines whether or not it exists. An object detection device for detecting a preset object from within an image,
A detection processing region selection unit for selecting a detection processing region for detecting the object in the image;
A region dividing unit that divides the detection processing region selected by the detection processing region selection unit into a plurality of partial regions;
A matching rate calculation unit that calculates a matching rate with each of the partial regions using a template that is set for each of the plurality of partial regions and detects a part of the object;
An object presence determination unit that determines whether or not the object exists in the image based on each matching rate calculated by the matching rate calculation unit;
A low beam determination unit that determines whether or not the vehicle is running in a low beam state ,
The region dividing unit divides the detection processing region into an upper partial region and a lower partial region,
The object presence determination unit is set in advance for an upper determination threshold set in advance for the upper partial region and in the lower partial region when the low beam determination unit determines that the low beam state is present. An object detection apparatus that performs object determination by adjusting a lower determination threshold. The object existence determination unit performs object determination by changing the upper determination threshold to a lower value than that in a non-low beam state when the low beam determination unit determines that the low beam state is present. 2. The object detection apparatus according to 2. The object existence determination unit performs object determination by changing the lower determination threshold to a higher value than that in the non-low beam state when the low beam determination unit determines that the low beam state is present. The object detection apparatus according to 2 or 3 . An object detection device for detecting a preset object from within an image,
A detection processing region selection unit for selecting a detection processing region for detecting the object in the image;
A region dividing unit that divides the detection processing region selected by the detection processing region selection unit into a plurality of partial regions;
A matching rate calculation unit that calculates a matching rate with each of the partial regions using a template that is set for each of the plurality of partial regions and detects a part of the object;
An object presence determination unit that determines whether or not the object exists in the image based on each matching rate calculated by the matching rate calculation unit;
An object position detection unit for detecting the position of the object,
The area dividing unit calculates a lateral movement speed of the object based on a detection signal from the object position detection unit, and the detection process is performed when the lateral movement speed exceeds a preset speed threshold value. Divide the area in the left and right direction into left partial area and right partial area,
The object presence determination unit sets a first threshold value that is set in advance for a partial region located on the front side in the movement direction of the object among the left partial region and the right partial region, and the lateral movement speed is the speed. An object detection apparatus that performs object determination by changing to a lower value than when it is equal to or less than a threshold value.

Images