JP2018197945A - Obstacle detection apparatus and obstacle detection method - Google Patents

Abstract

To improve detection accuracy of an obstacle.SOLUTION: An obstacle detection device according to an embodiment includes a first detection unit, a reference histogram calculation unit, a generation unit, a histogram calculation unit, a likelihood calculation unit, a weighting unit, and a second detection unit. The first detection unit detects an obstacle detection area from a first frame image. The reference histogram calculation unit calculates a reference hue histogram in the detection area. The generation unit generates a plurality of particles on a second frame image after the first frame image based on the detection area. The histogram calculation unit calculates a hue histogram within a designated range provided for each of the plurality of particles. The likelihood calculation unit calculates likelihood of the reference hue histogram and the hue histogram. The weighting unit weights each of the plurality of particles based on the likelihood. The second detection unit detects an obstacle from the second frame image based on the plurality of weighted particles.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an obstacle detection device and an obstacle detection method.

  Conventionally, there is an obstacle detection device that detects an obstacle such as a pedestrian from a frame image captured by an imaging device. Such an obstacle detection device detects an obstacle based on a dictionary in which various shapes of obstacles such as a person are learned and a frame image (see, for example, Patent Document 1).

JP 2001-351193 A

  However, the obstacle detection device described above has room for further improvement in terms of improving the obstacle detection accuracy. This is because the obstacle detection apparatus may not be able to detect an obstacle depending on the shape of the obstacle, the background of the obstacle, or occlusion.

  The present invention has been made in view of the above, and an object of the present invention is to provide an obstacle detection device and an obstacle detection method capable of improving the detection accuracy of an obstacle.

  An obstacle detection device according to an aspect of an embodiment includes a first detection unit, a reference histogram calculation unit, a generation unit, a histogram calculation unit, a likelihood calculation unit, a weighting unit, and a second detection unit. . The first detection unit detects a detection region where an obstacle exists from the first frame image captured by the imaging device. The reference histogram calculation unit calculates a reference hue histogram in the detection area detected by the first detection unit. The generation unit generates a plurality of particles on the second frame image captured after the first frame image based on the detection region. The histogram calculation unit calculates a hue histogram within a specified range provided for each of a plurality of particles. The likelihood calculating unit calculates the likelihood between the reference hue histogram calculated by the reference histogram calculating unit and the hue histogram calculated by the histogram calculating unit. The weighting unit weights each of the plurality of particles based on the likelihood calculated by the likelihood calculating unit. The second detection unit detects an obstacle from the second frame image based on the plurality of particles weighted by the weighting unit.

  According to one aspect of the embodiment, the obstacle detection accuracy can be improved.

FIG. 1 is a diagram illustrating an obstacle detection method according to the embodiment. FIG. 2 is a block diagram of the obstacle detection apparatus according to the embodiment. FIG. 3 is a diagram for explaining the detection range according to the embodiment. FIG. 4 is a diagram illustrating an example of a certain range according to the embodiment. FIG. 5 is a diagram illustrating a hue histogram calculation example by the histogram calculation unit according to the embodiment. FIG. 6 is a diagram for explaining an example of likelihood calculation by the likelihood calculating unit according to the embodiment. FIG. 7 is a diagram for explaining the weighting coefficient according to the embodiment. FIG. 8 is a diagram illustrating an example of obstacle detection by the second detection unit according to the embodiment. FIG. 9 is a flowchart illustrating a processing procedure executed by the obstacle detection apparatus according to the embodiment. FIG. 10 is a flowchart illustrating a processing procedure executed by the detection interpolation unit according to the embodiment. FIG. 11A is a diagram for describing a moving speed and a moving direction of an obstacle detected by the first detection unit according to Modification 1 of the embodiment. FIG. 11B is a diagram illustrating an example of particles generated by the generation unit according to Modification 1 of the embodiment. FIG. 12 is a diagram for explaining a certain range of the second modification according to the embodiment. FIG. 13 is a diagram illustrating another example of the certain range according to the second modification of the embodiment. FIG. 14 is a diagram illustrating another example of the certain range according to the second modification of the embodiment. FIG. 15 is a diagram illustrating a configuration of an obstacle detection apparatus according to Modification 3 of the embodiment. FIG. 16 is a diagram illustrating a generation example of particles according to the third modification of the embodiment. FIG. 17 is a diagram illustrating another generation example of particles according to the third modification of the embodiment. FIG. 18 is a diagram illustrating a configuration of an obstacle detection apparatus according to Modification 4 of the embodiment. FIG. 19 is a flowchart illustrating a processing procedure executed by the obstacle detection device according to the fifth modification of the embodiment.

  Hereinafter, an obstacle detection device and an obstacle detection method according to embodiments will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by this embodiment.

  In the following, a case where the obstacle detection device detects a person (hereinafter referred to as a pedestrian P) as an obstacle from frame images continuously taken by an imaging device (vehicle-mounted imaging device) mounted on the host vehicle will be described. To do. The obstacle detected by the obstacle detection device is not limited to the pedestrian P, and may be a vehicle or a sign, for example.

  Moreover, below, in order to simplify description, although the case where an obstacle detection apparatus detects one pedestrian P is demonstrated, you may detect several pedestrians P. FIG. When a plurality of pedestrians P are detected, the obstacle detection device executes the obstacle detection method for each pedestrian P.

(1. Obstacle detection method)
First, the obstacle detection method according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an obstacle detection method according to the embodiment. This obstacle detection method is executed by the obstacle detection apparatus 1 described later with reference to FIG.

  As shown in FIG. 1, in the obstacle detection method according to the embodiment, the obstacle detection apparatus 1 includes an area where the shape of a person who is an obstacle exists from the frame image F (hereinafter referred to as an obstacle detection area M). Is detected (step S1). For example, the obstacle detection apparatus 1 detects an obstacle detection region M based on machine learning. Specifically, the obstacle detection apparatus 1 detects an obstacle detection region M from the frame image F using person dictionary information that is a reference for determining whether or not the object shown in the frame image F is a pedestrian P. .

  Here, in the conventional obstacle detection method, for example, when the pedestrian P is hidden behind the obstacle S as shown in FIG. In this case, the pedestrian P will be overlooked. For this reason, the conventional obstacle detection method has room for improvement in terms of improving the detection of obstacles.

  Therefore, in the obstacle detection method according to the embodiment, when the obstacle detection region M cannot be detected by machine learning, the obstacle detection device 1 uses the particle filter to detect the obstacle detection region M in the undetected frame image Fn. It was decided to detect.

  The undetected frame image Fn is a frame image F in which the obstacle detection area M is not detected by machine learning. Here, it is assumed that the frame image F in which the obstacle detection area M is detected is a detected frame image Fd and is captured by the imaging device before the undetected frame image Fn.

  In FIG. 1, the frame image F captured at time t0 is set as a detection frame image Fd where the obstacle detection area M is detected, and the frame detection image M detected at the time t1 is detected by the obstacle detection area M. The undetected frame image Fn is not made.

  In the obstacle detection method according to the embodiment, the obstacle detection apparatus 1 first calculates a hue histogram (hereinafter referred to as a reference hue histogram) in the detection region M detected by the detection frame image Fd (step S2). For example, in FIG. 1, the obstacle detection apparatus 1 calculates a reference hue histogram within a detection range M1 that is a part of the detection region M. The reference hue histogram is a histogram representing the frequency for each hue of each pixel in the detection range M1. Details of the detection range M1 will be described later with reference to FIG.

  Next, the obstacle detection device 1 generates a plurality of particles Pa on the undetected frame image Fn based on the detection region M (step S3). In FIG. 1, the particles Pa are indicated by black circles. For example, the particles Pa are randomly generated in and around the detection region M.

  The obstacle detection device 1 calculates a hue histogram for each of the plurality of particles Pa (step S4). Details of the method of calculating the hue histogram for each particle Pa will be described later with reference to FIG.

  The obstacle detection device 1 calculates the likelihood of the reference hue histogram calculated in step S2 and the hue histograms of the plurality of particles Pa calculated in step S4 for each particle Pa (step S5). For example, the obstacle detection device 1 calculates the likelihood of each of the plurality of particles Pa by calculating the Bhattacharya coefficient.

  Next, the obstacle detection apparatus 1 weights each of the plurality of particles Pa based on the calculated likelihood (step S6). For example, the obstacle detection device 1 performs weighting by multiplying the particle Pa as a weighting coefficient by using the Bhattacharya coefficient calculated in step S5. In FIG. 1, the weighting coefficient is indicated by a black circle indicating the particle Pa.

  Then, the obstacle detection device 1 detects an obstacle detection area M3 including an obstacle from the undetected frame image Fn based on the plurality of weighted particles Pa (step S7). For example, the obstacle detection device 1 detects a region including the center of gravity of the weighted particle Pa as the detection region M3.

  That is, in the obstacle detection method according to the embodiment, the obstacle detection apparatus 1 detects the obstacle detection region M3 based on the particle filter for the undetected frame image Fn in which the obstacle detection region M is not detected. To do. Thereby, even when an obstacle cannot be detected by machine learning, the obstacle detection apparatus 1 can detect the obstacle detection area M3, and the detection of the obstacle can be interpolated.

  Thereby, in the obstacle detection method according to the embodiment, even when the pedestrian P cannot be detected from the frame image F based on machine learning, the obstacle detection device 1 determines the position of the pedestrian P based on the particle filter. Can be detected.

  Therefore, according to the obstacle detection method according to the embodiment, the obstacle detection accuracy can be improved.

(2. Obstacle detection system)
Next, the configuration of the obstacle detection apparatus 1 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram of the obstacle detection system according to the embodiment. The obstacle detection system includes an obstacle detection device 1, an imaging device 5, and a vehicle control device 6.

  In FIG. 2, constituent elements necessary for explaining the characteristics of the embodiment are represented by functional blocks, and descriptions of general constituent elements are omitted.

  In other words, each component illustrated in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the specific form of distribution / integration of each functional block is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed in arbitrary units according to various loads or usage conditions.・ It can be integrated and configured.

(2.1. Imaging device)
The imaging device 5 includes an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), for example, and the surroundings (for example, the front) of the host vehicle in a predetermined cycle (for example, 1/30 second cycle). Take an image. The frame images F captured by the imaging device 5 are sequentially output to the obstacle detection device 1.

(2.2. Vehicle control device)
The vehicle control device 6 performs vehicle control such as PCS (Pre-crash Safety System) and AEB (Advanced Emergency Braking System) based on the obstacle detection result by the obstacle detection device 1.

(2.3. Obstacle detection device)
As shown in FIG. 2, the obstacle detection device 1 is connected to an imaging device 5 and a vehicle control device 6.

  The obstacle detection device 1 includes a control unit 2 and a storage unit 3. The control unit 2 includes an acquisition unit 21, a first detection unit 22, a detection interpolation unit 23, and an output unit 24. The control unit 2 includes, for example, a computer having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), an input / output port, and various circuits.

  The CPU of the computer functions as the acquisition unit 21, the first detection unit 22, the detection interpolation unit 23, and the output unit 24 of the control unit 2, for example, by reading and executing a program stored in the ROM.

  In addition, at least one or all of the acquisition unit 21, the first detection unit 22, the detection interpolation unit 23, and the output unit 24 of the control unit 2 are made up of ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or the like. It can also be configured with hardware.

  The storage unit 3 corresponds to, for example, a RAM or an HDD. The RAM and HDD store, for example, person dictionary information 31, vehicle dictionary information 32, and various programs.

  The obstacle detection apparatus 1 may acquire the above-described program and various information via another computer or a portable recording medium connected via a wired or wireless network.

  The person dictionary information 31 is information serving as a reference for determining whether or not the object shown in the frame image F is a pedestrian P, and is created in advance by machine learning. For example, a predetermined number (for example, several hundred to several tens of images) of images of a plurality of types of people whose shapes are known and images of objects other than people are prepared as learning data.

  Subsequently, for example, an HOG (Histogram of Gradient) feature amount is extracted from each prepared image. Then, the previously prepared image is plotted on a two-dimensional plane based on the extracted HOG feature amount.

  Subsequently, for example, a separation line that separates an image of a person and an image of an object other than a person on a two-dimensional plane is generated by a discriminator such as SVM (Support Vector Machine). The information of the separation axis generated by the coordinate axes of the two-dimensional plane and the discriminator becomes the person dictionary information 31.

  Note that the feature amount extracted from the image prepared in advance is not limited to the HOG feature amount, and may be a SIFT (Scale Invariant Feature Transform) feature amount. Further, the discriminator used for separating a human image and an image of an object other than a human is not limited to the SVM, and may be a discriminator such as AdaBoost.

  The vehicle dictionary information 32 is information serving as a reference for determining whether or not the object shown in the frame image F is a vehicle, and is created in advance by machine learning in the same manner as the person dictionary information 31.

(2.3.1. Acquisition unit)
The acquisition unit 21 acquires the frame image F captured by the imaging device 5. The acquisition unit 21 sequentially outputs the acquired frame image F to the first detection unit 22 and the detection interpolation unit 23.

(2.3.2. First detection unit)
The first detection unit 22 detects an obstacle included in the frame image F input from the acquisition unit 21 by machine learning.

  For example, when the obstacle is a pedestrian P, the first detection unit 22 detects the obstacle detection area M using the person dictionary information 31 stored in the storage unit 3 for the frame image F.

  The first detection unit 22 superimposes the detected obstacle detection area M on the frame image F and outputs it to the detection interpolation unit 23 and the output unit 24.

  When the obstacle is a vehicle, the first detection unit 22 detects the obstacle detection area M using the vehicle dictionary information 32 stored in the storage unit 3. As described above, the first detection unit 22 detects the obstacle detection area M for each type using the dictionary information corresponding to the type of the obstacle.

(2.3.3. Detection Interpolation Unit)
The detection interpolation unit 23 detects an obstacle detection area M from the undetected frame image Fn in order to interpolate obstacle detection when the first detection unit 22 cannot detect the obstacle. For example, when the frame image F on which the detection region M is not superimposed is input, the detection interpolation unit 23 assumes that the first detection unit 22 has not detected an obstacle, and detects the frame image F as an undetected frame image. An obstacle is detected as Fn.

  As shown in FIG. 2, the detection interpolation unit 23 includes a reference histogram calculation unit 231, a generation unit 232, a histogram calculation unit 233, a likelihood calculation unit 234, a weighting unit 235, and a second detection unit 236. Prepare.

(Reference histogram calculation unit)
The reference histogram calculation unit 231 calculates the hue histogram of the obstacle included in the detection frame image Fd as the reference hue histogram according to the obstacle detection area M. The reference histogram calculation unit 231 outputs the calculated reference hue histogram to the likelihood calculation unit 234.

  As shown in FIG. 3, it is assumed that the first detection unit 22 detects an obstacle detection region M including a pedestrian P from the detection frame image Fd. The obstacle detection area M is a rectangular area having a height H and a width W.

  At this time, the reference histogram calculation unit 231 calculates a hue histogram in the detection range M1 smaller than the obstacle detection area M as the reference hue histogram Hh. FIG. 3 is a diagram for explaining the detection range M1 according to the embodiment.

  As shown in FIG. 3, the detection range M <b> 1 is a rectangular region having a height H / 2 and a width W, for example, and is located in the center region of the detection region M. In other words, the lower end side of the detection range M1 is located at a height H / 4 from the lower end side of the detection region M.

  As described above, the reference histogram calculation unit 231 calculates the reference hue histogram Hh in the detection range M1 smaller than the detection region M, so that the influence of the background included in the reference hue histogram Hh can be further reduced.

  This is because when the obstacle is the pedestrian P, the center area of the detection area M includes the trunk of the pedestrian P, so the background area is smaller than the other areas.

  For example, when the obstacle is a vehicle, the area below the detection area M may be set as the detection range M1. This is because the body body is often the same color, and is less affected by the reflection than the window glass. Therefore, by setting the lower area of the detection area M that includes a large amount of the vehicle body as the detection area M1, it is possible to further reduce the influence of background and landscape reflection on the reference hue histogram Hh.

  As described above, the reference histogram calculation unit 231 may calculate the reference hue histogram Hh in the detection range M1 corresponding to the type of obstacle. As a result, it is possible to reduce the influence of the background and scenery reflection on the reference hue histogram Hh.

  Although the detection range M1 is a rectangular area here, the present invention is not limited to this. As the shape of the detection range M1, various shapes such as a trapezoid, a circle, or a shape along the outline of an obstacle can be adopted.

(Generator)
Returning to FIG. The generation unit 232 generates a plurality of particles Pa on the undetected frame image Fn based on the detection range M1. The generation unit 232 generates a plurality of particles Pa at random positions within the detection range M1 and around the detection range M1.

  Here, the detection range M1 for generating the particle Pa is a range in which the detection range M1 detected from the detection frame image Fd is superimposed on the undetected frame image Fn. Hereinafter, the detection range M1 is referred to as a detection range M1 of the undetected frame image Fn. Since the detection range M1 is a part of the detection region M, it can be said that the generation unit 232 generates particles Pa in the detection region M of the undetected frame image Fn and in the vicinity of the detection region M.

  As described above, the generation unit 232 generates the particles Pa in the detection area M and the vicinity of the detection area M of the undetected frame image Fn, so that the detection interpolation unit 23 detects an obstacle existing around the detection area M. Can do. The generation unit 232 outputs the generated plurality of particles Pa to the histogram calculation unit 233.

(Histogram calculator)
The histogram calculation unit 233 calculates a hue histogram for each of the plurality of particles Pa. The histogram calculation unit 233 outputs the calculated hue histograms to the likelihood calculation unit 234.

  For example, the histogram calculation unit 233 calculates a hue histogram of a plurality of particles Pa included in a certain range R1 shown in FIG. FIG. 4 is a diagram illustrating an example of the certain range R1 according to the embodiment. In FIG. 4, the particles Pa are indicated by black circles.

  As shown in FIG. 4, the fixed range R1 is, for example, a circular region having a radius d centered on the center C of the detection region M of the undetected frame image Fn. Since the histogram calculation unit 233 calculates the hue histogram of the particles Pa within the certain range R1, the number of particles Pa for calculating the hue histogram can be reduced, and the processing amount of the obstacle detection apparatus 1 can be reduced. it can.

  Next, an example of a hue histogram calculated by the histogram calculation unit 233 will be described with reference to FIG. FIG. 5 is a diagram illustrating a hue histogram calculation example by the histogram calculation unit 233 according to the embodiment.

  Here, a case where the histogram calculation unit 233 calculates a hue histogram for the particle Pa1 shown in FIG. 4 will be described as an example.

  As shown in FIG. 5, the histogram calculation unit 233 calculates a hue histogram Hh1 within a specified range M2 including the particle Pa1. The designated range M2 is a range provided for each of the plurality of particles Pa.

  The designated range M2 is, for example, a rectangular region having a height H / 2 and a width W with the particle Pa1 as the center, as shown in FIG. The designated range M2 may have the same shape and size as the detection range M1 used for calculating the reference hue histogram Hh.

(Likelihood calculation unit)
Returning to FIG. The likelihood calculation unit 234 calculates the likelihood between the reference hue histogram Hh calculated by the reference histogram calculation unit 231 and the hue histogram calculated by the histogram calculation unit 233 for each particle Pa. The likelihood calculating unit 234 outputs the calculated likelihood to the weighting unit 235.

  As shown in FIG. 6, the likelihood calculating unit 234 calculates the likelihood between, for example, the reference hue histogram Hh and the hue histogram Hh1 of the particle Pa1 (see FIG. 4). The likelihood calculating unit 234 calculates, for example, a Bhattacharya coefficient as the likelihood. FIG. 6 is a diagram for explaining an example of likelihood calculation by the likelihood calculation unit 234 according to the embodiment.

  As described above, the histogram calculation unit 233 calculates the hue histogram of the particles Pa within the certain range R1. Therefore, the likelihood calculating unit 234 calculates the likelihood of the hue histogram and the reference hue histogram Hh of the particles Pa within the certain range R1.

  As described above, the likelihood calculating unit 234 calculates the likelihood of the particles Pa within the certain range R1, thereby reducing the number of particles Pa for which the likelihood is calculated. Can be reduced.

(Weighting part)
Returning to FIG. The weighting unit 235 weights the plurality of particles Pa based on the likelihood calculated by the likelihood calculating unit 234. For example, the weighting unit 235 weights the particle Pa by multiplying the particle Pa by a weighting coefficient corresponding to the likelihood. The weighting unit 235 outputs the plurality of weighted particles Pa to the second detection unit 236.

  For example, as illustrated in FIG. 7, the weighting unit 235 performs weighting of the particle Pa using a different weighting coefficient for each particle Pa. FIG. 7 is a diagram for explaining the weighting coefficient according to the embodiment.

  In FIG. 7, the particle Pa at each coordinate position of the undetected frame image Fn is indicated by a black circle. The size of the weighting coefficient is indicated by the size of a circle. That is, as the circle shown in FIG. 7 is larger, the likelihood is higher and the weighting coefficient is larger.

  For example, when the likelihood calculating unit 234 calculates a Bhattacharya coefficient, the weighting unit 235 uses the Bhattacharya coefficient as a weighting coefficient. Specifically, the likelihood calculating unit 234 performs weighting by, for example, multiplying each coordinate (X coordinate, Y coordinate) of the particle Pa by a Bhatterarya coefficient.

  For example, it is assumed that the weighting unit 235 does not weight the particles Pa outside the certain range R1. Thereby, the number of particles Pa to be weighted can be reduced, and the processing amount of the obstacle detection apparatus 1 can be reduced.

  Alternatively, the weighting unit 235 may apply a very small value (for example, 0.0001 or less) to the particle Pa outside the fixed range R1.

  As a result, the particle Pa outside the fixed range R1 can be prevented from being used for obstacle detection.

  If the undetected frame image Fn is imaged at the time t1 (see FIG. 1) next to the detected frame image Fd, the possibility that the pedestrian P moves outside the fixed range R1 at the time t1 is low. Therefore, in the obstacle detection device 1 according to the embodiment, the particle Pa outside the fixed range R1 is not used for obstacle detection. Thereby, the number of particles Pa used for obstacle detection can be reduced, and the processing amount of the obstacle detection apparatus 1 can be reduced while maintaining the accuracy of obstacle detection in the undetected frame image Fn. .

  Here, in the histogram calculation unit 233, the likelihood calculation unit 234, and the weighting unit 235, each process is performed on the particles Pa in the certain range R1, but the present invention is not limited to this.

  Processing may be performed on the particles Pa in the certain range R1 in at least a part of the histogram calculation unit 233, the likelihood calculation unit 234, and the weighting unit 235.

  Further, when the generation unit 232 generates a plurality of particles Pa, the particles Pa may be generated within the certain range R1.

(Second detection unit)
Returning to FIG. The second detection unit 236 detects an obstacle from the undetected frame image Fn based on the particles Pa weighted by the weighting unit 235.

  For example, the second detection unit 236 accumulates the particle Pa weighted by the weighting unit 235 for each coordinate, and calculates the average of the accumulated values, so that the gravity center C3 of the particle Pa is calculated as illustrated in FIG. Calculate the coordinates. FIG. 8 is a diagram illustrating an example of obstacle detection by the second detection unit 236 according to the embodiment.

  The second detection unit 236 detects the center of gravity C3 of the plurality of particles Pa as the center of gravity C3 of the obstacle in the undetected frame image Fn. The second detection unit 236 superimposes the detection region M3 centered on the center of gravity C3 on the undetected frame image Fn and outputs it to the output unit 24. For example, the detection area M3 is a rectangular area having the same size as the detection area M (see FIG. 3) detected based on the detection frame image Fd. Accordingly, the second detection unit 236 can accurately detect an obstacle from the undetected frame image Fn.

(2.3.4. Output unit)
The output unit 24 displays either the detected frame image Fd on which the detection region M detected by the first detection unit 22 is superimposed or the undetected frame image Fn on which the detection region M3 detected by the second detection unit 236 is superimposed. Output to the control device 6.

  When the output unit 24 receives the detection frame image Fd on which the detection region M is superimposed from the first detection unit 22, the output unit 24 outputs the detection frame image Fd to the vehicle control device 6.

  On the other hand, when the output unit 24 receives the detection frame image Fd on which the detection region M is not superimposed from the first detection unit 22, the output unit 24 receives the undetected frame image Fn on which the detection region M3 is superimposed from the second detection unit 236. To the vehicle control device 6. Alternatively, the output unit 24 outputs the undetected frame image Fn on which the detection region M3 is superimposed to the vehicle control device 6 when the detected frame image Fd is not received.

  Accordingly, even when the first detection unit 22 does not detect the detection region M, the output unit 24 can output the undetected frame image Fn on which the detection region M3 is superimposed to the vehicle control device 6.

(3. Detection process)
Next, a processing procedure of detection processing executed by the obstacle detection device 1 according to the embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating a processing procedure executed by the obstacle detection apparatus 1 according to the embodiment. Such detection processing is repeatedly executed by the control unit 2 while the imaging device 5 is capturing the frame image F, for example.

  First, the acquisition unit 21 of the control unit 2 acquires the frame image F from the imaging device 5 (step S101). Next, the 1st detection part 22 detects an obstruction based on machine learning (step S102).

  Subsequently, the detection interpolation unit 23 of the control unit 2 determines whether or not the first detection unit 22 has detected an obstacle (step S103). For example, the detection interpolation unit 23 determines whether or not the first detection unit 22 has detected an obstacle depending on whether or not the frame image F on which the detection region M is superimposed is input from the first detection unit 22. .

  When the first detection unit 22 detects an obstacle (Yes in step S103), the output unit 24 outputs the detection frame image Fd on which the detection region M is superimposed to the vehicle control device 6 (step S104), and performs processing. finish.

  On the other hand, when the 1st detection part 22 has not detected an obstruction (No of step S103), the detection interpolation part 23 performs an obstruction detection process (step S105). The output unit 24 outputs the undetected frame image Fn on which the detection region M3 detected by the detection interpolation unit 23 is superimposed to the vehicle control device 6 (step S106).

(3.1. Obstacle detection processing)
Next, the processing procedure of the obstacle detection process executed by the detection interpolation unit 23 according to the embodiment will be described with reference to FIG. FIG. 10 is a flowchart illustrating a processing procedure executed by the detection interpolation unit 23 according to the embodiment.

  The reference histogram calculation unit 231 of the detection interpolation unit 23 calculates a reference hue histogram Hh within the obstacle detection range M1 in the detected frame image Fd before the undetected frame image Fn (step S201).

  Next, the generation unit 232 of the detection interpolation unit 23 generates a plurality of particles Pa according to the detection range M1 of the undetected frame image Fn (step S202). The histogram calculation unit 233 of the detection interpolation unit 23 calculates a hue histogram for each particle Pa (step S203).

  The likelihood calculating unit 234 calculates the likelihood between the reference hue histogram Hh and the hue histogram (step S204). The weighting unit 235 performs weighting of the particle Pa according to the likelihood calculated by the likelihood calculating unit 234 (step S205).

  The second detection unit 236 detects the center of gravity C3 of the obstacle based on the weighted particle Pa and determines the detection region M3 (step S206).

  Subsequently, the detection interpolation unit 23 determines whether or not an obstacle has been detected based on all the obstacles detected in the detection frame image Fd, that is, the detection region M (step S207). If no obstacle is detected based on all the detection areas M (No in step S207), the process returns to step S201.

  On the other hand, when an obstacle is detected based on all the detection areas M (Yes in step S207), the process ends.

  As described above, the obstacle detection apparatus 1 according to the embodiment includes the first detection unit 22, the reference histogram calculation unit 231, the generation unit 232, the histogram calculation unit 233, the likelihood calculation unit 234, and the weighting unit. 235 and a second detection unit 236.

  The first detection unit 22 detects a detection region M including an obstacle from the first frame image (detection frame image Fd) captured by the imaging device 5. The reference histogram calculation unit 231 calculates a reference hue histogram Hh in the detection area M detected by the first detection unit 22.

  Based on the detection region M, the generation unit 232 generates a plurality of particles Pa on the second frame image (undetected frame image Fn) captured after the first frame image (detected frame image Fd) by the imaging device 5. The histogram calculation unit 233 calculates a hue histogram in the designated range M2 provided for each of the plurality of particles Pa.

  The likelihood calculating unit 234 calculates the likelihood between the reference hue histogram Hh calculated by the reference histogram calculating unit 231 and the hue histogram Hh1 calculated by the histogram calculating unit 233. The weighting unit 235 weights each of the plurality of particles Pa based on the likelihood calculated by the likelihood calculating unit 234. The second detection unit 236 detects an obstacle from the second frame image (undetected frame image Fn) based on the plurality of particles Pa weighted by the weighting unit 235.

  Therefore, the obstacle detection apparatus 1 according to the embodiment can interpolate the detection of the obstacle even when the obstacle cannot be detected by machine learning, for example, and can improve the detection accuracy of the obstacle. .

(4. Modifications)
The obstacle detection apparatus 1 shown in FIG. 2 is an example, and various modifications can be made.

(4.1. Modification 1)
In the above embodiment, the obstacle detection apparatus 1 generates the particle Pa in the detection area M and the vicinity of the detection area M of the undetected frame image Fn. However, the present invention is not limited to this. When the first detection unit 22 detects the moving speed and moving direction of the obstacle, the generating unit 232 may generate the particle Pa using at least one of the moving speed and moving direction.

  This point will be described with reference to FIGS. 11A and 11B. FIG. 11A is a diagram for describing a moving speed and a moving direction of an obstacle detected by the first detection unit 22 according to Modification 1 of the embodiment. FIG. 11B is a diagram illustrating an example of a particle Pa generated by the generation unit 232 according to the first modification of the embodiment.

  First, the point that the first detection unit 22 detects the moving speed and moving direction of the obstacle will be described. The first detection unit 22 determines whether or not the detection area M detected for the temporally continuous frame images F is the detection area M corresponding to the same pedestrian P. Then, when the detection area M corresponding to the same pedestrian P continues for a plurality of frames (for example, 5 frames) or more, the first detection unit 22 assigns a person ID to the obstacle detection area M. In other words, when the detection area M corresponding to the same pedestrian P is not continuously detected, no person ID is given to the obstacle detection area M.

  Thereby, even if the 1st detection part 22 is a case where the noise reflected in the frame image F is detected accidentally as an obstacle, the detection area M of this obstacle is from the alerting | reporting object to the vehicle control apparatus 6, for example. Can be excluded. In other words, the obstacle detection device 1 can output only information based on the obstacle detection region M with high accuracy to the vehicle control device 6.

  Note that the frame image F to which the person ID is assigned becomes the detection frame image Fd (see FIG. 1). Then, the first detection unit 22 outputs a detection frame image Fd in which the obstacle detection area M in the frame image F assigned with the person ID is superimposed to the detection interpolation unit 23.

  Thus, the obstacle detection area M output to the detection interpolation unit 23 is detected by the first detection unit 22 over a plurality of frames. Therefore, the first detection unit 22 can detect the moving speed and moving direction of the obstacle, as shown in FIG. 11A, based on the transition of the detection area M in a plurality of continuous frame images F. In FIG. 11A, the magnitude of the vector V indicates the moving speed, and the direction indicates the moving direction.

  The generation unit 232 according to the modification 1 generates the particle Pa moved from the detection region M of the detection frame image Fd according to at least one of the movement speed and the movement direction of the obstacle detected by the first detection unit 22. .

  Specifically, in the generation unit 232, first, the first detection unit 22 randomly generates a plurality of particles Pa in and around the detection region M of the detection frame image Fd. Thereafter, the generation unit 232 moves the generated plurality of particles Pa according to the vector V. For example, the generation unit 232 moves the particle Pa0 illustrated in FIG. 11B to the position of the particle Pa3 according to the vector V. The generation unit 232 outputs the moved particle Pa to the histogram calculation unit 233.

  Alternatively, the generation unit 232 may generate a plurality of particles Pa based on a region M4 obtained by moving the detection region M detected by the first detection unit 22 according to the vector V.

  That is, the generation unit 232 may generate a plurality of particles Pa at random positions in and around the region M4 in which the detection region M is moved according to the vector V.

  Further, when the generation unit 232 generates a plurality of particles Pa based on the moving direction, the generation unit 232 moves the detection region M or the generated particle Pa with the magnitude of the vector V as a predetermined value. . Note that the size of the vector V may be changed according to, for example, the type of the detected obstacle.

  Further, when the generating unit 232 generates a plurality of particles Pa based on the moving speed, the generating unit 232 moves the generated particles Pa in a random direction by a distance corresponding to the moving speed. Alternatively, the generation unit 232 may generate a plurality of particles Pa in the range around the detection region M and according to the moving speed.

  As described above, the generation unit 232 according to the first modification example uses the particles moved from the detection region M of the detection frame image Fd according to at least one of the movement speed and the movement direction of the obstacle detected by the first detection unit 22. Pa is generated. Thereby, particle | grains Pa can be produced | generated in the area | region after an obstruction moved, and the obstruction detection accuracy of the obstruction detection apparatus 1 can further be improved.

(4.2. Modification 2)
In the above embodiment, the obstacle detection apparatus 1 uses the particle Pa within the fixed range R1 of the radius d centered on the center C of the detection region M for obstacle detection of the undetected frame image Fn. It is not limited to.

  The obstacle detection device 1 may detect an obstacle from the undetected frame image Fn using the particles Pa within a certain range corresponding to the center C of the detection region M. Therefore, for example, as shown in FIG. 12, particles Pa within a certain range R2 having a radius d3 centered on a position C1 above the center C of the detection region M by an arrow V2 may be used for obstacle detection. . In addition, FIG. 12 is a figure for demonstrating the fixed range R2 of the modification 2 which concerns on embodiment.

  When the obstacle is the pedestrian P, the region located above the center of the detection region M includes a large amount of the pedestrian P's body, and therefore the background region is smaller than the other regions. Therefore, the obstacle detection apparatus 1 can detect an obstacle using the particle Pa having a small influence of the background, and can further improve the detection accuracy of the obstacle.

  The fixed range R2 is not limited to the circle shown in FIG. For example, as shown in FIG. 13, the fixed range R2 may be elliptical. FIG. 13 shows a case where the fixed range R2 is an ellipse having a major axis in the horizontal direction of the detection frame image Fd. For example, the fixed range R2 is an ellipse having a major axis in the vertical direction. Also good. In this way, by using the particle Pa in the elliptical fixed range R2 for obstacle detection, the obstacle detection apparatus 1 can detect the obstacle even when the obstacle is moving. Thereby, the detection accuracy of the obstacle detection apparatus 1 can be improved.

  Or when the 1st detection part 22 detects the moving speed and moving direction of an obstruction, it is good also considering the ellipse which has a long diameter according to a moving speed and a moving direction as the fixed range R2. For example, FIG. 14 shows a case where the fixed range R2 is an ellipse having the moving direction as the long axis and the moving speed as the long radius d4. 13 and 14 are diagrams illustrating another example of the certain range R2 according to the second modification of the embodiment.

  As described above, the obstacle detection device 1 further detects the obstacle from the undetected frame image Fn based on the particles Pa within the fixed range R2 corresponding to the moving speed and the moving direction, thereby further improving the obstacle detection accuracy. Can be made.

  Here, the position C1 above the center C of the detection region M by the arrow V2 is the center of the certain range R2, but the present invention is not limited to this. For example, when the first detection unit 22 detects the vehicle as an obstacle, a position below the center C of the detection region M by a certain distance may be set as the center of the certain range R2. This is because the vehicle body is included in the lower region of the detection region M as described above.

  Thus, the obstacle detection apparatus 1 further improves the obstacle detection accuracy by detecting the obstacle from the undetected frame image Fn based on the particles Pa within the certain range R2 corresponding to the type of the obstacle. be able to.

(4.3. Modification 3)
In the above embodiment, the generation unit 232 randomly generates a plurality of particles Pa. However, the present invention is not limited to this. For example, the particle Pa generated in the frame image before the undetected frame image Fn may be used to generate the particle Pa in the undetected frame image Fn. This point will be described with reference to FIGS. 15 and 16.

  FIG. 15 is a diagram illustrating a configuration of an obstacle detection device 1a according to the third modification of the embodiment. FIG. 16 is a diagram illustrating a generation example of the particle Pa according to the third modification of the embodiment.

  The obstacle detection device 1a shown in FIG. 15 has the same configuration as the obstacle detection device 1 shown in FIG. 2 except that the generation unit 232a has a particle selection unit 232b. In FIG. 15, the same components as those shown in FIG. 2 are denoted by the same reference numerals as those shown in FIG.

  As shown in FIG. 16, it is assumed that the obstacle detection device 1a generates a plurality of particles Pa for the undetected frame image Fn at time t1 and weights each particle Pa. In FIG. 16, the particles Pa are indicated by black circles, and in the undetected frame image Fn, the weighting is indicated by the size of black circles.

  Here, it is assumed that the generation unit 232a generates a plurality of particles Pa on the captured undetected frame image Fn2 at time t2 next to time t1. In this case, the particle selection unit 232b of the generation unit 232a selects, as the particle Pa of the undetected frame image Fn2, the particle Pa generated in the undetected frame image Fn having a large weight.

  For example, in FIG. 16, the particle selection unit 232b selects a particle Pa having a weighting coefficient equal to or greater than the threshold Th0 among the particles Pa generated in the undetected frame image Fn as the particle Pa in the undetected frame image Fn2. Note that the particle selection unit 232b obtains, from the weighting unit 235, a weighting coefficient of the particle Pa generated with, for example, the undetected frame image Fn.

  Note that, for example, the particle selection unit 232b may select N0 particles Pa as the particle Pa of the undetected frame image Fn2 from the particles Pa generated in the undetected frame image Fn having the larger weighting coefficient. .

  Thus, the particle selection unit 232b selects the particle Pa of the undetected frame image Fn2 according to the weighting of the particle Pa generated in the undetected frame image Fn. As a result, the obstacle detection apparatus 1 can use the particle Pa having a high likelihood at the previous time t1 for obstacle detection at the next time t2, thereby improving the obstacle detection accuracy. it can.

  For example, when the first detection unit 22 detects the moving speed and moving direction of the obstacle as in Modification 1, for example, the particle Pa selected by the particle selecting unit 232b is moved as shown in FIG. You may move according to at least one of a speed and a moving direction. FIG. 17 is a diagram illustrating another generation example of the particle Pa according to the third modification of the embodiment.

  As shown in the frame image Fn3 of FIG. 17, the particle selection unit 232b selects the particle Pa according to the weight of the particle Pa generated in the undetected frame image Fn. In the frame image Fn3 of FIG. 17, the particle Pa is indicated by a black circle.

  Next, the particle selection unit 232b moves the selected particle Pa according to the moving speed and moving direction indicated by the vector V. For example, in the undetected frame image Fn2 of FIG. 17, the particle Pa before movement is indicated by a dotted circle, and the particle Pa after movement is indicated by a black circle.

  Thus, the detection accuracy of the obstacle detection apparatus 1a can be further improved by moving the particle Pa selected by the particle selection unit 232b according to at least one of the moving speed and the moving direction.

(4.4. Modification 4)
In the above embodiment, the second detection unit 236 detects an obstacle using the particles Pa in the designated range M2, but the present invention is not limited to this. For example, an obstacle may be detected using a particle Pa having a likelihood greater than the threshold Th1 among the particles Pa in the designated range M2. This point will be described with reference to FIG.

  FIG. 18 is a diagram illustrating a configuration of an obstacle detection device 1b according to the fourth modification of the embodiment.

  The obstacle detection device 1b shown in FIG. 18 has the same configuration as the obstacle detection device 1 shown in FIG. 2 except that the second detection unit 236a includes the selection unit 25 and the obstacle detection unit 26. In FIG. 18, the same components as those shown in FIG. 2 are denoted by the same reference numerals as those shown in FIG.

  The selection unit 25 selects a particle Pa to be used for detecting an obstacle from the plurality of particles Pa generated by the generation unit 232 according to the likelihood calculated by the likelihood calculation unit 234. The selection unit 25 selects, for example, a particle Pa having a likelihood equal to or higher than a threshold Th1. Alternatively, the selection unit 25 may select N particles Pa in descending order of likelihood, for example.

  The obstacle detection unit 26 detects an obstacle based on the particle Pa selected by the selection unit 25. The obstacle detection method by the obstacle detection unit 26 is the same as the detection method of the second detection unit 236 shown in FIG.

  Thus, the obstacle detection unit 26 can further improve the obstacle detection accuracy by detecting the obstacle in the undetected frame image Fn using the particle Pa having a high likelihood.

(4.5. Modification 5)
In the said embodiment, although the 2nd detection part 236 detected the obstruction using the particle Pa, it is not limited to this. For example, when the likelihood of the particle Pa is smaller than the threshold Th2, the second detection unit 236 may stop detecting the obstacle. This point will be described with reference to FIG.

  FIG. 19 is a flowchart illustrating a processing procedure executed by the obstacle detection apparatus 1 according to the fifth modification of the embodiment. The procedure of the obstacle detection process executed by the detection interpolation unit 23 of the obstacle detection apparatus 1 according to the modification 5 will be described with reference to FIG.

  In addition, the detection process which the obstacle detection apparatus 1 which concerns on the modification 5 performs is the same as the detection process shown in FIG. The obstacle detection apparatus 1 according to the modification 5 executes the obstacle detection process shown in FIG. 19 instead of executing the obstacle detection process shown in FIG. 10 in step S105 of FIG. In FIG. 19, processing steps similar to those shown in FIG. 10 are denoted by the same reference numerals as those shown in FIG. 10.

  When the likelihood calculating unit 234 calculates the likelihood of the reference hue histogram Hh and the hue histogram in step S204 of FIG. 19, the weighting unit 235 determines whether the calculated likelihoods of all the particles Pa are less than the threshold Th2. Is determined (step S301).

  When the likelihood of all the particles Pa is less than the threshold Th2 (Yes in step S301), the process proceeds to step S207. Therefore, the detection interpolation unit 23 does not detect an obstacle using the particle Pa generated in step S202.

  On the other hand, when the likelihood of all the particles Pa is equal to or greater than the threshold Th2 (No in step S301), the process proceeds to step S205. Therefore, the detection interpolation unit 23 detects an obstacle using the particle Pa generated in step S202.

  Here, when the likelihood of all the particles Pa is less than the threshold Th2, the detection interpolation unit 23 is configured not to detect the obstacle, but the present invention is not limited to this. For example, when there are M or more particles Pa whose likelihood is less than the threshold Th2, the detection interpolation unit 23 may not detect the obstacle.

  The threshold value Th2 may be the same as or different from the threshold value Th1 used when the selection unit 25 selects the particle Pa in the fourth modification.

  In this way, by preventing the obstacle from being detected when the number of particles Pa having a likelihood smaller than the threshold Th2 is M or more, erroneous detection of the obstacle by the detection interpolation unit 23 can be reduced. The detection accuracy can be further improved.

(4.6. Other Modifications)
In the above embodiment, the detection interpolation unit 23 detects the obstacle when the first detection unit 22 cannot detect the obstacle. However, the present invention is not limited to this. For example, the detection interpolation unit 23 may perform obstacle detection using a particle filter for all the frame images F captured by the imaging device 5.

  In this case, the frame image F is input to the output unit 24 from both the first detection unit 22 and the detection interpolation unit 23. When the first detection unit 22 detects an obstacle, the output unit 24 outputs the frame image F on which the detection region M input from the first detection unit 22 is superimposed to the vehicle control device 6. On the other hand, when the first detection unit 22 cannot detect the obstacle, the output unit 24 outputs the frame image F on which the detection area M <b> 3 input from the detection interpolation unit 23 is superimposed to the vehicle control device 6.

  Moreover, in the said embodiment, although the detection interpolation part 23 detected the obstruction based on the detection area M of the obstruction detected from the detection frame image Fd, it is not limited to this. For example, when the first detection unit 22 cannot detect the detection region M from the frame image F, the detection interpolation unit 23 detects the obstacle based on the detection region M3 detected by the second detection unit 236 from the undetected frame image Fn. May be detected.

  In the above embodiment, the first detection unit 22 detects the obstacle detection area M based on machine learning, but the present invention is not limited to this. The first detection unit 22 only needs to be able to detect the obstacle detection area M. For example, the obstacle detection area M may be detected based on optical flow, template matching, or the like.

  Moreover, although the said embodiment demonstrated the case where the imaging device 5 imaged the front of the own vehicle, you may make it the imaging device 5 image the side and back of the own vehicle, for example. Or you may make it the some imaging device 5 image the surroundings of the own vehicle, for example. In this case, for example, the obstacle detection device 1 performs obstacle detection for one frame image F obtained by integrating the frame images F captured by the plurality of imaging devices 5.

  In the above embodiment, the frame image F on which the obstacle detection areas M and M3 detected by the obstacle detection device 1 are output is output to the vehicle control device 6, but the present invention is not limited to this. For example, the obstacle detection device 1 may output a frame image F on which the obstacle detection areas M and M3 are superimposed to a display device (not shown).

  Moreover, although the said embodiment demonstrated the case where the obstacle detection apparatus 1 was mounted in the own vehicle, it is not limited to this. That is, the obstacle detection device 1 can also detect an obstacle from a frame image F input from a fixed imaging device provided in a streetlight or a building, for example.

  Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS 1 Obstacle detection apparatus 5 Imaging device 22 1st detection part 23 Detection interpolation part 231 Standard histogram calculation part 232 Generation part 233 Histogram calculation part 234 Likelihood calculation part 235 Weighting part 236 2nd detection part 25 Selection part 26 Obstacle detection part

Claims (16)

A first detection unit that detects a detection region where an obstacle exists from the first frame image captured by the imaging device;
A reference histogram calculation unit for calculating a reference hue histogram in the detection area detected by the first detection unit;
A generating unit that generates a plurality of particles on a second frame image captured after the first frame image based on the detection region;
A histogram calculation unit for calculating a hue histogram within a specified range provided for each of the plurality of particles;
A likelihood calculating unit for calculating a likelihood between the reference hue histogram calculated by the reference histogram calculating unit and the hue histogram calculated by the histogram calculating unit;
A weighting unit that weights each of the plurality of particles based on the likelihood calculated by the likelihood calculating unit;
A second detection unit that detects an obstacle from the second frame image based on the plurality of particles weighted by the weighting unit;
An obstacle detection device comprising: The reference histogram calculation unit includes:
The obstacle detection device according to claim 1, wherein the reference hue histogram within a detection range smaller than the detection area detected by the first detection unit is calculated. The reference histogram calculation unit includes:
The obstacle detection device according to claim 2, wherein a central area of the detection area detected by the first detection unit is set as the detection range. The generator is
The obstacle detection device according to claim 1, wherein a plurality of the particles are generated in and around the detection region. The generator is
5. The plurality of particles moved from the detection region according to at least one of a movement speed and a movement direction of the obstacle detected by the first detection unit are generated according to claim 1. Obstacle detection device. The generator is
Among the plurality of particles generated on the third frame image captured before the second frame image, the particles having the high likelihood calculated by the likelihood calculating unit are selected as the particles on the second frame image. The obstacle detection device according to any one of claims 1 to 5, wherein the obstacle detection device is a particle. The second detector is
The obstacle detection device according to any one of claims 1 to 6, wherein an obstacle is detected based on the particles within a certain range corresponding to a center of the detection region. The second detector is
The obstacle detection device according to any one of claims 1 to 7, wherein an obstacle is detected based on the particles within a certain range according to the type of the obstacle detected by the first detection unit. The second detector is
The obstacle is detected based on the particles within a certain range according to the position moved from the center of the detection area according to at least one of the movement speed and movement direction of the obstacle detected by the first detection unit. The obstacle detection device according to any one of claims 1 to 8. The second detector is
The obstacle detection device according to claim 7, 8 or 9, wherein an obstacle is detected based on the particles having an elliptical shape within the predetermined range. The second detector is
The obstacle detection device according to any one of claims 1 to 10, wherein a gravity center position corresponding to the weighting performed by the weighting unit of a plurality of particles is detected as a gravity center position of an obstacle. The second detector is
A selection unit that selects the particles according to the likelihood calculated by the likelihood calculation unit from a plurality of the particles;
An obstacle detection unit for detecting an obstacle based on the particles selected by the selection unit;
The obstacle detection device according to any one of claims 1 to 11, further comprising: The selection unit includes:
The obstacle detection apparatus according to claim 12, wherein N particles are selected in descending order of the likelihood. The selection unit includes:
The obstacle detection device according to claim 12, wherein the particles having the likelihood equal to or greater than a first threshold value are selected. The second detector is
The obstacle detection according to any one of claims 1 to 14, wherein no obstacle is detected when there are M or more particles having the likelihood calculated by the likelihood calculation unit equal to or less than a first threshold value. apparatus. A first detection step of detecting a detection region where an obstacle exists from a first frame image captured by the imaging device;
A reference histogram calculation step of calculating a reference hue histogram in the detection region detected in the first detection step;
Generating a plurality of particles on a second frame image captured after the first frame image based on the detection region;
A histogram calculation step of calculating a hue histogram within a specified range provided for each of the plurality of particles;
A likelihood calculating step for calculating a likelihood between the reference hue histogram calculated in the reference histogram calculating step and the hue histogram calculated in the histogram calculating step;
A weighting step for weighting each of the plurality of particles based on the likelihood calculated in the likelihood calculating step;
A second detection step of detecting an obstacle from the second frame image based on the plurality of particles weighted in the weighting step;
An obstacle detection method including:

Images