JP2010231400A - Obstacle dangerous degree calculation device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately calculate the degree of danger in an area to be an obstacle candidate. <P>SOLUTION: An obstacle danger degree calculation device is provided with: a stereo image processor 1 for calculating parallax distribution information in the surroundings of a vehicle from a plurality of image data obtained by imaging the surrounding environment of the self-vehicle from different visual points; an obstacle candidate area generator 2 for generating a plurality of obstacle candidate areas from the parallax distribution information; and a danger degree calculator 6 for calculating the degree of danger from at least one of a first obstacle candidate, which is one of the plurality of obstacle candidate areas, a second degree of shielding, representing to what extent the first obstacle candidate area is shielded by a dangerous object recognized from the image data, and a second degree of shielding, representing to what extent the first obstacle candidate area is shielded by a preset dangerous object. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for calculating the danger level of an obstacle that is difficult for a driver to see from an image taken by a camera mounted on an automobile.

  Technology for detecting people, bicycles, motorcycles, and other obstacles from images taken by cameras plays an important role in ensuring safety in moving objects such as automobiles. In addition, for each three-dimensional object in the surrounding environment of the vehicle that the driver is driving (hereinafter referred to as the host vehicle), evaluate the “risk level” that quantifies how dangerous it is for the driver. If possible, it is possible to provide a safety device that is more beneficial to the driver, such as issuing a warning only to a really dangerous obstacle.

  Patent Document 1 considers that an object closer to the own vehicle has a higher degree of danger and divides the image into a plurality of areas, and executes obstacle detection processing first from the area close to the own vehicle. A technique that enables early detection is disclosed.

Japanese Patent No. 3540005

  However, obstacles that are dangerous to the driver are not limited to those nearby, but are dynamically determined according to the situation, such as those that are hidden behind other objects and those that are in the driver's blind spot. Can also exist. In addition, it is difficult for both the driver and the other party (when the obstacle is a pedestrian or the like) to recognize that there is a positional relationship that is difficult for the driver to see.

  The present invention has been made in view of the above, and an object of the present invention is to more accurately calculate the risk level of a candidate area for an obstacle.

  In order to solve the above-described problems and achieve the object, one aspect of the present invention calculates parallax distribution information around the host vehicle from a plurality of image data obtained by shooting the surrounding environment of the host vehicle from different viewpoints. A stereo image processing unit, an obstacle candidate region generation unit that generates a plurality of obstacle candidate regions from the parallax distribution information, and a first obstacle candidate region that is one of the plurality of obstacle candidate regions is the image. A first shielding degree that represents how much the dangerous object recognized from the data is shielded, and a second shielding that represents how much the first obstacle candidate area is shielded by a predetermined dangerous object A risk level calculation unit that calculates the risk level of the first obstacle candidate area from at least one of the degrees.

  According to the present invention, it is possible to more accurately calculate the risk level of an area that is a candidate for an obstacle.

It is a block diagram which shows the structure of the obstacle risk degree calculation apparatus concerning this Embodiment. It is a figure explaining the block matching in parallax calculation. It is a figure explaining the method of producing | generating an obstacle candidate area | region from a solid object area | region. It is a figure which shows an example of a vehicle periphery condition. It is a figure which shows the obstacle candidate area | region set obtained from the vehicle periphery condition of FIG. It is a figure explaining driver | operator shielding. It is another figure explaining driver | operator shielding. It is another figure explaining driver | operator shielding. It is a figure explaining the calculation method of driver obstruction degree. It is a figure explaining an obstacle candidate area | region shielding. It is a figure which shows the flowchart of the calculation method of an obstacle risk degree.

  Exemplary embodiments of an obstacle risk degree calculating apparatus and an obstacle risk degree calculating method according to the present invention will be explained below in detail with reference to the accompanying drawings. In the present embodiment, the risk level calculation for obstacles existing in front of the vehicle is mainly described. However, the risk level can be calculated by the same method for the entire vehicle periphery such as the rear of the vehicle. It is not limited to risk calculation only.

  In the present embodiment, the obstacle risk level calculation device is mounted on a traveling vehicle and calculates the risk level of an obstacle existing in front of the vehicle. The calculated obstacle danger level is used to detect a dangerous obstacle in front of the vehicle.

  It is also possible to perform driver warning and vehicle risk avoidance control using the position information and the risk level of the obstacle candidate area described later as they are, and this embodiment can be used as an obstacle detection device. By applying pattern recognition processing to the obstacle candidate area as preprocessing, for example, an apparatus configuration in which only pedestrians among obstacles are detected is possible. In this case, if the recognition process is performed in order from the highest risk level, it will be possible to detect the truly dangerous items at an earlier stage. It becomes possible to do.

  In addition, if the processing is terminated when the obstacle candidate areas are recognized and processed in descending order of the risk level, the number of obstacle candidate areas increases depending on the situation, and the recognition process is performed. It is possible to keep a calculation amount constant by preventing a situation where the calculation amount required is enormous. The risk level calculation in this embodiment can be calculated from the obstacle candidate area position information, and the calculation amount may be considered to be much smaller than the recognition process in which the calculation is performed on the pixels in the image.

  FIG. 1 is a block diagram showing the configuration of the obstacle risk degree calculation apparatus according to the present embodiment. The obstacle risk degree calculation device includes a stereo image processing unit 1, an obstacle candidate region generation unit 2, a head position detection unit 3, a driver shielding degree calculation unit 4, an obstacle candidate region shielding degree calculation unit 5, and a danger. The degree calculation unit 6 is provided.

  The stereo image processing unit 1 includes parallax distribution information within a shooting range from image data (stereo image pair) obtained by shooting a surrounding environment of the host vehicle by a stereo image acquisition device (not shown) including a plurality of stereo cameras. (Three-dimensional position information) is calculated.

  Here, when the point corresponding to the point a in one image is reflected in the point b in the other image, the parallax is calculated by the distance between the two points, and the difference between the distances becomes the parallax value. In general, the smaller the parallax, the farther away in the real world, and the larger the parallax, the closer to the camera. The parallax can be calculated for any region in the image. Therefore, the parallax distribution information is a two-dimensional array of parallax and represents how the parallax changes in the image.

  If the camera external parameters indicating the relative positional relationship between the stereo cameras and the camera internal parameters such as focal length and lens distortion are acquired, the three-dimensional position information (depth distance information) of the target area is obtained from the parallax value. ) Can be easily calculated. Therefore, since obtaining the parallax distribution information may be considered as obtaining three-dimensional position information (depth distance information) within the imaging range (around the vehicle), the stereo image processing unit 1 determines that the imaging range (vehicle It can be said that the three-dimensional position information (depth distance information) in the surroundings is calculated.

  In addition, camera external parameters and internal parameter calculation methods have been studied extensively in the field of computer vision and robot vision, and parameter calculation methods with sufficient accuracy for stereo image processing are known. Therefore, the detailed description is omitted. Here, the parallax estimation procedure in the case where the stereo image acquisition apparatus has two stereo cameras, which is the simplest case, will be described.

  FIG. 2 is a diagram illustrating block matching in parallax calculation. First, as a premise, it is assumed that the two stereo cameras are installed in parallel to the front side of the vehicle. In particular, it is possible to obtain parallax distribution information even if they are not arranged in parallel, but in the case of such a parallel camera, the vertical coordinates of corresponding points match in the left and right images, and the problem becomes simpler. In order to obtain the parallax, it is necessary to find a point corresponding to a pixel in one image from the other image. In this case, it is known that the vertical coordinate of the corresponding point is the same as the original pixel. The search may be performed while changing only the horizontal coordinate.

  As shown in FIG. 2, the search for corresponding points is performed by cutting out the peripheral partial image 13 of the pixel of interest 12 in the left image 11 and cutting out the partial image 15 of the same size in the right image 14 to evaluate the similarity. Realize. The degree of similarity can be evaluated by the normalized cross-correlation expressed by the expression (1). The larger the value (closer to 1), the higher the degree of similarity.

  In equation (1), L (i, j) is the pixel value of the peripheral partial image 13, L is the average pixel value of the peripheral partial image 13, R (i, j) is the pixel value of the partial image 15, and R is the partial image. 15 average pixel values are represented. Σ represents summing the values in the partial image. This similarity calculation is performed while moving the range of the partial image 15 to search for a point having the highest similarity. If a corresponding point is found, the partial image 15 is fixed and matching is performed on the left image 11. If the similarity is maximized around the target pixel 12, the matching is considered successful, and the partial image 13 and the partial image 13 are matched. The difference between the horizontal coordinates of the image 15 is set as the parallax value of the target pixel 12.

  The disparity distribution information is obtained by repeating the above processing over the entire image. Another process is required to obtain the depth distance from the parallax distribution. In the case of two cameras installed in parallel, the depth distance Z with respect to the parallax value d can be obtained by equation (2).

  In equation (2), f is the focal length of the camera, and B is the baseline distance between the stereo cameras. These values can be obtained by performing camera calibration. If this is repeated over the entire image, depth distance information can be obtained.

  The obstacle candidate area generation unit 2 extracts a three-dimensional object standing vertically from the road from the parallax distribution information (three-dimensional object extraction process), and further, an obstacle candidate area that is a unit for obstacle detection from the extracted three-dimensional object Are cut out (obstacle candidate region generation process).

In the three-dimensional object extraction process, first, each parallax distribution data in the parallax distribution information is voted (aggregated) in u (horizontal coordinate) -d (parallax) space. That is, the number of data having a certain horizontal coordinate u1 and parallax d1 is calculated, and the distribution is obtained. As described above, since the camera that captures the input image is installed so as to be substantially horizontal, the parallax value d is substantially constant in an area standing perpendicular to the road surface. Therefore, it can be considered that a part with a large number of votes (total number) in the ud space is a three-dimensional object region standing perpendicular to the ground. Therefore, by binarizing the ud space by threshold processing based on the voting results (aggregation results), it is possible to divide into three-dimensional objects-non-three-dimensional objects (road surface or sky). As a method of determining the threshold value, a constant value T _ud can be used for the entire _ud space, or, in the case of a stereo camera, there is a tendency that a parallax is larger, that is, an object closer to it is projected larger. For this reason, the threshold value may be determined by a function of d as shown in Equation (3) so as to be proportional to the parallax d.

In the formula (3), α and t ₀ are predetermined constants. In the case of a stereo camera, determining the threshold value according to the parallax d is more robust against noise such as erroneous parallax.

  Based on the result of the three-dimensional object-non-three-dimensional object determination, processing is performed to leave only a portion corresponding to the three-dimensional object in the original three-dimensional position information. The processing procedure is to read one piece of parallax distribution data and obtain u (horizontal coordinate), v (vertical coordinate), and d (parallax) data for each piece of data. Read the coordinate (u, d) data. If the read data is a three-dimensional object, the parallax information is left, and if the read data is a non-three-dimensional object, the parallax information is invalidated. By repeating the above for the entire parallax distribution, the parallax distribution of only the three-dimensional object region can be obtained.

  In the obstacle candidate area generation process, the obtained three-dimensional object area is divided into a set of obstacle candidate areas which are units of obstacle detection. In the case of processing such as extracting pedestrians from obstacles, it is difficult to directly compare the entire three-dimensional object region and the pedestrian model, so a partial region obtained by cutting out the size corresponding to the pedestrian from the three-dimensional object region Generated and used for the collation process. This partial area is an obstacle candidate area. This obstacle candidate area is used as a minimum unit for calculating the shielding degree and the risk degree.

  FIG. 3 is a diagram illustrating a method for generating an obstacle candidate region from a three-dimensional object region. The road surface 16 is a road surface in the surrounding environment of the host vehicle. It is assumed that a rectangular virtual obstacle 17 is standing at an arbitrary point on the road surface 16. This virtual obstacle 17 is projected on the image plane 18 of the camera, and becomes a rectangular area 19 in the image. If the camera parameters are obtained in advance, the position / size of the rectangular area 19 can be calculated from the three-dimensional coordinates and size of the virtual obstacle 17. The size of the virtual obstacle 17 is set in advance according to the minimum size of the obstacle to be detected. For example, when trying to detect an obstacle of a pedestrian size, the size of the virtual obstacle is specified to be about 800 mm in width and about 1600 mm in height. This size is for adults and smaller for children.

  As described above, the position / size of the projected image on the image of the virtual obstacle 17 can be calculated by the projection calculation, but the three-dimensional object does not always exist at the position of the virtual obstacle 17. Since it is only necessary to place the virtual obstacle 17 only at the place where the three-dimensional object exists, the determination is performed using the previously obtained parallax distribution information, and the virtual obstacle 17 that has passed the determination is set as an obstacle candidate region set. I will add.

  Whether the virtual obstacle 17 corresponds to the three-dimensional object region in the parallax distribution information is determined as follows. First, the upper left image coordinates (u1, v1) and the lower right coordinates (u2, v2) of the rectangular area 19 when projected onto the image plane 18 are calculated. Next, the parallax distribution in a rectangular partial region having (u1, v1) as the upper left corner and (u2, v2) as the lower right corner is evaluated. If the parallax at a certain pixel is d, the score at that pixel is obtained by equation (4).

In the equation (4), z (d) is a function for calculating the depth distance from the parallax d, and corresponds to the equation (2). z ₀ is the depth coordinate value of the virtual obstacle 17. σ is a parameter that determines how far a deviation between z (d) and z ₀ is allowed, and is set in advance. The score of equation (4) is calculated for the pixels in the partial area corresponding to the three-dimensional object area, and all the sums are calculated. Finally, the virtual obstacle score is obtained by dividing by the area of the partial area. It is confirmed from the disparity distribution information that the three-dimensional object is present at the position of the virtual obstacle 17 as the score increases. Become.

  Therefore, the score of the virtual obstacle 17 is compared with a predetermined threshold value. If the score is larger than the threshold value, it is determined that a three-dimensional object exists at the position of the virtual obstacle 17, and the position / size information of the virtual obstacle 17 is obtained as the obstacle. Add to candidate region set. When the score is smaller than the threshold value, it is determined that there is no solid object at the position of the virtual obstacle 17 and is not regarded as an obstacle candidate.

  FIG. 4 is a diagram illustrating an example of the situation around the vehicle. In FIG. 4, two stereo cameras 22 and 23 are installed in parallel to the front of the vehicle in the host vehicle 21 that the driver 20 drives. And the pedestrian 24 exists in the right front of the own vehicle 21, and the pedestrian 25 exists in the left front. Furthermore, a side wall 26 exists from the left side surface of the host vehicle 21 to the left front side.

  And an obstacle candidate area | region is produced | generated with respect to such a vehicle periphery condition. FIG. 5 is a diagram showing an obstacle candidate region set obtained from the vehicle surrounding situation of FIG. As shown in FIG. 5, an obstacle candidate area set including an obstacle candidate area group 27 corresponding to the pedestrians 24 and 25 and an obstacle candidate area group 28 corresponding to the side wall 26 can be obtained.

  If the standard size of the pedestrian is the size of the obstacle candidate area, the area corresponding to the pedestrians 24 and 25 is correctly extracted as the obstacle candidate area. It is divided into many obstacle candidate areas of size. However, such a part is characterized in that adjacent obstacle candidate regions appear continuously.

  In the obstacle candidate area set obtained as described above, each element holds the size and position information in the real space. Therefore, the obstacle candidate area generation unit 2 corresponds to simplifying the three-dimensional arrangement information of the three-dimensional object around the vehicle. Then, two types of shielding degrees (driver shielding degree and obstacle candidate area shielding degree) are calculated for each of the obstacle candidate area sets obtained by the obstacle candidate area generation unit 2.

  The head position detection unit 3 detects the head position of the driver 20. The head position is detected by image processing or the like.

  The driver shielding degree calculation unit 4 calculates the driver shielding degree by determining whether or not the vehicle is in the driver's blind spot area using the position information and area size of each obstacle candidate area. The driver shielding degree is a shielding degree that is uniquely determined with respect to the position / size of one obstacle candidate region. Here, the driver shielding is a shielding state in which it is difficult for the driver to visually recognize the obstacle among the driver and the obstacle.

  FIG. 6 is a diagram illustrating driver shielding. As shown in the figure, the driver shielded state means that a blind spot 30 is generated by a pillar 29 on the right front side of the host vehicle 21, the driver's 20 field of view is obstructed, and the driver 20 recognizes the presence of the pedestrian 24. An evaluation method is set so that the degree of shielding is high in such a case.

  7 and 8 are other diagrams for explaining driver shielding. In addition to the shielding by the pillar 29 in the driver's seat as shown in FIG. 6, the region 31 outside the visual field range of the driver 20 can also be regarded as being in a state where the driver is shielded as shown in FIG. Further, when the wiper is operating due to rain, as shown in FIG. 8, in addition to the region 31, it is considered that the driver is also shielded from the windshield wiper operating range and the region 32 corresponding to the side glass portion. be able to. In addition, the degree to which each obstacle candidate area (in the case of FIG. 6, the area where the pedestrian 24 exists) is shielded is evaluated numerically as the driver shielding degree.

  The driver shielding degree is calculated as follows. FIG. 9 is a diagram for explaining a method of calculating the driver shielding degree. Here, the degree of driver shielding when the shielding by the pillar 29 is occurring is calculated. First, with respect to the periphery of the host vehicle 21, a distribution of a shielding score representing an area where shielding occurs is obtained. In FIG. 9, the portion where the shielding occurs (dead angle 30) is represented by diagonal lines, and a shielding score of 0 to 1 is given to this shielding portion. The simplest way is to set the non-shielded portion to 0 and the shaded shaded portion to 1, but it is also possible to set an intermediate value for the boundary portion.

  In FIG. 9, the score distribution expresses that shielding is generated in the shadow area (dead angle 30) of the pillar 29 of the driver's seat, but further, the area 31 that falls outside the visual field range of the driver 20 and the windshield. It is also possible to create a score distribution combining different shielding factors such as the region 32 corresponding to the outside of the wiper operation range and the side glass portion. In this case, the maximum score among the scores for all shielding factors is adopted as the shielding score at each point.

  Since the driver's occlusion is generated when the visibility of the driver 20 is obstructed, strictly speaking, the occlusion score changes depending on the head position and the line-of-sight direction of the driver 20. Therefore, the shielding score is updated from the head position of the driver 20 detected by the head position detection unit 3 for each process. If it is not necessary to evaluate the shielding score so strictly, a certain degree of accuracy can be ensured even by a method in which an average head position and line-of-sight direction are assumed in advance and the shielding score is given fixedly.

  The driver shielding degree of the obstacle candidate area is calculated from the thus obtained shielding score distribution. Specifically, the shielding scores are calculated by summing the shielding scores within the obstacle candidate area and calculating the average area of the scores. The value of the driver shielding degree represents how much the obstacle candidate area has entered the shielding area.

  The obstacle candidate area shielding degree calculation unit 5 calculates the obstacle candidate area shielding degree by evaluating how much the obstacle candidate area is obstructed by other obstacle candidate areas and is difficult to see. Obstacle candidate area shielding means that the driver and the obstacle candidate area are difficult to see each other because the obstacle candidate area exists between the driver and the obstacle candidate area. Refers to that. Therefore, the obstacle candidate area shielding degree is a shielding degree determined by the relative positional relationship between one obstacle candidate area and a different obstacle candidate area, unlike the driver shielding degree.

Here, the obstacle candidate area shielding will be described in a little more detail. FIG. 10 is a diagram for explaining obstacle candidate area shielding. As shown in FIG. 10, an obstacle candidate area 33 corresponding to the pedestrian 24, an obstacle candidate area 34 corresponding to the pedestrian 25, an obstacle candidate area 35 corresponding to the side wall 26, and an obstacle candidate area 36 are included. Let us consider the obstacle candidate area group that we have. Each obstacle candidate area has 3D coordinates of the area center in real space and size information such as width and height as attributes, so that the position and size in the image plane can be obtained immediately. It has become. At this time, one obstacle candidate area is designated as an obstacle candidate area A, and another obstacle candidate area is designated as an obstacle candidate area B. Then, the degree OB _{→ A} where the obstacle candidate area A is blocked by the obstacle candidate area B is calculated by the equation (5).

In equation (5), u _A and u _B are horizontal coordinates in the images of the candidate areas A and B, and Z _A and Z _B are depth coordinates of the two areas in the real space. However, the depth coordinate value increases as the distance from the host vehicle increases. f _u and f _z are functions having the absolute value of the distance in the horizontal coordinate and the signed distance in the depth coordinate as inputs. An example of f _u is equation (6), and an example of f _z is equation (7).

In the equations (6) and (7), k ₁ and k ₂ are constants that determine the degree of change in the degree of shielding, and are parameters set in advance. w is the size when the standard width of the obstacle used in the obstacle candidate area generation unit 2 is projected onto the image plane. V is a standard size of the depth of the obstacle, and is set in advance.

Equation (6) becomes closer to 1 as Δu is close to 0, that is, the obstacle candidate area A and the obstacle candidate area B overlap in the image, and decreases toward 0 with w as a boundary. Go. On the other hand, the equation (7) is zero when ΔZ is negative, that is, the obstacle candidate area B is behind the obstacle candidate area A, and the obstacle candidate area B is in front of the obstacle candidate area A. The larger the value, the greater it will come. However, it is not a proportional relationship, but increases rapidly with V as a boundary. Therefore the value of the difference is smaller if f _z even if the obstacle candidate area B in front remains small.

By setting the product of the above two functions as the obstacle candidate area shielding degree OB _{→ A} , the obstacle candidate area 34 is set as the obstacle candidate area A and the obstacle candidate area 35 is set as the obstacle candidate area B. When the distance in the image is close and the depth is spaced as in the case, the value becomes large. On the other hand, obstacles obstacle candidate region 35 candidate region A, the case where the obstacle candidate region 36 and the obstacle candidate region B, the distance in the image is likewise close but the difference in depth small, the value of f _z As a result, OB _{→ A} becomes smaller. On the other hand, when the obstacle candidate region 33 and the obstacle candidate regions A, O _{B → A} is a small value close to 0 due to the effect of regardless f _u in the depth.

Expressions (6) and (7) are an example of a method for calculating the obstacle candidate area occlusion degree, and f _u has another form as long as it is a function that decreases from 1 to 0 with w as a boundary. There is no problem. As for f _z , another function may be used as long as it increases from 0 to 1 with 0 as a boundary in a negative region and V in a non-negative region.

By the above calculation, the shielding degree OB _{→ A} for the combination of the obstacle candidate area A and the obstacle candidate area B can be obtained. Finally, the obstacle candidate area shielding degree calculation unit 5 determines obstacle candidate areas B other than the obstacle candidate area A as obstacle candidate areas B, and obtains the shielding degree OB _{→ A} in each case. The shielding degree having the largest value among the shielding degree OB _{→ A} is determined as the obstacle candidate area shielding degree.

  The degree-of-risk calculation unit 6 calculates the degree of risk from the driver shielding degree calculated by the driver shielding degree calculation unit 4 and the obstacle candidate area shielding degree calculated by the obstacle candidate region shielding degree calculation unit 5. Finally, the output of the obstacle risk level calculation device is a set of the position / size of each obstacle candidate area and its risk level.

  In addition, as a calculation method of the risk level, the greater value of the two shielding levels (driver blocking level and obstacle candidate area blocking level) is set as the risk level, or the two shielding levels are respectively There is a method in which a risk value is obtained by multiplying different values and then summing those values. Whichever method is used, the result is that the higher the degree of shielding, the higher the degree of danger. What is necessary is just to set the calculation method before execution of an obstacle risk degree calculation apparatus according to the request | requirement of how much each shielding degree is considered as important.

  Next, an obstacle risk level calculation method by the obstacle risk level calculation apparatus configured as described above will be described. FIG. 11 is a diagram illustrating a flowchart of an obstacle risk degree calculation method.

  The stereo image processing unit 1 calculates parallax distribution information (three-dimensional position information) from the image data (stereo image pair) from the stereo image acquisition device 7 (step S101). Here, it is assumed that the image data is taken using a pair of stereo cameras installed substantially in the horizontal direction for processing in the obstacle candidate area generation unit 2 in the subsequent stage. When a camera is installed in a vehicle for photographing the surrounding environment, it is not necessary to tilt it up and down, so this is not a big limitation.

  Next, the obstacle candidate area generation unit 2 extracts a three-dimensional object area standing on the road surface from the parallax distribution information (step S102). Considering the environment around the vehicle, other vehicles, pedestrians, guardrails, walls, etc. can all be regarded as objects standing perpendicular to the road surface. Therefore, by this process, road surfaces and sky areas that do not correspond to obstacles that are dangerous for the host vehicle are removed from the parallax distribution information.

  Next, the obstacle candidate area generation unit 2 sets an obstacle candidate area in a predetermined area in the image area based on the size of the virtual obstacle, and collates it with the extracted three-dimensional object area (step S103). It is determined whether the object candidate area corresponds to the three-dimensional object area (step S104).

  When the obstacle candidate area generation unit 2 determines that the obstacle candidate area corresponds to the three-dimensional object area, that is, when it is determined that a three-dimensional object exists at the position of the virtual obstacle (step S104: Yes), The obstacle candidate area (virtual obstacle position / size information) is added to the obstacle candidate area set (step S105), and the process proceeds to step S106.

  The obstacle candidate area generation unit 2 determines that the obstacle candidate area does not correspond to the three-dimensional object area, that is, determines that there is no three-dimensional object at the position of the virtual obstacle (step S104: No). Without doing anything, the process proceeds to step S106.

  In step S106, the obstacle candidate area generation unit 2 determines whether or not obstacle candidate areas have been set in the entire image area. If the obstacle candidate area generation unit 2 determines that no obstacle candidate area is set in all the image areas (step S106: No), the obstacle candidate area generation unit 2 returns to step S103 and performs the following steps.

  If the obstacle candidate area generation unit 2 determines that the obstacle candidate area is set in all the image areas (step S106: Yes), the obstacle candidate area generation unit 2 proceeds to step S107. That is, the processing from step S103 to step S105 is applied to a predetermined region in the real space determined while moving the position of the virtual obstacle. In step S106, it is checked whether all the determinations have been completed.

  In this way, through steps S103 to S106, the three-dimensional object region obtained in step S102 is divided into a set of obstacle candidate regions that are units of obstacle detection.

  Next, the driver shielding degree calculation unit 4 selects one obstacle candidate area from the set of obstacle candidate areas (step S107). At this time, the selected obstacle candidate area is set as an obstacle candidate area A. Next, the driver shielding degree calculation unit 4 calculates the driver shielding degree of the obstacle candidate area A (step S108).

  Next, the obstacle candidate area shielding degree calculation unit 5 selects one obstacle candidate area other than the obstacle candidate area A from the set of obstacle candidate areas (step S109). At this time, the selected obstacle candidate area is set as an obstacle candidate area B.

  Next, the obstacle candidate area shielding degree calculation unit 5 assumes that the obstacle candidate area A is an obstacle and the obstacle candidate area B is a shield, and the obstacle candidate area A and the obstacle candidate area B are relative to each other. Based on the positional relationship, the obstacle candidate area shielding degree of the obstacle candidate area A is calculated. Then, if there is an obstacle candidate area shielding degree of the obstacle candidate area A calculated before, it is added to this (step S110). The previously calculated obstacle candidate area occlusion degree refers to the obstruction degree having the largest value among previously calculated obstacle candidate area occlusion degrees, and the total here is the obstacle candidate calculated this time Comparing the area occlusion degree with the previously calculated obstacle candidate area occlusion degree, the higher occlusion degree is determined as the obstacle candidate area occlusion degree.

  Next, the obstacle candidate area shielding degree calculation unit 5 determines whether all obstacle candidate areas other than the obstacle candidate area A are selected as the obstacle candidate area B (step S111). If the obstacle candidate area shielding degree calculation unit 5 determines that all obstacle candidate areas other than the obstacle candidate area A are not selected as the obstacle candidate area B (step S111: No), the process returns to step S109. Another obstacle candidate area is selected as an obstacle candidate area B, and the subsequent steps are repeated. If the obstacle candidate area shielding degree calculation unit 5 determines that all obstacle candidate areas other than the obstacle candidate area A are selected (step S111: Yes), the process proceeds to step S112.

  In step S112, the risk degree calculation unit 6 determines the obstacle from the driver shielding degree of the obstacle candidate area A calculated in step S108 and the obstacle candidate area shielding degree of the obstacle candidate area A calculated in step S110. The risk level of the object candidate area A is calculated.

  Next, the risk level calculation unit 6 determines whether or not all of the obstacle candidate areas have been selected as the obstacle candidate area A, that is, whether or not the risk level has been calculated for all of the obstacle candidate areas (step S113). If it is determined that all of the obstacle candidate areas have not been selected as the obstacle candidate area A (step S113: No), the risk level calculation unit 6 returns to step S107, and selects one of the other obstacle candidate areas. Select as an obstacle candidate area A and repeat the following steps. If it is determined that all of the obstacle candidate areas have been selected as the obstacle candidate area A (step S113: Yes), the risk level calculation unit 6 proceeds to step S114.

  In step S114, the risk level calculation unit 6 adds the risk level information to the information (virtual obstacle position / size information) of all the obstacle candidate areas carved in step S105, and outputs the information (step S114). S114). Through the above steps, the calculation of the obstacle risk is completed.

  With the obstacle candidate risk calculation mechanism as described above, if there is an obstacle candidate area with a high degree of danger, a warning is issued, and the danger is avoided by controlling the brake and steering device to avoid danger. It becomes possible to execute countermeasures according to high objects. For detection of dangerous goods, the risk level is calculated continuously in successive frames, and the risk avoidance action is executed when obstacles with a high risk level appear continuously or approach the host vehicle. It is possible to judge the condition.

  Furthermore, a pedestrian classifier that determines whether the obstacle candidate area is a pedestrian or not, a vehicle classifier that determines whether the vehicle is a vehicle, etc. are added to the subsequent stage of the obstacle risk degree calculation device according to the present invention. You can also In general, identification is a process with a relatively large amount of calculation, so by using the risk level calculated by the risk level calculation device, the identification process is executed in descending order of the risk level, and a warning is immediately issued when the target object is found. By giving a warning, the driver can be informed of the danger more quickly, or by identifying and processing a predetermined number in order of the highest degree of danger, giving priority to dangerous things, keeping the calculation time constant, or certain dangers It can also be applied to processing with higher priority given priority, such as identifying and processing only items that are higher than the specified level.

  Note that the obstacle risk degree calculation device according to the present embodiment includes a head position detection unit that detects the driver's head position, but is provided outside without including the head position detection unit. You may comprise so that a driver | operator's head position information may be received from a head position detection apparatus.

  Moreover, each part of the obstacle risk calculation apparatus according to the present embodiment can be realized by using a computer as hardware and causing the computer to execute a program.

  As described above, according to the obstacle risk degree calculating apparatus according to the present embodiment, the parallax distribution information around the own vehicle is calculated from the plurality of image data obtained by photographing the surrounding environment of the own vehicle from different viewpoints, An obstacle that extracts a three-dimensional object area from the parallax distribution information, generates a plurality of obstacle candidate areas from the three-dimensional object area, and indicates how much the selected one obstacle candidate area is shielded by other obstacle candidate areas The degree of risk of the selected obstacle candidate area from at least one of the degree of occlusion of the object candidate area and the degree of occlusion of the driver representing how much the selected obstacle candidate area is occluded by the driver's blind spot area Therefore, it is possible to calculate the risk of an obstacle that is difficult to see from the driver.

  The present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 Stereo image processing part 2 Obstacle candidate area | region production | generation part 6 Risk level calculation part 21 Own vehicle 33, 34, 35, 36 Obstacle candidate area | region

Claims (7)

A stereo image processing unit that calculates parallax distribution information around the host vehicle from a plurality of image data obtained by photographing the surrounding environment of the host vehicle from different viewpoints;
An obstacle candidate area generation unit that generates a plurality of obstacle candidate areas from the parallax distribution information;
A first degree of occlusion representing how much a first obstacle candidate area, which is one of the plurality of obstacle candidate areas, is covered by a dangerous object recognized from the image data; and the first obstacle A risk level calculation unit that calculates a risk level of the first obstacle candidate area from at least one of a second level of blockage that indicates how much the object candidate area is shielded by a preset dangerous object. Was it,
Obstacle risk calculation device characterized by the above. The dangerous object recognized from the image data is set as a second obstacle candidate area different from the first obstacle candidate area among the plurality of obstacle candidate areas,
A first shielding degree calculation unit that calculates a first shielding degree using a positional relationship between the first obstacle candidate area and the second obstacle candidate area;
The obstacle risk calculating apparatus according to claim 1, wherein: The dangerous goods set in advance are set as a driver's blind spot area range,
A second shielding degree calculation unit that calculates a second shielding degree using a positional relationship between the first obstacle candidate area and the blind spot area;
The obstacle risk degree calculation device according to claim 1 or 2, characterized in that   4. The risk level calculation unit according to claim 1, wherein the risk level calculation unit sets a higher value of the first shielding level and the second shielding level as the risk level. 5. Obstacle risk calculation device described.   The risk level calculation unit sets the risk level to a value obtained by adding values obtained by multiplying the first shielding level and the second shielding level by different coefficients, respectively. The obstacle risk degree calculation device according to any one of Items 1 to 3. An obstacle risk calculation method,
A stereo image processing unit that calculates parallax distribution information around the host vehicle from a plurality of image data obtained by photographing the surrounding environment of the host vehicle from different viewpoints;
An obstacle candidate area generation unit, wherein an obstacle candidate area generation unit generates a plurality of obstacle candidate areas from the parallax distribution information;
A first degree of shielding that represents how much the first obstacle candidate area, which is one of the plurality of obstacle candidate areas, is shielded by the dangerous object recognized from the image data; and A risk level for calculating the risk level of the first obstacle candidate area from at least one of the second level of blockages indicating how much the first obstacle candidate area is shielded by a predetermined dangerous object Including a calculating step,
Obstacle risk calculation method characterized by On the computer,
A stereo image processing unit that calculates parallax distribution information around the host vehicle from a plurality of image data obtained by photographing the surrounding environment of the host vehicle from different viewpoints;
An obstacle candidate area generation unit, wherein an obstacle candidate area generation unit generates a plurality of obstacle candidate areas from the parallax distribution information;
A first degree of shielding that represents how much the first obstacle candidate area, which is one of the plurality of obstacle candidate areas, is shielded by the dangerous object recognized from the image data; and A risk level for calculating the risk level of the first obstacle candidate area from at least one of the second level of blockages indicating how much the first obstacle candidate area is shielded by a predetermined dangerous object A calculation step;
Obstacle risk calculation program to execute.

Images