JP2003067727A - Environmental complexity arithmetic system, environmental recognition level estimation system and obstruction warning system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support obstruction recognition efficiently and accurately without presenting a driver with excessive information. SOLUTION: A microcomputer 70 divides a pickup image produced by an obstruction imaging CCD camera 21 into regions A to D, and computes complexity C of traffic environment in each of the regions B to D according to the following expression: C=α.C1+β.C2+γ.C1.C2, where the index C1 represents a total number of brightness-divided blocks, the index C2 represents a total number of red-system-divided blocks, and α, β and γ are weight coefficients of respective given values.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an environment complexity calculation device, an environment recognition degree estimation device and an obstacle warning device,
In particular, the present invention relates to an environment complexity calculation device, an environment recognition degree estimation device, and an obstacle warning device that support obstacle recognition of a driver according to the traffic environment around a vehicle.

[0002]

2. Description of the Related Art When driving a vehicle, it is necessary to check whether the vehicle is running straight ahead or out of the left or right, and when the vehicle is turning left or right, crossing objects to the left or right with respect to the traveling direction. Confirmation is important to prevent accidents. As the moving body that jumps out or crosses from the left and right, various kinds of pedestrians, two-wheeled vehicles, four-wheeled vehicles, etc. can be considered.

Japanese Patent Application Laid-Open No. 3-26013 proposes an environment recognition device for detecting such a moving body. The environment recognition device obtains contour information by extracting a difference in brightness information and a closed section region, and recognizes a moving object from the contour information. However, the lighting conditions surrounding the vehicle change from moment to moment, and due to the influence of ambient light, etc., it is rare that the contour can be extracted completely and clearly. Therefore,
Since the environment recognition device performs image recognition using only contour information, it is weak in coping with environmental changes due to the small amount of information. In particular, when it is mounted on a vehicle and detects an obstacle or the like, it is difficult to say that the moving body can be detected optimally because it is greatly affected by the above-mentioned influence. Further, it is difficult to directly extract the closed section from the lightness information, because it is easily affected by shadows and ambient light, and the lightness changed by the ambient light directly affects the change of the center of gravity and the area.

In addition, detection methods such as template matching and texture matching in which internal information is added to contour information have been proposed. However, there are problems that it takes a long processing time, and it takes time and effort to prepare a large amount of information about the template in advance.

Under such a background, for example, in recent years, a technique has been proposed, which is derived from a technique for compressing image information, and captures image pickup information of a camera as a distribution of lightness data.

In Japanese Laid-Open Patent Publication No. 11-142168, a principal component analysis, which is one of multivariate analysis, is performed, imaging information is mapped to an information space defined by normalized principal component feature quantities, and the information space is mapped in the information space. A technique has been proposed in which the image recognition method is improved by analyzing the behavior of the data of. This technique is to perform more robust environment recognition of the normalized principal component feature amount using vehicle motion information and operation information. Further, Japanese Patent Laid-Open No. 2000-19259 proposes a technique for recognizing an object using visible light image information and infrared image information other than visible light.

However, these techniques make it difficult to recognize a moving body when there are many factors that hinder recognition of an object such as a guardrail or a stopped vehicle. For example, there is a problem that a pedestrian on a sidewalk or an intersection other than a road surface on which a vehicle passes, a pedestrian who jumps out from a standby position of a bicycle, or between stopped vehicles cannot be recognized.

On the other hand, for example, Japanese Patent Laid-Open No. 11-301
No. 343 proposes a vehicle lighting device that predicts the probability of appearance of a moving obstacle by recognizing the environment (road shape) using not only the obstacle itself but also a camera (visible light, infrared). Further, Japanese Patent Application Laid-Open No. 11-232569 proposes a pedestrian warning system having a device for transmitting to a vehicle side whether or not a pedestrian approaches an intersection or the like. Further, in Japanese Unexamined Patent Publication No. 6-144129, a driver is selected by randomly selecting one of a plurality of caution items prepared for the same attention state without positively recognizing a moving obstacle. An alerting assist device has been proposed that alerts users by giving a notification.

As described above, by automatically recognizing a moving obstacle, by considering the appearance probability of a moving object, or by giving a warning of a pedestrian's approach, the moving obstacle recognition assists the driver in obstacle recognition. Support is proposed.

However, it is the driver himself who finally recognizes the moving obstacle. Unless the driver's visual perception and cognitive characteristics are taken into consideration, too much information is presented to the driver, which may rather confuse the driver. In addition, the method of randomly presenting cautionary items according to location is inefficient, although the amount of information presented is small.

The present invention has been proposed to solve the above-mentioned problems, and an environmental complexity calculation capable of efficiently and accurately supporting obstacle recognition without presenting excessive information to the driver. An object is to provide a device, an environment recognition degree estimation device, and an obstacle warning device.

[0012]

The invention according to claim 1 is
At least one of an image capturing unit that generates a captured image based on image capturing light from the environment, a distribution of changes in luminance included in the captured image generated by the image capturing unit, and a distribution of red pixels included in the captured image. And a complexity calculation means for calculating the complexity of the environment.

In the invention according to claim 1, it is preferable that the image pickup means is installed so as to pick up an image of a traffic environment from the front seat of the vehicle to the front. The complexity calculation means can obtain the spatial frequency indicating the complexity of the image from the distribution of the luminance change included in the captured image. Further, the complexity calculating means can obtain the complexity of the image from the distribution of the red pixels included in the captured image.

According to a second aspect of the present invention, in the first aspect of the present invention, the complexity calculation means sets a luminance division block in the imaged image generated by the image pickup means, and the luminance division block is set in the set luminance division block. Until it is determined that the luminance value of each pixel is uniform, the luminance division block is divided and the new luminance division block is set repeatedly, and the luminance value of each pixel is uniform with respect to the captured image. It is characterized in that the degree of complexity of the environment is calculated based on the number of the luminance division blocks.

According to the second aspect of the present invention, the complexity calculating means sets the luminance division block for the captured image. The luminance division block set first indicates an area for which the complexity is calculated. Then, in the luminance division block, the luminance division block is divided until the luminance value of each pixel is determined to be uniform, and the divided luminance division block is set as a new luminance division block. repeat. Here, that the brightness value of each pixel is uniform is not limited to the case where the brightness values of the pixels are the same, and the difference between the maximum value of the brightness value and the minimum value of the brightness value may be a predetermined value or less. That is, it is sufficient that the variation in the brightness value of each pixel is less than or equal to a predetermined value.

When it is determined that the luminance values of the pixels in all the luminance division areas are uniform, the number of all these luminance division blocks is calculated as the complexity of the environment.
The number of such luminance division blocks corresponds to the spatial frequency. That is, if the total number is large, there are many high-frequency components and the captured image is complicated. Further, when the total number is small, the high frequency component is small and the captured image is simple. In this way, by calculating the number of luminance division blocks, the complexity of the environment can be calculated.

According to a third aspect of the present invention, in the first aspect of the present invention, the complexity calculating means sets a red-based division block in the captured image generated by the imaging means, and the set red-based division is set. Repeating dividing the red-colored divided block and setting a new red-colored divided block until the block is configured only with the predetermined red-colored pixel, and only the predetermined red-colored pixel is added to the captured image. It is characterized in that the complexity of the environment is calculated based on the number of red-colored divided blocks configured by.

According to the third aspect of the invention, the complexity calculating means sets a red-color division block for the picked-up image.
The red-colored divided block set first indicates an area for calculating the complexity. The red-colored divided block is divided until the red-colored divided block is composed of only predetermined red-colored pixels. Then, the divided red-colored divided block is set as a new red-colored divided block, and this processing is repeated. If all the red-colored divided blocks are composed of only predetermined red-colored pixels, the number of all these red-colored divided blocks is obtained as the complexity of the environment. Red has a characteristic that it is more attractive than green, and the number of red-based divided blocks indicates the complexity of the image. Therefore, the complexity of the environment can be obtained by calculating the number of red-colored divided blocks.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the complexity calculating means sets a luminance division block in the picked-up image generated by the image pickup means, and within the set luminance division block. Until it is determined that the luminance value of each pixel is uniform, the luminance division block is divided and the new luminance division block is set repeatedly, and the luminance value of each pixel is uniform with respect to the captured image. Until a red-colored divided block is set in the captured image generated by the image pickup means, and the set red-colored divided block is composed of only predetermined red-colored pixels, Repeating the division of the red-colored divided block and setting of a new red-colored divided block, the red-colored divided block composed of only the predetermined red-colored pixels for the captured image. Calculates the number of click, the number of the luminance divided blocks, the number of red divided blocks, characterized by calculating the complexity of the environment based on.

In the invention according to claim 4, the complexity calculating means obtains the number of luminance division blocks in the same manner as in the invention in claim 2, and further, in the same manner as in the invention in claim 3, the red-based division is performed. The number of blocks is calculated, and the complexity of the environment is calculated based on these total numbers. As the complexity of the environment, the number of luminance division blocks, the number of red color division blocks, and the product of the number of luminance division blocks and the number of red color division blocks are multiplied by predetermined weighting factors, respectively, and the sum of these is added. Is preferred.

According to a fifth aspect of the present invention, the degree of recognition of the environment is estimated based on the complexity calculation device according to any one of the first to fourth aspects and the complexity calculated by the complexity calculation device. And an environment recognition degree estimating means for performing.

According to the fifth aspect of the present invention, the complexity calculated by the complexity calculating device indicates the complexity of the imaged image of the environment. Therefore, the parameter for estimating the degree of recognition of the environment by the observer. Used as. That is, the environment recognition degree estimation means can estimate the environment recognition degree based on the complexity.

According to a sixth aspect of the present invention, in the fifth aspect of the invention, the environment recognition degree estimating means determines the degree of environment recognition when the complexity calculated by the complexity calculating device is equal to or more than a threshold value. It is estimated that the degree of recognition of the environment is high when the degree of complexity calculated by the complexity calculating device is smaller than the threshold value.

According to the sixth aspect of the invention, a threshold value is set as a reference value of the degree of recognition of the environment. That is, the threshold is a value of complexity indicating that the environment is between complicated and simple. Therefore, the environment recognition degree estimation means uses this threshold value to estimate that the environment recognition degree is low when the complexity is equal to or higher than the threshold value, and the environment recognition degree is high when the complexity degree is lower than the threshold value. Can be estimated.

The invention according to claim 7 is the invention according to claim 5 or 6.
In the invention described above, further comprising an illuminance detection means for detecting the illuminance of the environment, the environment recognition degree estimation means sets the threshold value corresponding to the illuminance detected by the illuminance detection means, using the set threshold value It is characterized by estimating the degree of recognition of the environment.

According to the seventh aspect of the invention, even if the complexity is constant, when the outside of the vehicle becomes bright or dark, the observer sometimes feels that the environment is complicated or simple. Therefore, in order to correct the change in the degree of recognition of the environment due to such a change in illuminance, it is preferable to use a threshold table in which thresholds corresponding to illuminance are described in advance. Thereby, when the illuminance detecting means detects the illuminance, it is possible to set a threshold value corresponding to the detected illuminance. Then, the degree of environment recognition can be estimated using the set threshold. As a result, the recognition degree of the environment can be accurately estimated regardless of the value of the illuminance.

The invention according to claim 8 is the invention according to any one of claims 5 to 7, further comprising a posture change detecting means for detecting a posture change of the observer, wherein the environment recognition degree estimating means comprises: The threshold is set to be larger as the change in the posture of the observer detected by the posture change detection means becomes larger, and the threshold is set to be smaller as the change in the posture of the observer detected by the posture change detection means becomes smaller. It is characterized in that the degree of recognition of the environment is estimated using the thresholds.

According to the eighth aspect of the invention, the observer tends to recognize the environment more when the posture change is large. The environment recognition degree estimating means sets a larger threshold value as the observer's posture change becomes larger, and sets a smaller threshold value as the observer's posture change becomes smaller, so that the actual visual characteristics of the observer are met. Moreover, the degree of recognition of the environment can be accurately estimated.

The invention according to claim 9 is the environment recognition degree estimation device according to any one of claims 5 to 8 for estimating the degree of recognition of the environment for each predetermined area, and an obstacle for detecting the position of the obstacle. When a predetermined area including the position detection means and the position of the obstacle detected by the obstacle detection means is estimated by the environment recognition degree estimation means to have a low degree of environment recognition, a warning regarding the obstacle is issued. And an alarming means to perform.

In the ninth aspect of the invention, the obstacle position detecting means detects the position of the obstacle. At this time, the environment recognition degree estimation device estimates the degree of recognition of the environment in a predetermined area including the position where the obstacle is detected. Here, when the degree of environment recognition is high, the observer can sufficiently recognize the obstacle detected by the obstacle position detecting means. on the other hand,
When the degree of environment recognition is low, the observer may not be able to recognize the obstacle detected by the obstacle position detection means. Therefore, the warning means issues a warning about the obstacle when it is estimated that the degree of recognition of the environment in the predetermined area including the position of the obstacle is low. Thereby, even if there is an obstacle in a region where the degree of recognition of the environment is low, the observer can recognize in advance that there is an obstacle and avoid an accident. Note that the warning means can alert the observer by outputting an image or a sound. Further, the warning means is estimated to have a high degree of environment recognition in a predetermined area including the position of the obstacle. Occasionally, there is no warning about obstacles. Accordingly, the observer can recognize the obstacle when the obstacle is present in a region where the degree of recognition of the environment is high, and thus the observer is not bothered by an extra alarm.

According to a tenth aspect of the present invention, in the invention of the ninth aspect, the environment recognition degree estimation device divides an area excluding a lower portion of the captured image into a plurality of estimation areas, and each divided estimation area. The degree of recognition of the environment is estimated, and the warning unit is configured such that the position of the obstacle detected by the obstacle position detection unit is in any of the estimation regions and the degree of recognition of the environment of the estimation region is the environment recognition. When it is estimated that the degree is low by the degree estimating means, a warning is given regarding an obstacle existing in the estimation area having a high priority.

According to the tenth aspect of the present invention, the environment recognition degree estimation device divides the area excluding the lower part of the captured image into a plurality of estimation areas, and estimates the environment recognition degree for each of the divided estimation areas. That is, the degree of environment recognition is not estimated for the lower part of the captured image. The reason is that the lower part of the imaging screen is
This is because the image of the environment immediately before the observer is shown, and it is meaningless to estimate the degree of recognition of the environment.

In the alarm means, the position of the obstacle detected by the obstacle position detecting means is in any of the estimation areas, and the degree of environment recognition of the estimation area is low by the environment recognition degree estimating means. Determine if it was estimated. Here, all the estimation regions where the obstacle is detected and the degree of environment recognition is low are selected. Then, an alarm indicating that an obstacle is present is given to an estimated area having a high priority among the selected specified areas. With this, it is possible to avoid a large accident by alerting the observer to the most dangerous area, and by not notifying the less dangerous area to the observer, it is possible that the observer becomes confused by excessive information. Can be prevented.

[0034]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment] The first embodiment of the present invention can be applied to, for example, an obstacle warning device 1 having the configuration shown in FIG. The obstacle warning device 1 includes an infrastructure information detection unit 10 that detects infrastructure information that detects infrastructure information from an infrastructure outside the vehicle, an obstacle information detection unit 20 that detects information about roads and obstacles around the road, and A vehicle state detection unit 30 that detects an operating state, an environment information detection unit 40 that detects the environment of the driver or the outside of the vehicle,
A slider panel 50 for inputting visual characteristics indicating the degree of environmental recognition of the driver, an obstacle information output unit 60 for outputting obstacle information, and a microcomputer for performing overall control based on the information detected by each unit. 70,
Is equipped with.

The infrastructural information detection unit 10 uses the GPS (Gl
GPS Positioning Circuit 11 for receiving the signal of the Global Positioning System and a DVD drive 12 for reading the map information recorded on the DVD disc.
And an approach information receiving circuit 13 for receiving approach information.

The GPS receiving circuit 11 includes the GPS antenna 1
The GPS signal having the time and the position information of the GPS satellite is received via 1a and supplied to the microcomputer 70 via the data bus 5. The DVD drive 12 reads the map information from the DVD disk based on the position information of the vehicle currently traveling, and through the data bus 5,
It is supplied to the microcomputer 70. The approach information receiving circuit 13 receives the approach information transmitted from a data carrier reader, which will be described later, and supplies it to the microcomputer 70 via the data bus 5.

The obstacle information detection unit 20 includes an obstacle photographing CCD camera 21 for photographing obstacles on and around the road and an infrared ray for photographing obstacles on and around the road by infrared rays. A camera 22 and a radar transceiver 23 for recognizing a front obstacle are provided.

The CCD camera 21 for photographing obstacles and the infrared camera 22 are installed so that an object in the front direction of the vehicle can be photographed. Then, the CCD camera 21 for photographing obstacles
The infrared camera 22 supplies the captured image to the microcomputer 70 via the data bus 5. In order to recognize an obstacle ahead, the radar transceiver 23 sharply narrows the pulsed optical radar to the obstacle and transmits it in a two-dimensional direction, and receives the optical radar reflected by the obstacle. The radar transceiver 23 is not limited to one that transmits and receives an optical radar, but may be one that transmits and receives a radio wave radar.

The vehicle state detector 30 includes, for example, a wheel speed sensor, a steering angle sensor, a throttle valve sensor, a master cylinder oil pressure sensor, a yaw rate sensor, a longitudinal acceleration sensor, a lateral acceleration sensor, and the like. Then, the vehicle state detection unit 30 detects the vehicle speed, the steering wheel steering angle, the accelerator operation amount, the brake operation amount, the yaw rate, the roll rate, the pitch rate, the longitudinal acceleration, the lateral acceleration, and the blinker operation amount, and causes the microcomputer 70 to detect them. Supply.

The environment information detecting section 40 is provided with an illuminance sensor 41 for detecting the illuminance outside the vehicle and a driver photographing CCD camera 42 for photographing the driver. Here, only one CCD camera 42 for photographing the driver is provided, but three CCD cameras for respectively photographing the front front, right front, and left front of the driver are provided so that the posture change of the driver can be easily detected. It may be provided.

The slider panel 50 is for inputting visual characteristics indicating the complexity of the external environment in accordance with the subjectivity of the driver. The slider panel 50, as shown in FIG. 2, has a slider 51 that is movable left and right. For example, when the driver tends to overlook the traffic environment, that is, when it is difficult to recognize the traffic environment outside the vehicle, the driver operates the slider 51 in the “weak” direction. In addition, the driver operates the slider 51 in the “strong” direction when the traffic environment can be seen well, that is, when the traffic environment outside the vehicle can be easily recognized. As a result, the microcomputer 70 will be described in detail later.
Can estimate the degree of environmental recognition in consideration of the driver's current visual characteristics.

The obstacle information output section 60 is an LCD (Liquid Crystal) which outputs obstacle information by an image.
l Display) 61 and a speaker 62 for outputting obstacle information by voice.

The microcomputer 70 is a CPU (Central Processing) (not shown).
Unit), RAM (R which is a work area of data
and ROM (R) that stores a program for executing various routine processes described later, an appearance range estimation table, and a threshold table.
ead Only Memory). The microcomputer 70 estimates obstacles that may appear, estimates the degree of environmental recognition of the driver, and issues an obstacle information alarm based on the information from each unit.

(Main Routine) In the obstacle warning device 1 configured as described above, the microcomputer 7
0 executes the processing from step ST1 to step ST3 shown in FIG. Here, first, step ST1
The processing from step ST3 to step ST3 will be briefly described, and then a specific subroutine of each processing will be described.

The microcomputer 70 detects the position of the own vehicle based on the information detected by the infrastructure information detecting section 10 and estimates obstacles that may appear in the traveling direction of the own vehicle (step ST1). . Then, the captured image of the environment in the traveling direction of the own vehicle is divided, and the degree of environment recognition indicating whether the driver can easily recognize each divided area is estimated (step ST2). Finally, the microcomputer 70 warns the driver of an obstacle with an image or a voice as necessary based on the obstacle information that may appear and the degree of environment recognition for each predetermined area (step). ST3).

(Step ST1) In step ST1,
The microcomputer 70 specifically executes the processing from step ST11 to step ST14 shown in FIG. 4 in order to estimate an obstacle that may appear in the traveling direction of the vehicle.

In step ST11, the microcomputer 70 detects the position of the own vehicle, and also detects a moving obstacle around the position of the own vehicle and outside the driver's visual range. Here, for example, the infrastructure described below is used.

For example, as shown in FIG. 5, a pedestrian is an article (for example, a mobile phone) provided with the data carrier 16.
Always carry. A data carrier 16 is provided in a two-wheeled vehicle or a four-wheeled vehicle. Data carrier 16
Can output several types of approach information,
The data carriers 16 for wheeled vehicles and four-wheeled vehicles output different approach information. On the other hand, at intersections and pedestrian crossings with poor visibility, a data carrier reader 1 that receives approach information
7 is installed. When the data carrier 16 approaches, the data carrier reader 17 receives the approach information,
This approach information is transmitted to the obstacle warning device 1.

When the approach information receiving circuit 13 receives the approach information transmitted from the data carrier reader 17, the microcomputer 70 of the obstacle warning device 1 can specify the location of the obstacle, and the obstacle can be detected. Pedestrian,
It is possible to recognize whether the vehicle is a two-wheeled vehicle or a four-wheeled vehicle.

Then, the microcomputer 70 uses the G
GPS signal received by the PS receiver circuit 11 and DV
The position of the own vehicle is detected based on the map information read from the D drive 12 and the approach information received by the approach information receiving circuit 13, and the position of the own vehicle is detected and is outside the driver's visual range around the own vehicle position. A moving obstacle is detected, and the type of the moving obstacle (person, two-wheeled vehicle, or four-wheeled vehicle) is recognized.

Further, the microcomputer 70 refers to the appearance range estimation table shown in FIG. 6 and determines the range in which the moving obstacle may appear on the road around the vehicle based on the type of the moving obstacle. presume. For example, when recognizing a four-wheeled vehicle as a moving obstacle, the microcomputer 70 estimates that the four-wheeled vehicle may appear within a range of 10 m in the traveling direction from the current position. When a pedestrian is recognized as a moving obstacle, it is estimated that the pedestrian may appear within a radius of 1 m from the current position.

The microcomputer 70 sets a coordinate system (hereinafter referred to as a "coordinate system around the vehicle") consisting of the traveling direction axis of the vehicle and an axis orthogonal to the traveling direction axis, and the range of possibility of appearance of a moving obstacle. Is described in the own vehicle peripheral coordinate system, and the process proceeds to step ST12.

At step ST12, the microcomputer 70 detects an obstacle within the driver's visual field. Here, the microcomputer 70 drives the obstacle photographing CCD camera 21 and the infrared camera 22, and the obstacle photographing CCD camera 21 and the infrared camera 22.
The captured image generated by is acquired. Then, by obtaining the difference between the images generated by the cameras, a moving obstacle is detected by obtaining an image in which humans and vehicles, which are especially problematic in terms of safety, are sufficiently contrasted with those other than that.

The microcomputer 70 also obtains a distance image of the front obstacle of the vehicle based on the round-trip time of the optical radar from transmission to reception by the radar transceiver 23 and recognizes the front obstacle. Microcomputer 70
Describes the moving obstacle recognized in this way in the own vehicle peripheral coordinate system, and proceeds to step ST13.

In step ST13, the microcomputer 70 detects the vehicle state based on the information from the vehicle state detection unit 30. Here, for example, Japanese Patent Laid-Open No. 11-3
As described in Japanese Patent No. 01343, the microcomputer 70 uses a vehicle speed, a steering wheel steering angle, an accelerator operation amount, a brake operation amount, a yaw rate, a roll rate,
The pitch rate, the longitudinal acceleration, the lateral acceleration, and the blinker operation amount are detected, the traveling direction of the own vehicle is estimated, the estimation result of the traveling direction of the own vehicle is described in the own vehicle peripheral coordinate system, and step S
Move to T14.

At step ST14, the microcomputer 70 causes the moving obstacle which may appear within the predicted travel range of the own vehicle with respect to the own vehicle peripheral coordinate system obtained by the processing from step ST11 to step ST13. Keep only the information of and remove other information. And
The microcomputer 70 displays information about the moving obstacle described in the coordinate system around the vehicle as an obstacle display screen.
When it is displayed on the CD 61, the process exits the subroutine and proceeds to step ST2 shown in FIG.

Here, the obstacle display screen is, for example, as shown in FIG. 7, a map around the vehicle, the current position of the vehicle, the predicted travel range of the vehicle, and moving obstacles. (People, cars, etc.) and their possible appearance range are displayed. The size of the moving obstacle displayed on the obstacle display screen indicates a range in which the moving obstacle may appear.

(Step ST2) In step ST2,
The microcomputer 70 specifically executes the processing from step ST21 to step ST24 shown in FIG. 8 in order to estimate the degree to which the driver recognizes the vehicle surrounding environment.

In step ST21, the microcomputer 70 detects a change in the driver's posture using the information from the environment information detecting section 40. That is, the microcomputer 70 obtains a difference for each frame of the imaged image from the driver photographing CCD camera 42, and detects the difference value of the imaged image as the moving area amount of the driver.

The microcomputer 70 counts "1" when the movement area amount is equal to or larger than the predetermined threshold value, and does not count when the movement area amount is smaller than the predetermined threshold value. Then, for example, the count value for the past 5 minutes is obtained, and when the count value is less than the threshold TH1, it is determined that the posture change is “small”, and the count value is equal to or more than the threshold TH1 and less than the threshold TH2 (> TH1). When it is, it is determined that the posture change is “medium”, and the count value is the threshold TH2.
In the above case, it is determined that the posture change is “large”. In addition, the microcomputer 70 acquires the illuminance outside the vehicle detected by the illuminance sensor 41, and proceeds to step ST
Move to 22.

In step ST22, the microcomputer 70 calculates the environmental complexity C for each predetermined area of the obstacle display screen shown in FIG. The complexity C mentioned here is a parameter used to estimate the degree of recognition of the traffic environment of the driver.

First, the microcomputer 70 divides the obstacle display screen obtained in step ST1 into four areas in which obstacles and their appearance ranges are present in distance and direction. Here, as shown in FIG. 9, the obstacle display screen is divided into areas A, B, C and D. As a result, the obstacle display screen has an area A that is a front front short distance area of the vehicle, an area B that is adjacent to the area A and is a front front long distance area of the vehicle, the area A, and It has a region C that is adjacent to B and is a front left long distance region of the own vehicle, and a region D that is adjacent to regions A and B and is a front right long distance region of the own vehicle. . The obstacle display screen areas A to D are typically divided as shown in FIG.

Next, the microcomputer 70 calculates the complexity C of the traffic environment for each of the areas B, C and D based on the following equation (1).

[0065]

[Equation 1]

Index C1 is the number of luminance division blocks, index C
2 indicates the number of red-color divided blocks. Also, α,
β and γ are weighting coefficients and take predetermined values.

Here, the calculation of the index C1 will be described. Note that the microcomputer 70 calculates the index C1 for the region B, the region C, and the region D as described above, but here, the calculation of the index C1 for the region B using the captured image shown in FIG. 11 will be described. To do.

The microcomputer 70 drives the obstacle photographing CCD camera 21 to obtain a picked-up image as shown in FIG. 11, for example. Then, the entire captured image is set as a brightness division block.

The microcomputer 70 calculates the difference between the maximum value and the minimum value of the brightness value of each pixel in the set brightness division block. The microcomputer 70
It is determined whether the difference between the maximum value and the minimum value of the brightness value is equal to or more than a predetermined threshold value, and when the difference exceeds the predetermined threshold value, the brightness division block is divided into four. At this time, it is preferable that the microcomputer 70 divides the luminance division block so that it is composed of substantially the same number of pixels vertically and horizontally and is as large as possible.

When the brightness value of the picked-up image is as shown in FIG. 12A, for example, the microcomputer 70
Calculates the difference between the maximum brightness value (180) and the minimum brightness value (000). Then, the difference between the maximum and minimum luminance values (1
80) is greater than or equal to a threshold value (for example, 40). Since the above difference exceeds the threshold value (40) in FIG.
As shown in (A), the luminance division block is divided into four.

The microcomputer 70 divides the luminance division block into four until the difference between the maximum value and the minimum value of the luminance value of each pixel in each divided luminance division block becomes equal to or less than a predetermined threshold value. The setting of a new luminance division block is repeated.

For example, the brightness division block BK1 shown in FIG. 12A is composed of 4 × 4 pixels, and the maximum brightness value is 180 and the minimum brightness value is 20. The microcomputer 70 obtains the difference (160) between the maximum value and the minimum value of the brightness values of the brightness division block BK1, and the difference is the threshold value (4).
0) or more, and therefore, as shown in FIG. 12B, four luminance division blocks BK1 (BK2, BK3, BK4,
BK5).

Since the difference (39) between the maximum value and the minimum value of the brightness value of the brightness division block BK2 is not larger than the threshold value (40), the microcomputer 70 does not divide the brightness division block BK2. On the other hand, with respect to the brightness division blocks BK3, BK4, BK5, the difference between the maximum value and the minimum value of the brightness values is equal to or more than the threshold value (40), and therefore each brightness division block BK is divided as shown in FIG. To do.

Through the above processing, the microcomputer 70 obtains the index C1 indicating the number of brightness division blocks in which the difference between the maximum value and the minimum value of the brightness value of each pixel is equal to or less than the predetermined threshold value (40). . As a result, the microcomputer 70 makes the luminance values of the pixels forming the luminance division block uniform.

The index C1 is not limited to the number of brightness division blocks in which the difference between the maximum value and the minimum value of the brightness value of each pixel is less than or equal to a predetermined threshold value. For example, the index C1
May be a variance value of the brightness values of the captured image. Also,
The index C1 is a two-dimensional fast Fourier transform (FF) of the captured image.
T: Fast Fourier Transform)
It may be the sum of the high frequency components of.

The microcomputer 70 shown in FIG.
As shown in FIG. 2 (C), the luminance division block BK is 1 × 1.
Although the pixel is divided into pixels, the division may be ended when the luminance division block BK is divided into a predetermined size (for example, 4 × 4 pixels).

Next, the calculation of the index C2 will be described. Note that the microcomputer 70 calculates C2 indicating the number of red-colored divided blocks for each predetermined region as described above, but here, for the region B, the index C2 is obtained using the captured image shown in FIG. The calculation of is explained.

The microcomputer 70 drives the obstacle photographing CCD camera 21 to obtain a picked-up image as shown in FIG. 13 (A), for example. Then, the brightness of 0.3 or more and the saturation of 0.1 or more are extracted from the captured image to obtain a captured image as shown in FIG. 13B, for example. Furthermore, when the microcomputer 70 extracts a hue of 0.1 or less or 0.75 or more, a captured image as shown in FIG. 13C is obtained.

The microcomputer 70 is shown in FIG.
With respect to the captured image as shown in (C), block division is repeated so that the same red-colored divided block is occupied by only red-colored pixels, and as shown in FIG. Alternatively, a red-based divided block including pixels (0) other than the red-based pixels is obtained. Then, the microcomputer 70 obtains an index C2, which is the number of red-colored divided blocks composed of only red-colored pixels.

The microcomputer 70 uses the index C1 indicating the number of luminance division blocks and the index C2 indicating the number of red color division blocks thus obtained, and uses the above-mentioned formula (1).
The complexity C of the environment is calculated according to step ST23.
Move to. The value of the complexity C calculated as described above changes with time, as shown in FIG. 14, for example.

The index C2 is not limited to the number of red-colored divided blocks and may be, for example, the following value.

The microcomputer 70 determines that the Munsell color chart has a color chart of 2.
5RP, 5RP, 7.5RP, 10RP, 2.5R, 5
R, 7.5R, 10R, 2.5YR, and 5YR may be extracted, and only red pixels having a chroma of 2 or more and a brightness of 3 or more may be extracted. Then, until the same divided block is composed of only the red-colored pixels and pixels other than the red-colored pixels,
The division of the above divided blocks is repeated, and the obtained number of divided blocks may be used as the index C2.

Further, the microcomputer 70 applies color images of 2.5 RP, 5 RP, 7.5 RP, 10 RP, 2.5 in Munsell color chart to the imaged image of the predetermined area.
R, 5R, 7.5R, 10R, 2.5YR, 5YR may be extracted, and only the red pixels having a chroma of 2 or more and a brightness of 3 or more may be extracted, and the number of the red pixels may be used as the index C2.
Alternatively, the sum of the distances of the above-described red pixels from the center of the captured image may be used as the index C2.

At step ST23, the microcomputer 70 sets a threshold th for judging the degree of environment recognition. The threshold th is set to be different depending on the environmental illuminance outside the vehicle. The reason is that even if the value of the complexity C is constant, the driver may feel that the traffic environment is complicated or simple when the outside of the vehicle is bright or dark.

The microcomputer 70 stores, as shown in FIG. 15, for example, a threshold value table in which the threshold value th for the environmental illuminance is described. According to FIG. 15, the threshold th has a minimum value (constant) when the environmental illuminance is from zero to S1, and increases at a constant rate as the environmental illuminance increases from S1 to S2. And the threshold th
Becomes the maximum value (constant) when the environmental illuminance is from S2 to S3, and decreases at a constant rate as the environmental illuminance becomes larger than S3. The threshold value table shown in FIG. 15 is an example of the present embodiment, and the present invention is not limited to this.

Therefore, the microcomputer 70 refers to the threshold value table and sets the threshold value th based on the environmental illuminance detected by the illuminance sensor 41. Furthermore, the microcomputer 70 can reset the maximum value of the threshold value th according to the operation input value of the driver and the posture change of the driver.

First, the microcomputer 70 sets the maximum value of the threshold value th in accordance with the input value of the slider 51 provided on the slider panel 50. For example, when the driver slides the slider 51 toward “strong”, the microcomputer 70 sets the maximum value of the threshold th to be large as shown in FIG. On the contrary, when the driver slides the slider 51 toward “weak”,
The microcomputer 70 sets the maximum value of the threshold th to be small. When the slider 51 is between “strong” and “weak”, the microcomputer 70 may keep the default threshold value th set based on the environmental illuminance.

Next, the microcomputer 70 sets the maximum value of the threshold value th in accordance with the change in the attitude of the driver. The reason is that the driver tends to easily recognize various environments when the posture changes frequently, and tends to hardly recognize the environment when the posture does not change. Therefore, the microcomputer 70 sets the threshold value th as follows in consideration of the relationship between the recognizability of the traffic environment and the change in the posture of the driver.

For example, when the microcomputer 70 determines that the posture change of the driver is "small" in the above-mentioned step ST21, the microcomputer 70 sets the maximum value of the threshold th to be small as shown in FIG. On the contrary, for example, when it is determined that the driver's posture change is “large” in step ST21, the maximum value of the threshold th is set to be large. When it is determined that the posture change of the driver is “medium”, the microcomputer 70 may keep the default threshold value th set based on the environmental illuminance.

Thus, the microcomputer 70
Is a threshold value table in which the threshold value th corresponding to the ambient illuminance is described, and the slider 51 provided on the slider panel 50.
When the threshold value th is set based on the input value of, and the change in the posture of the driver, the process proceeds to step ST24.

In step ST24, the microcomputer 70 compares the complexity C calculated in step ST22 with the threshold value th set in step ST23, as shown in FIG. Then, when the complexity C does not exceed the threshold th, it is determined that the environment is not complex, that is, simple. If the complexity C exceeds the threshold th, it is determined that the environment is complicated.

In the microcomputer 70, the picked-up image obtained by the CCD camera 21 for picking up obstacles is shown in FIG.
In the case of the forward landscape as shown in FIG. 11, it is determined that the traffic environment is “simple”, and in the case of the forward landscape as shown in FIG. 19, the traffic environment is determined to be “complex”. And
After the subroutine processing is completed, step ST shown in FIG.
Move to 3.

(Step ST3) In step ST3,
The microcomputer 70 warns the driver of an obstacle. Here, it is determined whether or not there is a warning request flag for each area, and a predetermined warning is given to the driver by using the flag with a high priority. The warning request flag mentioned here is a flag indicating that the driver needs to be warned due to the presence of an obstacle or the like in the predetermined area.
The microcomputer 70 is specifically shown in FIG.
The processing from step ST31 to step ST42 shown in 0 is executed.

At step ST31, the microcomputer 70 determines whether or not to generate the warning request flag for the area A. Here, specifically, the subroutine from step ST51 to step ST54 shown in FIG. 21 is executed.

In step ST51, the microcomputer 70 determines whether an obstacle is detected in the area A or whether the current vehicle position is within the obstacle appearance range.
If either condition is satisfied, the process proceeds to step ST52, and if neither condition is satisfied, the subroutine process ends.

At step ST52, the microcomputer 70 determines whether or not the own vehicle is taking an avoiding action with respect to the obstacle based on the own vehicle state detected at the above-mentioned step ST13, and takes the avoiding action. If so, the process proceeds to step ST53, and if no avoidance action is taken, the subroutine process ends.

In step ST53, the microcomputer 70 determines whether or not the own vehicle cannot avoid the obstacle based on the information about the obstacle detected in step ST1 and the state of the own vehicle. Then, for example, even if the driver depresses the brake pedal or operates the steering wheel to take the avoidance action against the obstacle, if the own vehicle cannot avoid the obstacle, the process proceeds to step ST54 and the own vehicle is operated. When is able to avoid the obstacle, the subroutine processing is ended.

In step ST54, the microcomputer 70 generates an alarm request flag in the area A and ends the subroutine processing. Microcomputer 7
0, when such a subroutine process is completed,
Then, the process proceeds to step ST32 shown in 0.

In step ST32, the microcomputer 70 determines whether to generate the warning request flag for each of the areas B, C and D. Specifically, the subroutine processing from step ST61 to step ST64 shown in FIG. 22 is executed. Since the same process is performed in any of the areas, the process in the area B will be described as an example.

At step ST61, the microcomputer 70 determines whether an obstacle is detected in the area B or whether the current vehicle position is within the obstacle appearance range.
When either one of the conditions is satisfied, the process proceeds to step ST62, and when neither condition is satisfied, the subroutine process ends.

At step ST62, the microcomputer 70 determines at step ST2 that the area B is complex, that is, at step ST2, the complexity C of the area B exceeds the threshold th. Then, when the complexity C of the area B exceeds the threshold th, the process proceeds to step ST63, and when the complexity C does not exceed the threshold th, the subroutine process is ended.

In step ST63, the microcomputer 70 determines whether the obstacle or the place in the obstacle appearance range is the place where the alarm is issued for the first time. If it is the place where the alarm is issued for the first time, the microcomputer 70 proceeds to step ST64 and When it is not the place to issue an alarm, the subroutine processing is ended.

At step ST64, the microcomputer 70 generates an alarm request flag in the area B and ends the subroutine processing. Microcomputer 7
When 0 completes the above subroutine processing, the process proceeds to step ST33 shown in FIG.

At step ST33, the microcomputer 70 determines whether or not there is the warning request flag for the area A, and when there is the warning request flag for the area A, the microcomputer 70 proceeds to step ST39 and the warning request flag for the area A. If there is not, the process proceeds to step ST34.

At step ST34, the microcomputer 70 determines whether or not there is a warning request flag for the area C or the area D. Then, if there is an alarm request flag in at least one of the areas, the process proceeds to step ST38, and if there is no alarm request flag in any of the areas, the process proceeds to step ST35.

At step ST35, the microcomputer 70 determines whether or not there is the warning request flag for the area B, and when there is the warning request flag for the area B, the microcomputer 70 proceeds to step ST36 and the warning request flag for the area B. If there is not, the subroutine processing is ended.

Thus, the microcomputer 70
Performs the processing in the order of step ST33 to step ST35 to obtain area A, area C or area D, area B.
The priority order is set in the order of, and the warning of the obstacle is given from the area having the higher priority order.

In step ST36, the microcomputer 70 determines whether or not there is the warning request flag for the area A in the previous determination processing, and when there is the warning request flag for the area A, the process proceeds to step ST37 and the warning for the area A is issued. When there is no request flag, the subroutine processing ends.

In step ST37, the microcomputer 70 warns the driver of the area B. The microcomputer 70, as an alarm to the driver,
"Be careful of obstacles ahead,""You are approaching the intersection. Be careful of pedestrians jumping out."
, And the subroutine processing is ended.

In step ST34, the area C and the area D
In step ST38 when it is determined that there is at least one of the alarm request flags, the microcomputer 70
Alert the driver. An example of the alarm request is the same as step ST37.

On the other hand, when it is determined in step ST33 that the warning request flag of the area A is present, the process proceeds to step S
At T39, it is determined whether or not the number of warning request flags in the area A is equal to or larger than a predetermined number. If the number of warning request flags is equal to or larger than a predetermined number, the process proceeds to step ST40. Transition.

In step ST40, the microcomputer 70 determines whether or not the vehicle speed detected by the vehicle state detecting section 30 is equal to or higher than a predetermined value. If the vehicle speed is equal to or higher than the predetermined value, the process proceeds to step ST41 and the vehicle speed is When is not greater than or equal to the predetermined value, the process proceeds to step ST42.

In step ST41, the microcomputer 70 gives the driver a first alarm in the area A. The microcomputer 70 causes the LCD 61 to display, for example, “Collision caution. Please decelerate.” As a first warning to the driver, or causes the speaker 62 to output by voice, and then the subroutine processing ends.

In step ST42, the microcomputer 70 gives the driver a second alarm in the area A. The microcomputer 70 causes the LCD 61 to display, for example, "Avoid obstacles" as the second alarm for the driver, or causes the speaker 62 to output by voice, and then the subroutine processing ends.

The microcomputer 70 executes the step S.
When the subroutine processing of T3 is completed, the warning to the driver is ended. In this way, the microcomputer 70 sets the priority to the warning request flag of each area and performs the processing, thereby suppressing the warning to the driver to the minimum and preventing the driver from being confused due to excessive information. can do.

As described above, the obstacle warning device 1 according to the first embodiment automatically recognizes a moving obstacle, estimates the appearance range of the moving obstacle, and further detects the moving obstacle. By considering the degree of recognition of the traffic environment of the driver at the appearance location, the driver can be made to recognize the moving obstacle without confusing the driver. Also,
When the amount of information to be presented to the driver is large, the obstacle warning device 1 can prompt the vehicle speed of the host vehicle to be reduced and prevent an accident from occurring.

When the obstacle warning device 1 detects an obstacle in any of the areas A to D shown in FIG. 10, it gives the highest priority to the area A immediately before the own vehicle, and the obstacle existing in the area A. Give an alarm to an object. As a result, the obstacle warning device 1 can avoid the crisis that is approaching immediately before.

[Other Embodiments] Next, other embodiments of the present invention will be described. The same parts as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

(Aspect of Dividing Imaged Image) In the first embodiment, the microcomputer 70 divides the imaged region into four regions A to D as shown in FIG. While the indices C1 and C2 were calculated in the regions C and D, the microcomputer 70, for example, changes the captured image shown in FIG. 23 (A) into four regions as shown in FIG. 23 (B). You may divide into. That is, the microcomputer 70 divides the rectangular captured image along the two diagonal lines to divide the captured image into four regions. Then, as in the first embodiment, the indices C1 and C2 may be calculated for each region.

Further, the microcomputer 70, for example, as shown in FIG. 24 (B), the captured image shown in FIG. 24 (A) is divided into a region (1), two regions (2) and a region (3). It may be divided into two areas. Here, the region (1) is
This is an area showing the vehicle immediately in front of the own vehicle. Area (2)
Are areas on both sides of the area (1). Area (3) is
Areas other than the area (1) and the area (2).

The microcomputer 70 has the area (1)
For, the luminance division block is divided into 1 × 1 pixels,
The region (2) can be divided into 2 × 2 pixels, and the region (3) can be divided into 2 × 2 pixels, and the index C1 can be calculated in each region. Further, the microcomputer 70 can divide the red-colored divided block into 1 × 1 pixel in each area to obtain the image shown in FIG. 24C, and calculate the index C2 in each area.

Further, the microcomputer 70 can divide the picked-up image as follows. Figure 2
As shown in FIG. 5, the captured image may be divided into “bottom part of screen”, “center part of screen”, “left part of screen”, and “right part of screen”.
The "lower part of the screen" is a rectangular area of about 1/5 to 1/4 in the vertical direction on the lower side of the captured image. "Center of screen" is
It is a triangular area, and one side of the triangle is adjacent to the “lower part of the screen”, and the diagonal of the one side is located at the upper end of the imaging screen. The “left side of the screen” is a trapezoidal area on the left side of the area other than the “lower part of the screen” and the “center of the screen”. The “right side of the screen” is a trapezoidal area on the right side of the area other than the “bottom of the screen” and the “center of the screen”.

The center of the screen may be further divided as shown in FIG.

(Setting of Threshold Th) The microcomputer 70, like the first embodiment, sets the threshold th according to the input value of the slider 51 operated by the individual driver.
However, the setting may be made according to each input value of the slider 51 operated by a plurality of drivers.

FIG. 27 is a diagram in which the degree of recognition of the environment input by the driver is plotted when 10 captured images are presented to 10 drivers. The microcomputer 70 can set an appropriate threshold th by calculating the average of these values, for example.

(Other Calculation Method of Complexity C) The microcomputer 70 calculates the complexity C according to the equation (1),
Although the degree of environment recognition is estimated by comparing the complexity C and the threshold th, the present invention is not limited to this.

The microcomputer 70 is shown in FIG.
As shown in FIG. 8, a recognition degree map indicating the degree of recognition determined by the indicators C1 and C2 may be stored.
At this time, the microcomputer 70 may calculate the index C1 and the index C2 in the same manner as in the first embodiment, and estimate the environment recognition degree by referring to the recognition degree map. For example, the microcomputer 70 uses (C1,
When C2) = (a, b), it can be determined that the degree of environment recognition is high, and when (C1, C2) = (c, d), it can be determined that the degree of environment recognition is low.

(Three-dimensional space arrangement) The microcomputer 70 divides the picked-up image into predetermined areas in advance, calculates the indexes C1 and C2 for each predetermined area as in the first embodiment, and further calculates the indexes. Calculate C1 and C2. And
It is also possible to arrange the indices C1, C2, and C1 and C2 in a three-dimensional vector space as shown in FIG. 29, and use the value (size, orientation) of the movement vector on each pixel as the complexity C. it can.

At this time, for example, as shown in FIG.
In the dimensional vector space, a first area estimated to have a low degree of environment recognition and a second area estimated to have a high degree of environment recognition may be provided. Microcomputer 7
0 determines which of the above-mentioned areas the complexity C obtained using the three-dimensional space belongs to, and if the complexity C is in the first area, the driver estimates that the degree of environment recognition is low. If the complexity C is in the second area, the driver can estimate that the degree of environment recognition is high.

(Neural Net) The microcomputer 70 may estimate the degree of environment recognition by using a neural net. FIG. 31 is a block diagram showing the functional configuration of the microcomputer 70 that estimates the degree of recognition of the environment using a neural network.

The microcomputer 70 performs a process of subtracting an estimated value from an input value, a neural net 71 formed of a conversion function having an updatable database, a recognition degree estimation unit 72 for estimating the degree of recognition of the driver's environment. And a computing unit 73. Note that the computing unit 73, which may have a learning function by a statistical method instead of the neural network 71, is estimated by the recognition degree estimating unit 72 from the input value of the slider 51 shown in FIG. The estimated value of the degree of recognition of the environment is subtracted. The neural network 71 is a calculator 7
Sequential learning is performed based on the subtraction value subtracted by 3 and the complexity C, and the learned complexity C is supplied to the recognition degree estimation unit 72. The recognition degree estimation unit 72 uses the neural network 7
The degree of recognition of the environment may be estimated for each driver using the learned complexity C from 1.

As a result, the microcomputer 70
Sequential learning can be performed in consideration of the driver's actual environment recognition, and the degree of recognition of the traffic environment can be accurately estimated according to the visual characteristics of the driver.

[0133]

The environmental complexity calculation apparatus according to the present invention is based on at least one of the distribution of changes in luminance included in the captured image generated by the image capturing means and the distribution of red pixels included in the captured image. Then, by calculating the complexity of the environment, it is possible to obtain the degree of difficulty in recognizing the environment, that is, the complexity indicating the complexity of the environment.

The environment recognition degree estimating apparatus according to the present invention estimates the degree of environment recognition based on the complexity calculated by the complexity calculating apparatus, thereby objectively and accurately estimating the degree of environment recognition of the observer. Can be estimated.

The obstacle warning device according to the present invention relates to the obstacle when the position of the obstacle is detected and it is estimated that the predetermined area including the detected position of the obstacle has a low degree of environment recognition. By issuing an alarm, when there is an obstacle in a region where the degree of recognition of the environment is low, it is possible to warn the observer of the presence of the obstacle. On the other hand, since the alarm is not issued when there is an obstacle in the region where the degree of recognition of the environment is high, it is possible to prevent the observer from being bothered by the extra alarm.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an obstacle warning device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a slider panel included in the obstacle warning device.

FIG. 3 is a flowchart showing a main routine of an operation procedure of a microcomputer provided in the obstacle warning device.

FIG. 4 is a flowchart showing a specific operation procedure of step ST1 in the main routine.

FIG. 5 is a diagram for explaining an outline of a data carrier and a data carrier reader.

FIG. 6 is a diagram showing an appearance range estimation table stored in a microcomputer.

FIG. 7 is a diagram showing an obstacle display screen displayed on the LCD.

FIG. 8 is a flowchart showing a specific operation procedure of step ST2 in the main routine.

FIG. 9 is a diagram illustrating a state in which the obstacle display screen is divided into an area A and an area D.

FIG. 10 schematically shows an obstacle display screen from area A to area D
It is a figure explaining the state divided into.

FIG. 11 is a diagram illustrating an example of a captured image when the index C1 is calculated in the area B.

FIG. 12 is a diagram showing an example of luminance values of a captured image.

FIG. 13 is a diagram showing an example of a captured image.

FIG. 14 is a diagram illustrating a state in which the complexity C changes with time.

FIG. 15 is a diagram showing a threshold value table in which threshold values for environmental illuminance are described.

FIG. 16 is a diagram illustrating a state in which the threshold th changes according to a change in the posture of the driver.

FIG. 17 is a diagram for estimating the degree of environment recognition by comparing the complexity C with a threshold th.

FIG. 18 is a diagram showing an example of a landscape image with a simple traffic environment.

FIG. 19 is a diagram showing an example of a landscape image with a complicated traffic environment.

FIG. 20 is a flowchart showing a specific operation procedure of step ST3 in the main routine.

FIG. 21 is a flowchart showing a specific operation procedure of step ST31 in the subroutine.

FIG. 22 is a flowchart showing a specific operation procedure of step ST32 in the subroutine.

FIG. 23 is a diagram for explaining another method of dividing a captured image.

FIG. 24 is a diagram for explaining another method of dividing a captured image.

FIG. 25 is a diagram for explaining another method of dividing a captured image.

FIG. 26 is a diagram showing an example of division at the center of the screen.

FIG. 27 is a diagram in which the degree of recognition of the environment input by the driver is plotted when 10 captured images are presented to 10 drivers.

FIG. 28 is a diagram showing a recognition degree map showing the degree of recognition determined by the indicators C1 and C2.

FIG. 29 is a diagram showing a three-dimensional vector space including indices C1, C2, and C1 · C2.

FIG. 30 is a diagram illustrating a state in which a first region estimated to have a low degree of environment recognition and a second region estimated to have a high degree of environment recognition are provided in a three-dimensional vector space. Is.

FIG. 31 is a block diagram showing a functional configuration of a microcomputer that estimates a degree of environment recognition using a neural network.

[Explanation of symbols]

1 Obstacle warning device 10 Infrastructure information detector 20 Obstacle information detector 40 Environmental information detector 60 Obstacle information output section 70 Microcomputer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) B60R 21/00 626 B60R 21/00 626A 626G 628 628F G06T 7/00 100 G06T 7/00 100B 7/20 300 7/20 300Z (72) Inventor Hiroyuki Konishi Nagakute-cho, Aichi-gun, Aichi 1-41 Yokomichi, Nakamichi 1 F-term in Toyota Central Research Laboratory Co., Ltd. (reference) 5B057 AA06 AA16 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE11 CE17 DA07 DA13 DA15 DB03 DB06 DB09 DC23 DC25 DC39 5L096 AA02 AA06 AA09 BA04 CA02 DA03 FA37 FA52 FA64 GA07 GA38 GA40 GA41 GA51

Claims (10)

[Claims] 1. An image pickup unit that generates a picked-up image based on picked-up light from an environment, a distribution of changes in luminance included in the picked-up image generated by the image pickup unit, and a red-based pixel included in the picked-up image. An environment complexity calculation device comprising: a complexity calculation means for calculating the complexity of an environment based on at least one of distributions. 2. The complexity calculation unit sets a brightness division block in the captured image generated by the imaging unit, and it is determined that the brightness value of each pixel is uniform in the set brightness division block. Up to the step of dividing the luminance division block and setting a new luminance division block, the environmental complexity based on the number of luminance division blocks in which the luminance value of each pixel is uniform in the captured image. The environmental complexity calculation device according to claim 1, wherein the device calculates a degree. 3. The complexity calculation unit sets a red-colored divided block in the captured image generated by the image-pickup unit, and the set red-colored divided block is composed of only predetermined red-colored pixels, Repeatedly dividing the red-colored divided block and setting a new red-colored divided block, the environment based on the number of red-colored divided blocks configured by only the predetermined red-colored pixels with respect to the captured image. 2. The environmental complexity calculation device according to claim 1, wherein the complexity is calculated. 4. The complexity calculating unit sets a brightness division block in the captured image generated by the imaging unit, and determines that the brightness value of each pixel is uniform in the set brightness division block. Up to the step of dividing the luminance division block to set a new luminance division block, calculating the number of luminance division blocks in which the luminance value of each pixel is uniform in the captured image, A red-based divided block is set in the captured image generated by the means, and the red-based divided block is divided into new red-based divided blocks until the set red-based divided block includes only predetermined red-based pixels. By repeating the setting of blocks, the number of red-colored divided blocks composed only of the predetermined red-colored pixels is calculated for the captured image, and the number of luminance divided blocks and red The environment complexity calculating device according to claim 1, wherein the environment complexity is calculated based on the number of color system division blocks. 5. A complexity calculation device according to claim 1, and an environment recognition degree estimation means for estimating a recognition degree of the environment based on the complexity calculated by the complexity calculation device. And an environment recognition degree estimation device comprising. 6. The environment recognition degree estimating means estimates that the degree of environment recognition is low when the complexity calculated by the complexity calculating device is equal to or more than a threshold value, and is calculated by the complexity calculating device. The environment recognition degree estimation device according to claim 5, wherein when the complexity is smaller than a threshold value, it is estimated that the degree of environment recognition is high. 7. An illuminance detection means for detecting the illuminance of the environment is further provided, wherein the environment recognition degree estimation means sets the threshold value corresponding to the illuminance detected by the illuminance detection means, and uses the set threshold value. 7. The environment recognition degree estimation device according to claim 5, wherein the degree of environment recognition is estimated by using the above method. 8. A posture change detecting means for detecting a posture change of the observer is further provided, and the environment recognition degree estimating means increases the threshold value as the posture change of the observer detected by the posture change detecting means increases. 6. The threshold value is set to be smaller as the posture change of the observer detected by the posture change detection means becomes smaller, and the degree of environment recognition is estimated using the set threshold value. 7. The environment recognition degree estimation device according to claim 7. 9. The environment recognition degree estimation device according to claim 5, which estimates the degree of environment recognition for each predetermined area, and obstacle position detection means for detecting the position of an obstacle, A predetermined area including the position of the obstacle detected by the obstacle detection means, when the environment recognition degree estimation means is estimated to have a low degree of recognition of the environment, an alarm means for issuing an alarm about the obstacle; Obstacle warning device equipped. 10. The environment recognition degree estimation device divides an area excluding a lower portion of a captured image into a plurality of estimation areas, estimates an environment recognition degree for each divided estimation area, and the alarm unit When the position of the obstacle detected by the obstacle position detecting means is in any one of the estimation areas and the degree of recognition of the environment of the estimation area is estimated to be low by the environment recognition degree estimation means, priority is given. The obstacle warning device according to claim 9, wherein the obstacle warning device issues a warning about an obstacle existing in a high estimation area.