JP2003178290A - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To calculate a distance between one's own vehicle and an obstacle without depending on whether or not the obstacle is already known in a vehicle periphery monitoring device. <P>SOLUTION: This vehicle periphery monitoring device 1 is composed of front side cameras 10a and 10b, an image processing means 20, a vehicle speed sensor 30, an ultrasonic wave sensor 40, an announcing processing means 50, an announcing indicator 60, an announcing buzzer 70, and an on-vehicle monitor 80. A side obstacle distance calculating means 22 is composed of a characteristic area extracting part 22a for extracting a characteristic area from image information, a characteristic area luminance acquiring part 22b for acquiring the luminance distribution of the characteristic area, a retrieving area extracting part 22c for acquiring a retrieving area, a retrieving area luminance acquiring part 22d for acquiring the luminance distribution of the retrieving area, a moving distance calculating part 22e for calculating a moving distance on an obstacle image from coincident retrieval of the two luminance distributions, and a distance calculating part 22f for calculating a distance up to the obstacle by the moving distance on the obstacle image, a camera moving distance, and a camera parameter. The announcing processing means 50 indicates proper announcing output on the basis of a calculated distance. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle surroundings monitoring device for calculating a distance between an own vehicle and an obstacle, and particularly to an obstacle for the own vehicle regardless of whether or not the size of the obstacle is known. The present invention relates to a vehicle surroundings monitoring device capable of calculating a distance to an object.

[0002]

2. Description of the Related Art A technique of calculating a distance between a vehicle and an obstacle from image information of an obstacle taken by an in-vehicle camera and issuing a warning to a driver when the distance is within a predetermined distance is known.
It is disclosed in Japanese Patent Laid-Open No. 2001-213254.

Here, as a method of calculating the distance between the own vehicle and the obstacle, an object of a known size, for example, an obstacle is a vehicle from the image information picked up by a camera mounted on the own vehicle. If there is, extract the license plate of the vehicle,
A method is known in which the distance between the vehicle and the obstacle is calculated based on the size of the license plate on the image.

[0004]

However, in the conventional method for calculating the distance between the own vehicle and the obstacle, the following method is used based on the size of the object having a known size on the image.
Since the distance between the host vehicle and the obstacle is calculated, there is no particular problem in the case of an obstacle having an object of known size.For example, in the case of an obstacle such as a telephone pole having a plurality of sizes, Would make it difficult to accurately extract obstacles.

The present invention has been made in view of the above problems of the prior art, and calculates the distance between the host vehicle and the obstacle regardless of whether the size of the obstacle is known. It is an object of the present invention to provide a vehicle surroundings monitoring device that can be used.

[0006]

(1) In order to achieve the above object, according to the present invention, an image pickup means which is mounted on a vehicle and picks up an image of a situation around the vehicle, and the image pickup means. By performing image processing on the first image information imaged at the first time by means of, a characteristic region extraction unit that extracts a characteristic region of an object existing around the own vehicle, and obtains a luminance distribution of the characteristic region. A characteristic region luminance acquisition unit, a search for extracting a search region which is a region obtained by expanding the region of the characteristic region, from the second image information captured by the image capturing unit at a second time different from the first time. A region extraction unit, a search region luminance acquisition unit that acquires the luminance distribution of the search region, searches for a matching portion between the luminance distribution of the search region and the luminance distribution of the characteristic region, and based on this search result, the characteristic region A movement amount calculation unit that calculates the movement amount on the image of Even if it does not, the vehicle surroundings monitor including an image processing unit including a distance calculation unit that calculates the distance between the object and the own vehicle from the amount of movement of the characteristic region on the image and the amount of movement of the own vehicle. A device is provided.

The characteristic region is extracted from the first image information imaged by the image pickup means at the first time, the brightness distribution of the characteristic region is acquired, and the image pickup means is operated at the second time different from the first time. A search area, which is an expanded area of the feature area, is extracted from the captured image information, the brightness distribution of the search area is acquired, and the search is performed for a portion of the search area brightness distribution that matches the feature area brightness distribution. It is possible to calculate the amount of movement of the object on the image, and to calculate the distance between the own vehicle and the characteristic object based on at least the amount of movement of the characteristic object on the screen and the amount of movement of the own vehicle.

(2) Further, in the above invention, although not particularly limited, it is more preferable that the image pickup means is mounted on a front portion of the own vehicle to pick up an image of a situation of a front side of the own vehicle.

It is possible to calculate the distance to the characteristic object existing around the front side of the own vehicle by mounting the image pickup means on the front part of the own vehicle and picking up the situation of the front side of the own vehicle. Become.

(3) Further, in the above invention, although not particularly limited, it is provided with notifying means for notifying the driver, and notifying means for instructing the notifying means to output the notification. The notification processing unit instructs the notification unit to output a notification when the distance between the vehicle and the object calculated by the distance calculation unit of the image processing unit is equal to or less than a predetermined distance value. It is more preferable to carry out.

The distance between the host vehicle and the obstacle calculated by the distance calculator of the image processing means is compared with a predetermined distance value, and the distance between the host vehicle and the obstacle calculated by the distance calculating means. However, when the distance value is equal to or less than the predetermined distance value, the notification processing means instructs the notification means to output the content of the notification, and when it is less necessary to notify the driver, that is, the own vehicle and the obstacle. When the distance to the object is larger than the predetermined distance value, the notification processing means controls so as not to output the notification content, so that frequent notifications can be eliminated and more accurate notification can be performed. Becomes

(4) Further, in the above invention, although not particularly limited, the image processing means is configured to detect the object based on a change in the width of the luminance distribution of the characteristic region and the width of the luminance distribution of the search region. The notification processing unit has a proximity determination unit that determines the proximity, and the notification processing unit outputs a notification to the notification unit based on the proximity of the object determined by the proximity determination unit of the image processing unit. It is more preferable to give the instructions.

The image processing means is provided with a degree-of-approach judging unit, and the degree-of-approach judging unit determines the brightness distribution of the characteristic region based on the change in the width of the brightness distribution of the feature and the width of the brightness distribution of the search region. When the width is smaller than the width of the brightness distribution in the search area, it is determined that the vehicle is approaching the obstacle, and when the width of the brightness distribution in the feature area is larger than the width of the brightness distribution in the search area, It is possible to determine that the host vehicle is far from the obstacle. Further, a more appropriate notification is given to the driver based on the judgment of the approaching distance or the leaving distance of the own vehicle with respect to the width of the brightness distribution in the characteristic area and the width of the brightness distribution in the search area. Is possible.

[0014]

According to the first aspect of the present invention, the amount of movement of the characteristic object on the image is calculated, and based on at least the amount of movement of the characteristic object screen and the amount of movement of the own vehicle, the own vehicle and the characteristic object are detected. Since it is possible to calculate the distance between the vehicle and the obstacle, it is possible to calculate the distance between the host vehicle and the obstacle regardless of whether the size of the feature (obstacle) is known. Become.

According to the second aspect of the present invention, it is possible to calculate the distance to the obstacle existing around the front side of the host vehicle.

According to the third aspect of the present invention, when the calculated distance is equal to or less than the predetermined distance value based on the comparison between the calculated distance between the vehicle and the obstacle and the predetermined distance value, The notification processing means instructs the notification means to output the content of the notification,
If it is not necessary to notify the driver, that is, if the calculated distance is greater than the predetermined distance value between the vehicle and the obstacle, the notification content is not output. By performing the control by the notification processing means in this manner, it becomes possible to eliminate frequent notifications and give accurate notifications to the driver.

According to the fourth aspect of the present invention, in addition to the calculation of the distance between the own vehicle and the obstacle, the own vehicle approaches the obstacle based on the change in the width on the image of the obstacle. It is possible to judge when the vehicle is moving or when the host vehicle is far from the obstacle. Furthermore, it becomes possible to make a notification based on the judgment of approaching or moving away from the obstacle of the own vehicle, and compared with the notification when only the distance between the own vehicle and the obstacle, the driver can be notified. It becomes possible to provide more appropriate notification.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment] FIG. 1 is a block diagram showing the basic arrangement of a vehicle surroundings monitoring apparatus 1 according to the first embodiment. FIG. 2 is a bird's-eye view for explaining a method of calculating a distance from the own vehicle 1003 to an obstacle existing on the side of the own vehicle 1003 (hereinafter, also referred to as side obstacle distance calculation). FIG. 3 is a diagram showing an image for explaining a method of calculating a lateral obstacle distance.

FIG. 2 is a bird's-eye view for explaining the method of calculating the lateral obstacle distance. The own vehicle 1003 at time T-Δt and time T travels along the traveling road 1005 (indicated by an arrow in FIG. 2). 2) and the position of the own vehicle 1003 at time T is shown in FIG.
The position 1004 of the host vehicle at the time T-Δt is indicated by a solid white line in FIG. Further, as shown in FIG. 2, along the traveling road 1005, on the right side with respect to the traveling direction of the host vehicle 1003, the wall 10
01 is installed side by side, and the own vehicle 1003 operates at time T-Δt.
During the time period T and the time T, the actual movement amount ΔX is moved at the own vehicle speed V in the traveling direction while maintaining a constant distance from the wall 1001. Further, as shown in FIG. 2, an electric pole 1002, which is an obstacle, is installed close to the wall 1001, and the host vehicle 1003 is viewed from the electric pole 1002 at both time T-Δt and time T. The distance Y1 is maintained in the inside y-axis direction. Further, as shown in FIG.
The telephone pole 1002 is included in the angle of view α1 of the front side camera (right) 10a at time T, and at time T-Δt.
Also included in the angle of view α2 of the front side camera (right) 10a in FIG. The method of calculating the lateral obstacle distance will be described later in detail.

As shown in FIG. 1, a vehicle surroundings monitoring device 1
Are two front side cameras 10a and 10b, an image processing unit 20, a vehicle speed sensor 30, an ultrasonic sensor 40, a notification processing unit 50, a notification indicator 60, a notification buzzer 70, and an in-vehicle monitor 80. It consists of and.

The front side cameras 10a and 10b are used for the vehicle 1
Image pickup means for picking up an image of the front side of 003, which is mounted on the left and right of the front part of the vehicle 1003. For example, in FIG. 2, the front side camera (right) 10
The optical axis OP of a is mounted on the front part of the own vehicle 1003 so that the optical axis OP is perpendicular to the right direction with respect to the traveling direction of the own vehicle 1003. Then, the front side camera (right) 10
The image information captured in a is constantly transmitted to the connected image processing means 20. The image information transmitted to the image processing means 20 is used by the image processing means 2 for image processing described later.
It is acquired inside 0 and transmitted to the in-vehicle monitor 80 installed in the vehicle compartment. Although not shown in FIG. 2, in the front side camera (left) 10b, for example, the optical axis OP of the front side camera (left) 10b is perpendicular to the traveling direction of the host vehicle 1003 in the left direction. The image information captured by the front side camera (left) 10b is always transmitted to the connected image processing unit 20. The image information transmitted to the image processing unit 20 is acquired inside the image processing unit 20 for image processing described later, and is also transmitted to the in-vehicle monitor 80 installed in the vehicle interior, and the in-vehicle monitor 80 causes the driver to operate. Is displayed against.

The vehicle speed sensor 30 is a vehicle speed detecting means for detecting the own vehicle speed V of the own vehicle 1003, and the own vehicle speed information detected by the vehicle speed sensor 30 is obtained by the image processing means 20.
And is constantly transmitted to the notification processing means 50.

The ultrasonic sensor 40 emits an ultrasonic wave, the progress of the ultrasonic wave is blocked by an obstacle, and the reflected ultrasonic wave is received, so that the time difference between the transmission and the reception of the ultrasonic wave is a time difference. It is an ultrasonic wave detecting means for detecting information, and the front side of the own vehicle 1003 is set as a detection range.
It is mounted in the front part of the own vehicle 1003. The time difference information detected by the ultrasonic sensor 40 is constantly transmitted to the connected notification processing means 50.

The image processing means 20 includes the front and side cameras 10.
The approaching object recognition means 2 is an image processing means for performing image processing on the image information picked up and transmitted by a and b.
1 and side obstacle distance calculation means 22. Approaching object recognizing means 2 constituting the image processing means 20
1 is an image corresponding to the image information captured while the vehicle is temporarily stopped near the intersection to check the condition of the intersection or while traveling slowly based on the vehicle speed information detected and transmitted by the speed sensor 30. It is a means for performing processing and recognizing an approaching object at an intersection. In addition, the lateral obstacle distance calculation means 22 constituting the image processing means 20 is
It is means for calculating the distance between the own vehicle and an obstacle existing on the side of the own vehicle.

The image processing means 20 comprises the front side camera 1
0a, 10b, vehicle speed sensor 30, and notification processing means 50
Are connected so as to transmit information, and are connected so as to transmit information to the notification processing means 50 and the in-vehicle monitor 80. The image processing means 20 receives image information from the front and side cameras 10a and 10b, receives own vehicle speed information from the vehicle speed sensor 30, and issues an instruction from the notification processing means 50 to transmit image information to the in-vehicle monitor 80. An image transmission instruction and an image processing start instruction which is an instruction to start image processing are received. Further, the image processing means 20 uses the approaching object recognition information recognized by the approaching object recognizing means 21 with respect to the image information captured and transmitted by the front and side cameras 10a and 10b, and the lateral obstacle distance calculating means 22. Obstacle distance information, which is the calculated obstacle distance Y1, is transmitted to the notification processing unit 50, and image information is transmitted to the vehicle-mounted monitor 80.

The lateral obstacle distance calculating means 22 further includes a characteristic region extracting section 22a and a characteristic region luminance acquiring section 22b.
A search area extraction unit 22c and a search area brightness acquisition unit 22d
And a movement amount calculation unit 22e and a distance calculation unit 22f. The characteristic region extraction unit 22a is means for extracting the characteristic region 200 for calculating the lateral obstacle distance from the image information captured and transmitted by the front and side cameras 10a and 10b at time T-Δt, The luminance acquisition unit 22b is a unit that acquires a characteristic region luminance distribution 400 that is the luminance distribution of the extracted characteristic region 200. Then, the search area extraction unit 22c calculates the time Δt from the time T−Δt.
Front side cameras 10a, 1 at time T, which is a later image
0b from the image information captured and transmitted by the search area 30
Is a means for extracting 0, and is a search area brightness acquisition unit 22d.
Is a means for acquiring a search area brightness distribution 500 that is the extracted brightness distribution of the search area 300. Further, the movement amount calculation unit 22e compares the characteristic region luminance distribution 400 and the search region luminance distribution 500 with each other, and the movement amount on the image of the characteristic region 200, that is, the utility pole 100 which is the obstacle in FIG.
The distance calculating unit 22f is means for calculating the moving amount Δx1 on the obstacle image, which is the moving amount in the horizontal direction on the second image.
Is at least the calculated movement amount Δx on the obstacle image.
1 and the camera movement amount ΔX in FIG. 3, which is the movement amount of the own vehicle 1003, is a means for calculating an obstacle distance Y1 which is a distance between the own vehicle 1003 and an obstacle. In addition,
The details of each of the above parts and FIG. 3 will be described later.

The notification processing means 50 calculates the distance between the obstacle and the host vehicle 1003 based on the time difference information detected and transmitted by the ultrasonic sensor 40, and compares the calculated distance with a predetermined distance. Alternatively, it is means for comparing obstacle distance information, which is the distance between the obstacle and the host vehicle 1003, calculated by the image processing means 20, with a predetermined distance value. Further, based on the result of the comparison or the approaching object recognition information transmitted from the approaching object recognizing means 21 of the image processing means 20, a connected notification indicator 60,
The notification buzzer 70 is means for transmitting a notification content output instruction, which is an instruction for outputting the notification content to the driver. The notification processing means 50 uses the vehicle speed sensor 30.
, The ultrasonic sensor 40, and the image processing means 20 are connected so that information is respectively transmitted, and instructions can be transmitted to the image processing means 20, the notification indicator 60, and the notification buzzer 70, respectively. Are connected as.

The notification processing means 50 is the ultrasonic sensor 4
Time difference information from 0, vehicle speed information from the speed sensor 30,
Obstacle distance information and approaching object recognition information are received from the image processing means 20. Further, the notification processing means 50 is based on the obstacle distance information that is the distance between the obstacle and the own vehicle 1003 or the approaching object recognition information, and the notification indicator 60.
And an informing content output instruction that is an instruction to output the informing content to the informing buzzer 70, and the in-vehicle monitor 80 of the image information is transmitted.
An image transmission instruction, which is an instruction for transmission to the image processing unit 20, and an image processing start instruction, which is an instruction for starting image processing, are transmitted to the image processing unit 20 according to the vehicle speed information. The notification processing unit 50 is transmitted from the ultrasonic sensor 40 as information for calculating the distance between the own vehicle 1003 and the obstacle according to the own vehicle speed information detected and transmitted by the vehicle speed sensor 30. It is also a means for determining whether to use the time difference information or the obstacle distance information calculated by the image processing means 20.

The notification indicator 60 is provided in the passenger compartment for outputting the obstacle distance information, which is the distance from the host vehicle 1003 to the obstacle, which is recognized and calculated by the image processing means 20, and the distance to the obstacle. The notification means is installed and visually outputs the content of the obstacle and the obstacle distance information to the driver based on the notification content output instruction transmitted from the connected notification processing means 50. Here, the notification to the driver is a simple lighting display, or
It may be output by blinking display, for example,
An obstacle distance Y1 that is the calculated distance between the own vehicle 1003 and the obstacle may be output as a numerical display.

The notification buzzer 70 detects the presence / absence of the obstacle recognized and calculated by the image processing means 20 and the own vehicle 1003.
Is a notification means for audibly notifying the driver of obstacle distance information that is the distance from the vehicle to the obstacle, and the presence or absence of the obstacle based on the notification content output instruction transmitted from the connected notification processing means 50. , The content of notification such as the distance to the obstacle is audibly output to the driver. Here, the notification to the driver may be output by a mere warning sound, or, for example, information of the obstacle distance Y1 which is the calculated distance between the own vehicle 1003 and the obstacle may be output by voice. .

The in-vehicle monitor 80 includes a front side camera 10a,
The image processing means 2 further captures and transmits the image by 10b.
It is a display unit for displaying the image information transmitted from 0, and is installed in the vehicle compartment so that the driver can visually recognize it.

The operation will be described below.

FIG. 4 is a flow chart showing the processing by the notification processing means 50 of the vehicle surroundings monitoring apparatus 1. Based on the flowchart, the notification processing means 50 will be described below.
The entire processing of the vehicle surroundings monitoring device 1 by means of will be described.

In step S50, the notification processing means 5
0 receives the own vehicle speed information detected and transmitted by the speed sensor 30. Next, in step S100,
The notification processing means 50 transmits to the image processing means 20 an image transmission instruction which is an instruction to transmit to the vehicle-mounted monitor 80 image information captured and transmitted by the front and side cameras 10a, 10b, and the image processing means 20. Transmits image information to the in-vehicle monitor 80 based on the instruction, and the in-vehicle monitor 80 outputs the transmitted image information to an image.

Next, in step S200, the notification processing means 50 causes the vehicle speed information detected and transmitted by the vehicle speed sensor 30 and a predetermined vehicle speed value (for example, 20 km /
h) is compared.

In the comparison, if the vehicle speed information is equal to or less than the predetermined vehicle speed value (YES in step S200).
Performs approaching object notification processing in step S300. In the approaching object notifying process, although the flow of the process is not particularly shown, an obstacle existing in the detection range by the notifying process means 50 based on the time difference information detected and transmitted by the ultrasonic sensor 40, or Approaching object and own vehicle 1
The distance to 003 is calculated. Further, the notification processing means 5
0 compares the calculated distance between the obstacle or the approaching object and the vehicle 1003 with a preset first predetermined distance value. Then, when the calculated distance is equal to or smaller than the first predetermined distance value, the notification content output instruction is transmitted to the notification indicator 60 and the notification buzzer 70. In response to the notification content output instruction from the notification processing means 50, the notification indicator 60 visually outputs the notification content to the driver, and the notification buzzer 70 acoustically outputs the notification content to the driver. .

In step S200, for example, when the own vehicle speed information is 0 km / h to 20 km / h (YES in step S200), that is, the own vehicle 1
If 003 is stopped or slowing down, step S
In 300, the notification processing unit 50 performs the approaching object notification process on the notification indicator 60 and the notification buzzer 70 based on the approaching object recognition information transmitted from the approaching object recognition unit 21 of the image processing unit 20. An appropriate notification content output instruction for an approaching object may be transmitted.

If it is determined in step S200 that the vehicle speed information is greater than the predetermined vehicle speed value (NO in step S200), then in step S400, the notification processing means 50 causes the image processing means 20 to perform processing. An image processing start instruction, which is an instruction to start image processing, is transmitted. Then, in accordance with the image processing start instruction, the image processing means 20 takes in the image information picked up and transmitted by the front and side cameras 10a and 10b in step S100 described above, and the own vehicle 1003 performs image processing on the image information. And an obstacle existing on the side of the own vehicle 1003, for example, a telephone pole 1002 in FIG. 2, is calculated, and obstacle distance information is the calculated distance Y1 between the own vehicle 1003 and the obstacle. Is transmitted to the notification processing means 50. The method of calculating the lateral obstacle distance in step S400 will be described in detail later.

Next, in step S500, the notification processing means 50 compares the obstacle distance information calculated and transmitted by the image processing means 20 with a preset second predetermined distance value, and the comparison is made. Based on the result, a notification content output instruction is transmitted to the notification indicator 60 and the notification buzzer 70. The first predetermined distance value and the second predetermined distance value may be equal distance values or different distance values.

The notification indicator 60 that has received the notification content output instruction outputs visual information to the driver, for example, by outputting numerical information of the obstacle distance Y1 based on the obstacle distance information. . Also, the notification processing means 5
Notification buzzer 70 that has received the notification content output instruction from 0
Is the obstacle distance Y1 based on the obstacle distance information, for example.
The numerical information of is output to the driver by voice.

By informing the driver of the content of the notification based on the distance between the own vehicle and the obstacle as described above,
It becomes possible to give an appropriate notification.

Further, in steps S300 and S500 of FIG. 4, the calculated distance value is compared with a predetermined distance value, and if the calculated distance is less than or equal to the predetermined distance value, the notification processing means 50 In the case where it is less necessary to give an instruction to output the notification content to the notification indicator 60 and the notification buzzer 70 to notify the driver, that is, the distance between the host vehicle and the obstacle is a predetermined distance. If it is larger than the value, the notification processing means 50 may control not to output the notification content. As a result, frequent notifications can be eliminated and more accurate notifications can be performed.

In step S200, the ultrasonic wave sensor is used as the information for the notification processing means 50 to calculate the distance between the own vehicle 1003 and the obstacle when the own vehicle speed information is equal to or less than a predetermined vehicle speed value. By using the time difference information detected by 40 and by using the obstacle distance information which is the obstacle distance Y1 calculated by the image processing means 20 when the own vehicle speed information is larger than the predetermined vehicle speed value, the own vehicle speed The information is selectively used for the notification process according to the above. Due to the proper use of this information, the accuracy of measurement is poor in the middle and high speed regions due to erroneous detection of wind noise, and the disadvantage of the ultrasonic sensor 40, and the accuracy of measurement is poor because a sufficient movement amount cannot be secured in the low speed region. Image processing means 2
It becomes possible to offset the disadvantage of 0 and to secure stable measurement accuracy from the low speed region to the high speed region. As a result, even when driving at a place where high speed is required, the distance between the road and the wall is relatively narrow, and the possibility of approaching the wall during driving may occur, such as on a highway. It is possible to give appropriate notification to the person.

Further, the camera and the vehicle-mounted monitor used in the conventional vehicle surroundings monitoring device are, for example, like the approaching object recognizing means 21 of the image processing means 20 described above, the own vehicle 1003 for confirming the condition of the intersection. Is often used when the vehicle is stopped or slowing down, and in the middle and high speed regions, these devices are rarely operated and the frequency of use thereof may be low. However, these devices can be used in the low speed region to confirm the condition of the intersection by the conventional technique, and can be used in the medium and high speed regions as in the first and second embodiments. This makes it possible to use these devices in the entire speed range.

FIG. 5 is a flow chart showing the process of calculating the lateral obstacle distance by the image processing means 20 in step S400 of FIG. The process of calculating the lateral obstacle distance by the image processing means 20 will be described below based on the flowchart.

Here, FIG. 2 is as described above, and is a bird's-eye view for explaining the method of calculating the lateral obstacle distance.
The front side camera (right) 10b controls the vehicle 1003 so that the optical axis OP is perpendicular to the traveling direction of the vehicle 1003.
It is mounted on the front left side of the.

Further, FIG. 3 is a diagram showing an image for explaining a method of calculating a lateral obstacle distance. The upper square in FIG. 3 shows an image when the image information captured by the front side camera (right) 10a at the position 1004 of the vehicle at time T-Δt in FIG. 2 is displayed on the screen. In this case, in FIG. 2, the telephone pole 1002 is the optical axis OP of time T-Δt during the angle of view α2 of the front side camera (right) 10a.
Since the image information captured by the front side camera (right) 10a is located on the left side of FIG. Is displayed to the left. Further, the lower square portion in FIG. 3 indicates the own vehicle 10 at time T in FIG.
The image information when the image information imaged from the front side camera (right) 10a of No. 03 is displayed on the screen is shown. In this case, in FIG. 2, since the telephone pole 1002 is located on the right side with respect to the optical axis OP at the time T in the angle of view α1 of the front side camera (right) 10a, the front side camera (right) 10a.
In the image information captured in the above, the telephone pole 1002 is displayed on the right side of the center of the image, as shown in the lower square of FIG. It should be noted that the difference between the position of the telephone pole 1002 on the image at the time T-Δt of the upper square portion in FIG. 3 and the position of the telephone pole 1002 on the image at the time T of the lower square portion is the obstacle. Telephone pole 100
2 corresponds to the moving amount Δx1 on the obstacle image, which is the moving amount on the image.

As described above, in step S200 of FIG. 4, when it is determined that the vehicle speed information detected by the vehicle speed sensor 30 and transmitted by the notification processing means 50 is larger than the predetermined vehicle speed value, the process proceeds to step S400. (NO in step S200), step S410 in FIG.
Proceed to. In step S410, the amount of movement Δx1 on the obstacle image necessary for calculating the lateral obstacle distance is calculated, and the process proceeds to step S420. Note that step S4
A method of calculating the movement amount Δx1 on the obstacle image in 10 will be described in detail later.

In step S420, the distance calculation unit 22f of the lateral obstacle distance calculation means 22 determines the vehicle speed sensor 30.
From the own vehicle speed information detected and transmitted by the inter-frame time Δt, which is the difference between the time T-Δt at which the image information in a certain frame was captured and the time T at which the image information in the next frame immediately after that was captured. The camera movement amount ΔX, which is the actual movement amount of the vehicle 1003 and the actual movement amount of the front side cameras 10a and 10b, is calculated.

Next, in step S430, the distance calculation unit 22f of the lateral obstacle distance calculation means 22 detects the vehicle speed information detected by the vehicle speed sensor 30, and the step S4.
The movement amount Δx1 on the obstacle image calculated in 10;
Camera movement amount ΔX calculated in step S420
And the mounting positions of the front and side cameras 10a and 10b, the optical axis OP
Front side camera 10 such as direction, number of pixels, angle of view, focal length, etc.
From the camera parameters of a and 10b, for example, based on the principle of triangulation, the side obstacle and the front side camera 10a,
The obstacle distance, which is the distance Y1 between 10b, is calculated. In addition, the method of calculating the distance of the side obstacle is as follows: the telephone pole 1002, which is an obstacle existing on the side of the vehicle 1003, and the front and side cameras 10a and 10b during the short inter-frame time Δt. 2 in the y-axis direction in FIG. 2 is substantially constant, that is, the own vehicle 1003 and the wall 1
The method can be adopted only when the distance from 001 is substantially constant.

Next, returning to FIG. 4, in step S500, the notification processing means 50 sets the obstacle distance information which is the obstacle distance Y1 calculated in step S430 and the preset second predetermined distance value. Based on the result of the comparison, the notification content output instruction is transmitted to the notification indicator 60 and the notification buzzer 70, and the notification indicator 6
0, the notification buzzer 70 outputs appropriate notification content to the driver in accordance with the notification content output instruction.

FIG. 6 is a flowchart showing the process of calculating the moving amount Δx1 on the obstacle image by the image processing means 20 in step S410 of FIG. The process of calculating the moving amount Δx1 on the obstacle image by the image processing means 20 will be described below based on the flowchart. 7, FIG. 9, FIG. 9, FIG. 10, FIG. 11, and FIG. 12 are images for explaining the method of calculating the movement amount Δx1 on the obstacle image, or the extracted feature area or search area and It is a figure which shows those acquired brightness distributions.

As described above, in step S200 of FIG. 4, when it is determined that the vehicle speed information detected by the vehicle speed sensor 30 and transmitted by the notification processing means 50 is larger than the predetermined vehicle speed value, the process proceeds to step S400. (NO in step S200), the image processing means 20 receives the image processing start instruction transmitted from the notification processing means 50. Next, in step S410 of FIG. 5, further,
It progresses to step S411 of FIG. Step S411
At, the lateral obstacle distance calculation means 22 of the image processing means 20 causes the front lateral cameras 10a, 1a at the time T-Δt.
The image information captured and transmitted by 0b is acquired inside the lateral obstacle distance calculation means 22. FIG. 7 shows time T-Δ
It is a figure which shows the image which displayed on the screen the image information imaged and transmitted by the front side camera (right) 10a in t, and acquired inside the said side obstacle distance calculation means 22. As shown in FIG. 7, the image information includes the traveling road 10
05 and the wall 1001 attached to the road 1005
And a telephone pole 1002, which is an obstacle installed near the wall 1001, are imaged.

Next, in step S412, the characteristic region extraction section 22a of the lateral obstacle distance calculation means 22 first
The acquired image information is subjected to horizontal differentiation for extracting a change amount of abrupt change of the signal in the image information, and the vertical edge 100 is extracted. FIG. 8 shows the characteristic region extraction unit 22a for the image information captured and transmitted by the front side camera (right) 10a at time T-Δt.
FIG. 6 is a diagram showing an image on which a process of extracting the vertical edge 100 has been performed according to FIG. In FIG. 8, the vertical edge 100 corresponding to the contour of the telephone pole 1002 that is an obstacle is extracted.

The above edge processing is not limited to the extraction of the vertical edge, but may be the extraction of an edge having a vertical component by spatial differentiation. Therefore, in the above example, the method of calculating the distance Y1 between the own vehicle 1003 and the telephone pole 1002 which is an obstacle based on the telephone pole 1002 has been described, but the distance Y1 from the obstacle is measured. The feature to be processed is a telephone pole 1002.
The present invention is not limited to the one close to the wall 1001 like the above, but a part of the wall 1001 having a vertical component edge such as a joint between walls or a poster attached to the wall, or an object attached to the wall 1001. May be The wall 10
01 itself is often unsuitable for the processing, since it is often plain or a pattern is repeatedly repeated in many cases, and does not have the characteristic of obtaining a strong edge.

Next, in step S413, the characteristic region extraction section 22a of the lateral obstacle distance calculation means 22 selects a portion having a high edge strength from the extracted vertical edges 100, and the selected portion is selected. A wide area in the horizontal direction on the screen including a portion having a strong edge strength is extracted as the characteristic area 200. In FIG. 9, the processing up to step S412 is performed on the image information captured and transmitted by the front and side cameras 10a and 10b at time T-Δt, and in step S413, the edge strength of the vertical edge 100 is strong. A portion is selected, and a wide area in the horizontal direction on the image including the portion having the strong edge strength is selected as the characteristic region 200.
It is a figure which shows the image extracted as (thick broken line part).

Next, in step S414, the characteristic region luminance distribution acquisition section 22b of the lateral obstacle distance calculation means 22.
Acquires a characteristic region luminance distribution 400, which is a horizontal luminance distribution of the extracted characteristic region 200. FIG. 10 is a diagram showing the extracted characteristic region 200 in the upper stage and the acquired horizontal characteristic region luminance distribution 400 of the extracted characteristic region 200 in the lower stage. Figure 10
In the characteristic region luminance distribution 400 in the lower part, a portion having a high waveform indicates a portion having a strong luminance, and a portion having a low waveform indicates a portion having a weak luminance. In step S414, when the characteristic region luminance distribution 400 is acquired, the horizontal position of the characteristic region luminance distribution 400 on the image is calculated, and the horizontal position on the image of the characteristic region 200 at time T-Δt is calculated. The characteristic area position information, which is the position, is stored in the lateral obstacle distance calculation means 22, for example.

Next, in step S415, the lateral obstacle distance calculating means 22 acquires the image information captured and transmitted by the front lateral cameras 10a, 10b at the time T inside the lateral obstacle distance calculating means 22. To do.

Next, in step S416, the search area extraction unit 22c of the lateral obstacle distance calculation means 22 extends the area of the above-mentioned characteristic area 200 on the image over the entire area in the horizontal direction on the image. Search area 300, which is an area expanded to
To extract. Further, the search area brightness acquisition unit 22d of the lateral obstacle distance calculation means 22 causes the extracted search area 300 to be extracted.
A search area brightness distribution 500, which is a brightness distribution in the horizontal direction of, is acquired. FIG. 11 shows the front side camera 10 at time T.
It is a figure which shows the image which extracted the search area | region 400 (thick broken line part) by the process of step S416 with respect to the image information imaged and transmitted by a and 10b. Note that FIG.
Comparing 1 with FIG. 9, it can be seen that the horizontal length of the search area 300 is longer than the horizontal length of the feature area 200.

Next, in step S417, the movement amount calculation means 22e of the lateral obstacle distance calculation means 22 first searches the search area luminance distribution 5 acquired in step S416.
In 00, a search is performed for a portion that substantially matches the pattern of the characteristic region luminance distribution 400. FIG. 12 shows step S
With respect to the search range luminance distribution 400 obtained at 416,
Feature area luminance distribution 300 acquired in step S414
It is the figure which searched the place which substantially corresponds to the pattern of. The upper part of part (a) of FIG. 12 shows the characteristic region 200 extracted in step S413, and the lower part shows the characteristic region luminance distribution 400 acquired in step S414. In addition, the upper part of the part (b) of FIG.
The search area 300 extracted in 16 is shown below,
Search area brightness distribution 5 acquired in step S416
00 is shown. In the characteristic region luminance distribution 400 in the lower part of FIG. 12A and the search region luminance distribution 500 in the lower part of FIG. 12B, the portion with a high waveform is the portion with a strong luminance, and the waveform The low part indicates a part with low brightness. In FIG. 12, the pattern of the characteristic region luminance distribution 400 in (a) is the pattern of the search region luminance distribution 500 in (b).
12 shows that the positions substantially coincide with each other.

By the above search, the position of the obstacle on the image at time T is grasped after the obstacle at time T-Δt moves on the image during the interframe time Δt. It becomes possible.

Next, in step S418, the movement amount calculating means 22e of the lateral obstacle distance calculating means 22 stores the time T- stored in the lateral obstacle distance estimating means 22 in step S414 as described above, for example. From the characteristic area position information, which is the horizontal position on the image of the characteristic area 200 at Δt, and the horizontal position on the image of the matching portion of the searched search area luminance distribution 500, the obstacle on the image information. The amount of movement Δx1 on the obstacle image, which is the amount of movement, is calculated. In FIG. 3, the moving amount Δx1 on the obstacle image, which is the moving amount of the telephone pole 1002 on the image information, is shown.

Next, returning to FIG. 5, proceeding to step S420, the distance calculation unit 22f of the lateral obstacle distance calculation unit 22
The camera movement amount ΔX is calculated from the own vehicle speed information and the inter-frame time Δt. Then, in step S430,
The distance calculation unit 22f of the lateral obstacle distance calculation unit 22 determines the vehicle speed information, the obstacle image movement amount Δx1 calculated in step 418 of FIG. 6, and step S420.
Based on the camera movement amount ΔX calculated in step S1 and the camera parameters of the front and side cameras 10a and 10b,
An obstacle distance Y1 which is a distance from 03 to the obstacle is calculated.

Next, returning to FIG. 4, in step S500, the notification processing means 50 compares the obstacle distance information, which is the calculated obstacle distance Y1, with the preset second predetermined distance value. Then, based on the result of the comparison, the notification content output instruction is transmitted to the notification indicator 60 and the notification buzzer 70, and the notification indicator 60 and the notification buzzer 70 notify the driver of the notification content according to the notification content output instruction. Is output.

As described above, the image processing is performed on the two images picked up by the difference in the inter-frame time, the movement amount on the image of the obstacle is calculated, and based on the movement amount, the own vehicle is detected. By calculating the distance to the obstacle, it becomes possible to calculate the distance between the host vehicle and the obstacle regardless of whether the size of the obstacle is known or not. By instructing the notification content based on, it becomes possible to output appropriate notification content to the driver.

In the above embodiment, the case where the host vehicle is running in parallel to the obstacle has been described, but the host vehicle 1003 is present laterally during the interframe time Δt. The vehicle surroundings monitoring device 1 when approaching an obstacle or moving away from an obstacle existing on the side of the vehicle 1003 will be described.

[Second Embodiment] FIG. 1 is a block diagram showing the basic arrangement of a vehicle surroundings monitoring apparatus 1 according to the second embodiment. FIG. 14 is a bird's-eye view for explaining the determination of the degree of approach, and FIG. 15 is a diagram showing an image for explaining when the own vehicle 1003 approaches an obstacle.

As shown in FIG. 1, the vehicle surroundings monitoring apparatus 1 according to the second embodiment is similar to that of the first embodiment.
Two front side cameras 10a and 10b, and image processing means 2
0, a vehicle speed sensor 30, an ultrasonic sensor 40, a notification processing means 50, a notification indicator 60, a notification buzzer 70, and an in-vehicle monitor 80.

The front and side cameras 10a and 10b, the vehicle speed sensor 30, the ultrasonic sensor 40, the notification processing means 50,
The notification indicator 60, the notification buzzer 70, and the vehicle-mounted monitor 80 all have the same means and configuration as in the first embodiment.

The image processing means 20 has an approaching object recognition means 21.
And the lateral obstacle distance calculation unit 22, and the approaching object recognition unit 21 is the same unit as that of the first embodiment.

The lateral obstacle distance calculating means 22 includes a characteristic region extracting section 22a, a characteristic region luminance acquiring section 22b, a retrieval region extracting section 22c, and a retrieval region luminance acquiring section 22d, as in the first embodiment. The moving amount calculation unit 22e and the distance calculation unit 22f are included, and in the second embodiment, an approach degree determination unit (not shown) is included.
The approach degree determination unit is means for determining the degree of approach of the host vehicle 1003 to the obstacle based on the width of the obstacle on the image. Here, in the judgment of the approach degree judgment unit,
Since it is determined whether the host vehicle 1003 approaches the obstacle or the host vehicle 1003 moves away from the obstacle,
Along with this, as information transmitted from the image processing unit 20 to the notification processing unit 50, obstacle approach information and obstacle separation information are added to the approach object recognition information and obstacle distance information in the first embodiment. It

In response to this, the notification processing means 50 adds the notification content output instruction based on the obstacle approach information and the obstacle separation information to the notification content output instruction transmitted to the notification indicator 60 and the notification buzzer 70, Along with this, the notification indicator 60 and the notification buzzer 70 are configured to be able to output the notification content of the content of approaching the obstacle and the content of leaving the obstacle to the driver.

The operation will be described below.

FIG. 4 is a flowchart showing the processing in the notification processing means 50 of the vehicle surroundings monitoring apparatus 1.
The processing by the notification processing means 50 in the second embodiment is
It is similar to the processing of the notification processing means 50 of the first embodiment,
The same process as the process according to the flowchart shown in FIG. 4 described above is performed.

FIG. 13 is a flow chart showing the process of calculating the lateral obstacle distance by the image processing means 20 in step S400 of FIG. Based on the flowchart, the image processing means 20 when the own vehicle 1003 approaches or departs from an obstacle during the inter-frame time Δt.
The process of determining the degree of approach of the obstacle by the method will be described below.

As described above, in step S200 of FIG. 4, when it is determined that the vehicle speed information detected by the vehicle speed sensor 30 and transmitted by the notification processing means 50 is larger than the predetermined vehicle speed value, the process proceeds to step S400. (NO in step S200), the image processing means 20 receives the image processing start instruction transmitted from the notification processing means 50. Next, proceeding to step S440 of FIG. 13, in step S440, the amount of movement Δx1 on the obstacle image, which is the amount of movement on the image of the obstacle necessary for calculating the lateral obstacle distance, is calculated. It proceeds to step S450.
The method of calculating the moving amount Δx1 on the obstacle image is the same as the process of FIG. 6 in the above-described first embodiment.

In step S450, the approach degree determination unit of the lateral obstacle distance calculation means 22 determines the degree of approach of the host vehicle 1003 to the obstacle based on the width on the image of the obstacle.

Here, a method of judging the degree of approach to the obstacle will be specifically described below with reference to FIGS. 14 and 15 and the above-described FIGS. 2 and 3.

FIG. 14 is a bird's-eye view for explaining a method of judging the degree of approach to an obstacle, which is almost the same as FIG. The difference from FIG. 2 in FIG. 14 is that the own vehicle 100
3 is the time T-
The distance Y2 between the position 1004 of the own vehicle at Δt and the telephone pole 1002 changes from the distance Y1 at the time T to the distance Y1 between the position of the own vehicle 1003 and the telephone pole 1002, that is, the own vehicle. The point 1003 approaches the direction of the telephone pole 1002 during the interframe time Δt.

Further, FIG. 15 is a diagram showing an image for explaining a method of judging the degree of approach to an obstacle. Figure 15
The upper square portion of FIG. 14 shows an image when the image information captured by the front side camera (right) 10a at the position 1004 of the vehicle at time T-Δt in FIG. 14 is displayed on the screen. In this case, the telephone pole 1002 is the optical axis OP of time T-Δt during the angle of view α2 of the front side camera (right) 10a.
Since it is located on the left side with respect to the left side, the telephone pole 1002 is displayed on the left side with respect to the center on the image captured by the front side camera (right) 10a, as shown in the upper square in FIG. Has been done. However, the position of the own vehicle is 100
4 and the telephone pole 1002 have a distance Y2 larger than the obstacle distance Y1 at time T, the width w2 of the telephone pole 1002 in the upper square image of FIG.
The width is displayed narrower than the width w1 of the telephone pole 1002 on the upper square image.

The lower square in FIG. 15 shows an image when the image information taken by the front side camera (right) 10a of the vehicle 1003 at time T in FIG. 14 is displayed on the screen. . In this case, the telephone pole 1002
Is located on the right side with respect to the optical axis OP of the time T in the angle of view α1 of the front side camera (right) 10a, and therefore, as shown in the lower square in FIG. It is displayed to the right of the center on the image captured by the front side camera (right) 10a. In addition, the own vehicle 10
03 and the telephone pole 1002 which is an obstacle are the same as the distance Y1 between the own vehicle 1003 and the telephone pole 1002 which is an obstacle in the lower square of FIG. The width w1 of the telephone pole 1002 on the partial image is shown in FIG.
It is displayed with the same width as the width w1 of the telephone pole 1002 on the image of the upper square part.

Therefore, from FIG. 15, the own vehicle 1003 is
It can be seen that the width on the image of the obstacle increases between the time T-Δt and the time T when the utility pole 1002, which is the obstacle, is approached.

As described above, the determination of the degree of approach in the second embodiment of the present invention is based on the relationship that the width of the obstacle on the image changes as the vehicle 1003 approaches or departs. Do it. When the width of the obstacle image at time T-Δt is smaller than the width of the obstacle image at time T, it is determined that the host vehicle 1003 is far from the obstacle, and the time T If there is no change between the width of the obstacle image of −Δt and the width of the obstacle image of time T, the vehicle 10
It is determined that 03 is traveling parallel to the obstacle.

Here, as a concrete process of the judgment in the approaching degree judging section, for example, the side obstacle distance calculating means 22 of the image processing means 20 is similar to the processing described in the first embodiment. The characteristic region extraction unit 22a extracts the characteristic region 200 based on the extraction of the vertical edge, and the characteristic region luminance acquisition unit 22b acquires the characteristic region luminance distribution 400,
Further, the search area extraction unit 22c extracts the search area 300, and the search area brightness acquisition unit 22d extracts the search area brightness distribution 50.
Get 0. Next, by comparing the characteristic region luminance distribution 400 and the search region luminance distribution 500 by the movement amount calculation unit 22e,
For example, a portion where the patterns of the luminance distributions in the horizontal direction are substantially similar to each other is searched, and the width of the characteristic region luminance distribution 400 of the portion where the patterns are substantially similar to each other by the approach degree determination unit on the image. And the width of the portion of the search area luminance distribution 500 where the luminance is strong on the image are compared.

In the determination, for example, by comparing the rate of change in the width of the image of the obstacle with a predetermined rate of change, for example, the host vehicle 1003 approaches the obstacle abruptly and gently. It is possible to more accurately notify the driver based on the recognition that the vehicle is approaching.

Returning to FIG. 13, when it is determined in step S450 that the vehicle 1003 is approaching or distant from the obstacle (YE in step S450).
S) Since it is necessary to immediately notify the driver regardless of the distance between the own vehicle 1003 and the obstacle, the driver does not need to calculate the own vehicle 1003 and the obstacle, and next, step S460. In the lateral obstacle distance calculation means 22,
Obstacle approach information, which is information that the own vehicle 1003 is approaching an obstacle, or obstacle separation information, which is information that the own vehicle 1003 is far from the obstacle, is informing processing means 50.
Send to.

Next, returning to FIG. 4, step S50 in FIG.
At 0, the notification processing unit 50 transmits a notification content output instruction to the notification indicator 60 and the notification buzzer 70 so as to output appropriate notification content according to the obstacle approach information or the obstacle separation information. The notification indicator 60 that has received the notification content output instruction outputs a visual notification content to the driver indicating that the driver is approaching the obstacle or moving away from the obstacle. Further, the notification buzzer 70, which has received the notification content output instruction from the notification processing means, outputs to the driver the audible notification content indicating that the driver is approaching the obstacle or moving away from the obstacle. .

Further, in the determination of step S450 of FIG. 13, the approach degree determination unit of the lateral obstacle distance estimation means 22 indicates that the width of the obstacle on the image does not change and the own vehicle 10
If 03 is determined to be traveling in parallel to the obstacle without approaching or moving away from the obstacle (NO in step S450), 1
In step S420 of 3, the distance calculation unit 22f of the lateral obstacle distance calculation means 22 calculates the camera movement amount ΔX from the own vehicle speed information and the interframe time Δt. The details of this step are the same as step S420 of FIG. 5 in the above-described first embodiment.

Next, in step S430 of FIG. 13, the distance calculation unit 22f of the lateral obstacle distance calculation means 22.
Is the vehicle speed information, the moving amount Δx1 on the obstacle image calculated in step S440 in FIG. 13, and step S4.
The obstacle distance information Y1 is calculated from the camera movement amount ΔX calculated in 20 and the camera parameters of the front and side cameras 10a and 10b. Note that the details of this step are the same as step S430 in FIG. 5 described above.

Next, returning to FIG. 4, in step S500, the notification processing means 50 causes step S430 in FIG.
The obstacle distance information calculated in step 1 is compared with a preset second predetermined distance value, and an appropriate notification is output according to the distance to the obstacle based on the comparison result. To the notification indicator 60 and the notification buzzer 70, and the notification indicator 60 and the notification buzzer 70 output the notification content to the driver.

As described above, the distance between the own vehicle and the obstacle calculated in the first embodiment is determined by determining the approach of the own vehicle to the obstacle based on the change in the width of the obstacle on the image. In addition to this, it is possible to determine the degree of approach of the host vehicle to the obstacle, and it is possible to provide a more appropriate notification to the driver.

In the first and second embodiments, the camera is mounted so that the front side is at the angle of view, but the camera is not limited to the front side and can be applied to all areas around the vehicle. Is.

The embodiments described above are described for facilitating the understanding of the present invention, and not for limiting the present invention. Therefore, each element disclosed in the above-described embodiment is intended to include all design changes and equivalents within the technical scope of the present invention.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of a vehicle surroundings monitoring device according to a first embodiment and a second embodiment of the present invention.

FIG. 2 is an overhead view for explaining a method of calculating a lateral obstacle distance according to the first embodiment of the present invention.

FIG. 3 is a diagram showing an image for explaining a method of calculating a lateral obstacle distance in the first embodiment of the present invention.

FIG. 4 is a flowchart showing processing by the notification processing means 50 of the vehicle surroundings monitoring device 1 according to the first and second embodiments of the present invention.

FIG. 5 is a flowchart showing a process of calculating a lateral obstacle distance by the image processing means according to the first embodiment of the present invention.

FIG. 6 is a flowchart showing a process of calculating a moving amount on an obstacle image by the image processing means in the first embodiment and the second embodiment of the present invention.

FIG. 7 is a diagram showing an image captured by a front side camera at a time T-Δt according to the first embodiment of the present invention.

FIG. 8 is a diagram showing an image in which vertical edges are extracted from image information captured by a front side camera at a time T-Δt according to the first embodiment of the present invention.

FIG. 9 is a diagram showing an image in which a characteristic region is extracted from image information captured by a front side camera at time T-Δt according to the first embodiment of the present invention.

FIG. 10 is a time T-Δt according to the first embodiment of the present invention.
FIG. 9 is a diagram showing a characteristic region and a characteristic region luminance distribution extracted by image processing from image information captured by the front and side cameras.

FIG. 11 is a diagram showing an image in which a search area is extracted by image processing from image information captured by a front side camera at time T in the first embodiment of the present invention.

FIG. 12 is a diagram showing a comparison between a characteristic region luminance distribution and a search region luminance distribution in the first embodiment of the present invention.

FIG. 13 is a flowchart showing a process of calculating a lateral obstacle distance by the image processing means according to the second embodiment of the present invention.

FIG. 14 is an overhead view for explaining a method of determining a degree of approach in the second embodiment of the present invention.

FIG. 15 is a diagram showing an image for explaining a method of determining a degree of approach in the second embodiment of the present invention.

[Explanation of symbols]

1 ... Vehicle surroundings monitoring device 10a ... Front side camera (left) 10b ... Front side camera (right) 20 ... Image processing means 21 ... Approaching object recognition means 22 ... Side obstacle distance calculation means 22a ... Feature area extraction unit 22b ... Characteristic region brightness acquisition unit 22c ... Search area extraction unit 22d ... Search area brightness acquisition unit 22e ... Movement amount calculation unit 22f ... Distance calculation unit 30 ... Vehicle speed sensor 40 ... Ultrasonic sensor 50 ... Notification processing means 60 ... Notification indicator 70 ... Notification buzzer 80 ... Car monitor 100 ... Vertical edge 200 ... Characteristic area 300 ... Search area 400 ... Feature area brightness distribution 500 ... Search area brightness distribution 1001 ... wall 1002 ... Telephone pole 1003 ... Own vehicle 1004 ... Position of own vehicle at time T-Δt 1005 ... Driving road α1 ... Angle of view at time T α2 ... Angle of view at time T-Δt OP ... optical axis V ... Vehicle speed ΔX ... camera movement amount w1 ... width of telephone pole on image at distance Y1 w2 ... Width of electron column on image at distance Y2 Δx1 ... Amount of movement on the obstacle image Y1 ... Distance from obstacle (telephone pole) Y2 ... Distance from obstacle (telegraph pole)

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI Theme Coat (reference) B60R 21/00 B60R 21/00 621M 622 622C 622F 622L 622Q 622T 624 624C 624E 626 626B 626B 001G 612G 612G 601G 11/00 H G06T 7/00 200 G06T 7/00 200B 250 250 7/20 7/20 B 7/60 180 7/60 180B H04N 7/18 H04N 7/18 JF term (reference) 2F065 AA06 AA09 FF64 FF67 JJ03 JJ05 JJ19 JJ26 QQ31 SS02 SS13 SS15 5B057 AA16 CA12 CH01 DA08 DA16 DC16 DC23 DC32 5C054 AA04 CA04 CE12 CH01 DA01 EA01 FC04 FC12 FC15 FE07 FF06 HA30 5L096 BA02 BA04 CA04 DA03 FA06 FA37 FA66 GA08 HA04

Claims (4)

[Claims] 1. An image pickup means which is mounted on a host vehicle and which captures an image of a situation around the host vehicle, and which performs image processing on first image information imaged at a first time by the image capturing means. A characteristic region extraction unit that extracts a characteristic region of an object existing around the vehicle, a characteristic region luminance acquisition unit that obtains a luminance distribution of the characteristic region, and a second region different from the first time by the imaging unit. A search area extraction unit that extracts a search area that is an area obtained by expanding the area of the characteristic area from second image information captured at time, a search area brightness acquisition unit that acquires a brightness distribution of the search area, and the search And a movement amount calculation unit that calculates a movement amount on the image of the characteristic region based on the search result by searching a matching portion between the luminance distribution of the characteristic region and the luminance distribution of the characteristic region, and an image of the characteristic region. The amount of movement above and the amount of movement of the vehicle Vehicle surrounding monitoring device including an image processing unit, a containing distance calculation unit that calculates a distance between the vehicle and the object from. 2. The image pickup means is mounted on a front portion of the own vehicle and picks up an image of a situation of a front side of the own vehicle.
The vehicle surroundings monitoring device described. 3. An informing means for informing a driver, and an informing processing means for instructing the informing means to output an informing, wherein the informing processing means is the distance of the image processing means. 3. The vehicle surrounding monitoring device according to claim 1, wherein when the distance between the vehicle and the object calculated by the calculation unit is equal to or less than a predetermined distance value, the notification unit is instructed to output a notification. 4. The image processing means, based on a change in the width of the brightness distribution of the characteristic area and the width of the brightness distribution of the search area,
An approach degree determination unit that determines the degree of approach of the object, the notification processing unit, based on the approach degree of the object determined by the approach degree determination unit of the image processing unit,
The instruction to output the notification is given to the notification means.
The vehicle surroundings monitoring device according to any one of 1 to 3.