JP2007015460A - Obstacle position indicating system and method of indicating obstacle position - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an obstacle position indicating system and a method therefor capable of making sure the position where an obstacle detected by the picked-up image data is present, by lighting up light emitting bodies installed on a steered road wheel. <P>SOLUTION: The obstacle position indicating system acquires the image data picked up by an image pickup device to pick up the outside image of a vehicle, applies an image acknowledging process, detects the obstacle on the basis of the result from processing, and indicates the detected position of the obstacle, wherein the steered road wheel is equipped with a plurality of light emitting bodies, and determination is made whether present or not for obstacle in the image data acquired from the image pickup device, and if determined present, the distance from the obstacle and its direction are calculated to serve for calculation of the hazardousness of colliding on the basis of the calculated distance and direction, and in case the calculated hazardousness is larger than the prescribed value, that light emitting body should be lighted up which is positioned in the direction to indicate the position where the obstacle is present actually when viewed from the driver. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  In the present invention, when the presence of an obstacle such as a pedestrian is detected based on image data captured by an imaging device that captures the outside, the position of the detected obstacle is determined by lighting the light emitter on the steering wheel. The present invention relates to an obstacle position indicating system and an obstacle position indicating method that can be displayed.

  A vehicle such as an automobile is equipped with a far-infrared imaging device equipped with a bolometer or a pyroelectric image sensor. For example, when driving at night, the driver recognizes the presence of pedestrians and bicycles in front of the vehicle. A number of ambient monitoring systems have been developed.

  For example, when a human is imaged with a far-infrared imaging device, the human being, which is a heat source, has high brightness relative to other background objects. A conventional ambient monitoring system stores a reference pattern indicating a luminance distribution corresponding to a human body temperature distribution in advance, and pattern-matches image data captured by a far-infrared imaging device and the stored reference pattern. This makes it possible to detect the presence of a human being.

For example, in Patent Document 1, when the presence of a human being is detected as the presence of an obstacle, a warning display is displayed on the head-up display by calculating the distance and direction to the obstacle and calculating the possibility of a collision. There is disclosed a periphery monitoring device provided with means for alerting a driver, such as outputting and sounding a horn.
JP 2001-018738 A

  Many of the conventional surrounding monitoring systems described above display a warning to the driver by displaying an image of the detected obstacle on the head-up display when the presence of an obstacle such as a pedestrian or bicycle is detected. ing. However, there is a problem in that it is necessary to frequently move the line of sight between the head-up display and the front window, and there is a risk that safety may be threatened due to a reduction in time for the driver to gaze ahead.

  Further, when an obstacle image is displayed on the head-up display as in Patent Document 1, there is a difference between the displayed obstacle position and the obstacle position that the driver sees through the front window. However, since it is difficult for the driver to immediately determine the position where the obstacle actually exists, there is also a problem that it takes a considerable time to enter the avoiding operation of the detected obstacle.

  The present invention has been made in view of such circumstances, and confirms the position of an obstacle detected from captured image data by lighting a light emitter provided on a steered wheel. An object of the present invention is to provide an obstacle position indicating system and an obstacle position indicating method.

  In order to achieve the above object, an obstacle position indicating system according to a first aspect of the present invention obtains image data captured by an imaging device that images the outside of a vehicle, performs image recognition processing, and selects an obstacle based on the processing result. In the obstacle position indicating system for detecting and indicating the position of the detected obstacle, the steering wheel is provided with a plurality of light emitters, and the obstacle for determining the presence or absence of the obstacle in the image data acquired from the imaging device When it is determined that there is an obstacle by the judging means and the obstacle judging means, the obstacle position calculating means for calculating the distance and direction to the obstacle, and the risk of collision based on the calculated distance and direction. A risk calculating means for calculating; a determining means for determining whether or not the calculated risk is greater than a predetermined value; and when the determining means determines that the risk is greater than a predetermined value, an obstacle viewed from the driver Is real Characterized in that it comprises a steering wheel turned means for lighting the light emitting body located in the direction indicated position.

  The obstacle position indicating system according to a second aspect of the present invention is the obstacle position indicating system according to the first aspect, wherein the steering wheel lighting means includes a means for detecting the steering angle of the steering wheel as positive (counterclockwise) positive, Center around the center of rotation of the steering wheel at a position where a line segment connecting the means for acquiring the position of the head and the position of the acquired head and the detected position of the obstacle intersects the plane including the rotation surface of the steering wheel Means for calculating the angle with the upper apex direction of the steering wheel passing through the center of rotation as a reference direction and positive (counterclockwise) clockwise, and the position indicated by the angle obtained by subtracting the detected steering angle from the calculated center angle And a means for lighting a light emitter arranged in the vicinity.

  According to a third aspect of the present invention, there is provided the obstacle position indicating system according to the first or second aspect, wherein the steering wheel lighting means lights a light emitter located in an upper half of the steering wheel. And

  The obstacle position instruction method according to the fourth aspect of the invention acquires image data captured by an imaging device that images the outside of the vehicle, performs image recognition processing, detects an obstacle based on the processing result, and detects the obstacle. In the obstacle position instruction method for indicating the position of an obstacle, when a plurality of light emitters are provided on a steering wheel, the presence or absence of an obstacle is determined in image data acquired from the imaging device, and it is determined that there is an obstacle , Calculate the distance and direction to the obstacle, calculate the risk of collision based on the calculated distance and direction, determine whether the calculated risk is greater than a predetermined value, and the risk is a predetermined value When it is determined that the size is larger, the light emitter located in the direction indicating the position where the obstacle is actually seen from the driver is turned on.

  In the first invention and the fourth invention, a plurality of light emitters are provided on the steered wheels, the presence / absence of an obstacle in the image data acquired from the imaging device is determined, and if it is determined that there is an obstacle, The distance and direction of the vehicle are calculated, the risk of collision is calculated based on the calculated distance and direction, and when the calculated risk is greater than a predetermined value, the direction indicating the position where the obstacle is actually seen from the driver Turn on the light emitter located at. As a result, the driver can confirm the presence of an obstacle by moving the line of sight similar to the normal driving state, and it is not necessary to move the line of sight largely in order to check the position where the obstacle exists. It becomes possible to confirm the position where the obstacle exists while ensuring the safety of driving.

  In the second invention, the steering angle of the steered wheel is detected as clockwise (counterclockwise) as positive, the position of the driver's head is acquired, and the acquired position of the head and the position of the detected obstacle are determined. The central angle around the rotation center of the steering wheel at the position where the connecting line intersects the plane including the rotation surface of the steering wheel is clockwise (counterclockwise) with the upper vertex direction of the steering wheel passing through the rotation center as the reference direction. The light emitters that are calculated as positive and are arranged in the vicinity of the position indicated by the angle obtained by subtracting the detected steering angle from the calculated center angle are turned on. Thus, regardless of the angle at which the steered wheel is steered, the light emitter located in the vicinity of the direction in which the driver detects the obstacle can be turned on, and the driver can perform normal driving operation. As an extension, the presence and direction of the obstacle can be confirmed.

  In 3rd invention, the light-emitting body located in the upper half of a steering wheel is lighted. As a result, regardless of the angle at which the steered wheel is steered, the illuminator disposed in the upper half of the steered wheel is used as the illuminant located in the vicinity of the direction in which the driver detects the obstacle. It can be turned on, and the driver can surely check the presence and direction of the obstacle.

  According to the first and fourth inventions, the driver can confirm the presence of an obstacle by moving the line of sight in the same manner as in a normal driving state, and increase the line of sight to confirm the position where the obstacle exists. Since it is not necessary to move, it is possible to confirm the position where the obstacle exists while ensuring the safety of driving.

  According to the second invention, regardless of the angle at which the steered wheel is steered, the light emitter located in the vicinity of the direction in which the driver detects the obstacle can be turned on. As an extension of the driving operation, the presence and direction of the obstacle can be confirmed.

  According to the third invention, regardless of the angle at which the steered wheel is steered, it is arranged in the upper half of the steered wheel as a light emitter located in the vicinity of the direction in which the driver detects an obstacle. A certain light emitter can be turned on, and the driver can surely check the presence and direction of an obstacle.

  FIG. 1 is a diagram schematically showing a configuration of an obstacle position indicating system according to an embodiment of the present invention. In this embodiment, an obstacle is detected by an image recognition system mounted on the vehicle, the position where the obstacle exists is calculated, and the presence of the obstacle is displayed when there is a possibility of collision. Reference numerals 1 and 2 denote far-infrared video cameras (imaging devices) for imaging a pedestrian at night, a person on a bicycle, and the like. The video cameras 1 and 2 are juxtaposed in a substantially horizontal direction at an appropriate length in the front grill of the vehicle. The captured image data is transmitted to the image processing apparatus 3 connected via the in-vehicle LAN cable 7 compliant with IEEE1394.

  The image processing device 3 is connected via an in-vehicle LAN cable 7 to the video cameras 1 and 2, a display device 4 having an operation unit, and an alarm device 5 that issues an audible warning by sound, sound effects, and the like. . In addition, the image processing apparatus 3 is connected via an in-vehicle LAN cable 7 to an ECU 62 that controls lighting of a plurality of light emitters 61, 61,.

  FIG. 2 is a block diagram showing the configuration of the far-infrared video camera 1 of the obstacle position indication system according to the embodiment of the present invention. The image capturing unit 11 includes an image sensor that converts an optical signal into an electrical signal. The image capturing unit 11 reads an infrared light image around the vehicle as an RGB (R: red, G: green, B: blue) analog signal, and the read RGB signal via the internal bus 15 is a signal processing unit. 12 to send.

  The signal processing unit 12 is an LSI substrate, converts an analog signal received from the image capturing unit 11 into an RGB digital signal, a process for removing various distortions generated in the optical system, a low-frequency noise removal process, Perform correction processing to correct the gamma characteristics. Further, the RGB signal is converted into a YUV (Y: luminance, U, V: color difference) signal, and the converted YUV signal is stored in the image memory 13 as image data.

  The communication interface unit 14 is an LSI substrate and transmits / receives data to / from the image processing apparatus 3 via the in-vehicle LAN cable 7. The communication interface unit 14 sends the image data stored in the image memory 13 to the image processing device 3 in accordance with a command sent from the image processing device 3 and the transfer rate according to the resolution of the images taken by the video cameras 1 and 2. Conversion, generation of packet data for sending image data, and the like are performed.

  FIG. 3 is a block diagram showing a configuration of the image processing apparatus 3 according to the embodiment of the present invention. The communication interface unit 31 transmits commands to the video cameras 1 and 2 and receives image data from the video cameras 1 and 2. The communication interface unit 31 stores the image data received from the video cameras 1 and 2 in the image memory 32 in synchronization with each frame.

  The communication interface unit 31 transmits image data to the display device 4 such as a liquid crystal display, and transmits an output signal such as a synthesized sound to the alarm device 5 such as a buzzer or a speaker. Further, an instruction signal for instructing lighting / extinguishing is transmitted to the ECU 62 that controls lighting of the plurality of light emitters 61, 61,.

  The image memory 32 is an SRAM, a flash memory, or the like, and stores image data received from the video cameras 1 and 2 via the communication interface unit 31.

  The LSI 33 is a substrate that performs image processing, reads image data stored in the image memory 32 in units of frames, and determines whether or not a portion that is recognized as a straight line portion or a corner portion exists in the read image data. . When the LSI 33 determines that there is a part recognized as a straight line part or a corner part, the LSI 33 deletes the recognition area including the straight line part or the corner part from the image data and deletes it to the display device 4 via the communication interface unit 31. The processed image data is sent out. Further, the LSI 33 has a built-in RAM that stores data generated during the arithmetic processing.

  FIG. 4 is a block diagram schematically showing the configuration of the steered wheels 6. The steering wheel 6 is provided with LEDs as light emitters 61, 61,... Each of the light emitters 61 is connected to an ECU 62 that controls lighting / extinguishing disposed in the center of the steering wheel 6, and turns on a predetermined light emitter 61 in accordance with an instruction signal received.

  The light emitters 61, 61,... Are not limited to LEDs, and may be any light emitter such as a halogen lamp. Further, the position where the light emitters 61, 61,... Are arranged is not particularly limited as long as it is on the periphery of the steering wheel 6, but it is necessary to sensuously grasp the position where the obstacle exists. In order to make it easy, it is preferable to arrange them at regular intervals.

  Detailed processing in the LSI 33 will be described below. FIG. 5 is a flowchart showing a procedure of an obstacle detection process such as a pedestrian of the LSI 33 of the image processing apparatus 3 according to the embodiment of the present invention.

  The LSI 33 reads the image data stored in the image memory 32 (step S501), and extracts an edge portion based on the pixel value (step S502). The method for extracting the edge portion is not particularly limited. For example, as shown below, an edge portion is extracted using a Laplacian filter.

  The Laplacian filter compares the pixel value of a predetermined pixel with the pixel values of the pixels existing around the pixel, and outputs “1” when the difference value of the pixel value is larger than the predetermined threshold value. If it is smaller, '0' is output. That is, when the difference between the surrounding pixels and the pixel value is larger than a predetermined threshold, “1” is output as an edge portion. (Equation 1) is a mathematical expression showing a calculation process in a Laplacian filter when pixel values of pixels existing above, below, left and right of a predetermined pixel are used.

  In (Expression 1), f (x, y) is a pixel value of a pixel at a predetermined position in received image data, g (x, y) is a pixel value of an edge pixel, and TH is a predetermined threshold value. Respectively. A pixel having a pixel value “1” is extracted as an edge portion by a Laplacian filter.

  The LSI 33 correlates by matching a feature amount used to determine whether or not a pedestrian exists, for example, a template indicating a standard human temperature distribution and image data composed of extracted edge portions. A value is calculated (step S503), and it is determined whether or not there is an area where the correlation value is greater than a predetermined threshold (step S504).

  Note that the LSI 33 determines whether or not there is a pedestrian as a feature quantity, and includes image data (a Laplacian filter applied) including a template indicating a standard human temperature distribution and an extracted edge portion. In the case of using the correlation value R calculated by matching the subsequent image data), the correlation value R is calculated based on (Expression 2).

  In (Expression 2), N is the total number of pixels in the region to be matched, k is an integer of 0 ≦ k ≦ (N−1), and Fk is the kth in the template indicating a standard human temperature distribution. Gk indicates the pixel value of the kth pixel in the image to be matched.

  Note that, as a feature amount used to determine whether or not the extracted edge portion is an image to be recognized as a pedestrian, the above-described template showing a standard human temperature distribution and image data are matched. It is not limited to the correlation value, but it is possible to specify the area recognized as a pedestrian, such as the size of the area recognized as a pedestrian, the aspect ratio, the average value of pixel values, the variance, etc. Any amount may be used, or a combination of these may be determined.

  When the LSI 33 determines that there is an area where the correlation value is greater than the predetermined threshold (step S504: YES), the LSI 33 determines that an obstacle such as a pedestrian has been detected and detects the vehicle front center point. The distance from the obstacle and the angle from the vehicle front center point indicating the direction of the obstacle are calculated (step S505). The angle indicating the direction is calculated as a positive angle on the left side and a negative angle on the right side with respect to the traveling direction of the vehicle.

  The distance between the obstacle recognized as a pedestrian and the own vehicle, and the angle indicating the direction to the obstacle are the video data of the left and right video cameras 1 and 2 in a stereo arrangement. Is calculated by performing stereo image processing. The method of calculating the position and direction of the detected obstacle is not particularly limited to this, and an obstacle that is directly recognized as a pedestrian and the own vehicle by separately mounting a radar device at the lower front of the vehicle. And the angle indicating the direction to the obstacle may be measured.

  The LSI 33 determines whether there is a possibility of colliding with the obstacle based on the calculated position and direction of the obstacle. Specifically, the LSI 33 calculates a risk P indicating the degree of possibility of collision with the detected obstacle (step S506), and determines whether the calculated risk P is greater than a predetermined value, for example, 1. (Step S507). The risk P indicating the degree of possibility of collision with the detected obstacle can be expressed by (Equation 3) where L is the distance to the obstacle, V is the vehicle speed V, and t is the avoidable time.

(Equation 3)
P = t / (L / V)

  That is, the risk of collision increases as the risk level P is greater than 1. The risk level P is not particularly limited to (Equation 3), and any index can be used as long as it can indicate the degree of risk of collision. In this embodiment, an example is described in which the risk of collision increases as the risk P value increases, but it goes without saying that the risk of collision may increase as the risk P value decreases. Yes.

  When the LSI 33 determines that the calculated risk P is greater than 1 (step S507: YES), the LSI 33 determines that there is a possibility of a collision and communicates with the display device 4 and the alarm device 5 via the communication interface. Alarm information indicating that an obstacle has been detected is transmitted via the unit 31 (step S508).

  FIG. 6 is a flowchart showing a processing procedure for transmitting an instruction signal instructing the position of an obstacle in the LSI 33 of the image processing apparatus 3 according to the embodiment of the present invention. When the LSI 33 determines that the calculated risk P is greater than 1 (step S507: YES), the LSI 33 acquires the position of the driver's head (step S601). The position of the driver's head may be stored in the RAM as an initial value, or may be acquired as a three-dimensional coordinate by providing a room imaging device, for example, by stereo vision.

  The LSI 33 calculates a center angle around the rotation center of the steering wheel 6 at a position where a line segment connecting the acquired head position and the detected obstacle position intersects a plane including the rotation surface of the steering wheel 6. The clockwise direction (counterclockwise) is calculated as positive with the upper apex direction of the steering wheel 6 passing through as a reference direction (step S602). The method for specifying the position of the head is not particularly limited, and for example, the coordinate value of the volume center of the head may be used, or the coordinate value corresponding to the average position of the eyeball may be used.

  The steered wheel 6 rotates according to the steering. Therefore, in order to turn on the light emitters arranged in the upper half of the steering wheel 6 so that the driver can easily see, it is necessary to correct the center angle according to the steering angle.

  The LSI 33 acquires the steering angle detected as positive in the clockwise direction or the counterclockwise direction from a steering angle sensor (not shown) of the steering wheel 6 disposed outside (step S603). The LSI 33 subtracts the acquired steering angle from the calculated center angle (step S604), and corrects the position of the light emitter 61 to be lit. The LSI 33 specifies the light emitter 61 arranged at or near the position obtained by subtracting the steering angle from the calculated center angle as the light emitter 61 to be lit (step S605), and specifies the specified light emitter. An instruction signal for lighting 61 is transmitted to the ECU 62 (step S606). The ECU that has received the instruction signal lights up the light emitter 61 specified by the instruction signal.

  As shown in FIG. 4, the light emitters 61, 61,... Are arranged on the periphery of the steered wheels 6 at regular intervals, for example, at intervals of 5 degrees central angles. Therefore, among the two light emitters 61 and 61 sandwiching the corrected center angle, the light emitter 61 disposed at the position indicated by the center angle close to the corrected center angle is specified as the light emitter 61 to be lit. When the corrected center angle is the center of the two light emitters 61, 61, both may be lit or one of them may be lit.

  By doing in this way, the light-emitting body 61 located in the upper half of the steering wheel 6 can be turned on regardless of the steering angle, and it becomes easy for the driver to visually check during driving.

  As described above, according to the present embodiment, the driver can confirm the presence of an obstacle by the same line of sight movement as in a normal traveling state, and looks at the line of sight to confirm the position where the obstacle exists. Therefore, it is possible to confirm the position where the obstacle exists while ensuring the safety of driving. In addition, regardless of the angle at which the steered wheel is steered, the light emitter located in the vicinity of the direction in which the driver detects the obstacle can be turned on, and the driver can extend normal driving operation. As a result, it is possible to confirm the presence and direction of the obstacle.

It is a figure showing typically the composition of the obstacle position directions system concerning an embodiment of the invention. It is a block diagram which shows the structure of the far-infrared video camera of the obstacle position instruction | indication system which concerns on embodiment of this invention. It is a block diagram which shows the structure of the image processing apparatus which concerns on embodiment of this invention. It is a block diagram which shows typically the structure of a steering wheel. It is a flowchart which shows the procedure of obstacle detection processing, such as LSI pedestrian, of the image processing apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process sequence which transmits the instruction | indication signal which instruct | indicates the position of the obstruction of LSI of the image processing apparatus which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 Video camera 3 Image processing apparatus 4 Display apparatus 5 Alarm apparatus 6 Steering wheel 31 Communication interface part 32 Image memory 33 LSI
61 Light Emitter 62 ECU

Claims (4)

In the obstacle position indicating system that acquires image data captured by an imaging device that captures the outside of the vehicle, performs image recognition processing, detects an obstacle based on the processing result, and indicates the position of the detected obstacle.
A plurality of light emitters are provided on the steering wheel,
Obstacle determination means for determining the presence or absence of an obstacle in the image data acquired from the imaging device;
An obstacle position calculating means for calculating a distance and a direction to the obstacle when the obstacle determining means determines that there is an obstacle;
A risk level calculating means for calculating a risk level of collision based on the calculated distance and direction;
A determination means for determining whether or not the calculated risk is greater than a predetermined value;
Steering wheel lighting means for lighting a light emitter located in a direction indicating a position where an obstacle actually exists when viewed from the driver when the determination means determines that the degree of danger is greater than a predetermined value. Obstacle location indication system. The steering wheel lighting means is
Means for detecting the steering angle of the steered wheel as clockwise (counterclockwise) as positive;
Means for obtaining the position of the driver's head;
A steered wheel that passes through the center of rotation about the center angle around the center of rotation of the steered wheel at a position where a line connecting the acquired position of the head and the position of the detected obstacle intersects a plane including the plane of rotation of the steered wheel. Means for calculating positive (counterclockwise) clockwise with the upper vertex direction as a reference direction;
The obstacle position indicating system according to claim 1, further comprising means for lighting a light emitter arranged in the vicinity of a position indicated by an angle obtained by subtracting the detected steering angle from the calculated center angle. The steering wheel lighting means is
3. The obstacle position indicating system according to claim 1, wherein a light emitter located in an upper half of the steering wheel is turned on. In the obstacle position indicating method of acquiring image data captured by an imaging device that images the outside of the vehicle, performing image recognition processing, detecting an obstacle based on the processing result, and indicating the position of the detected obstacle,
A plurality of light emitters are provided on the steering wheel,
Determining the presence or absence of obstacles in the image data acquired from the imaging device;
If it is determined that there is an obstacle, calculate the distance and direction to the obstacle,
Calculate the risk of collision based on the calculated distance and direction,
Determine whether the calculated risk is greater than a predetermined value,
An obstacle position indicating method characterized in that, when it is determined that the degree of danger is greater than a predetermined value, a light emitter located in a direction indicating a position where an obstacle actually exists is viewed from the driver.

Images