JPH11112968A - Monitor for environment inside and outside of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the monitor for environment inside and outside of a vehicle with good economy and utilizing efficiency, while the installation area is saved by using image information photographed by a single CCD camera by plural electronic devices. SOLUTION: A CCD camera 4 having an ultra-wide angle lens 3 is placed to a front part of a ceiling of a vehicle 2. A monitor is provided with an image processing unit 5 that processes image information photographed by the CCD camera 4, with an image recorder 6, that records the image information processed by the image processing unit 5 and provides an output of the recorded image information in response to a request, with a monitor display device 7 that displays the image information processed by the image processing unit 5. The vehicle 2 is provided with plural on-vehicle electronic devices 8. The image processing unit 5 supplies data to each on-vehicle electronic device 8 so as to allow each on-vehicle electronic device 8 for executing respective functions.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle interior / external environment monitoring device which is mounted on a vehicle such as an automobile and monitors the environment inside and outside the vehicle and appropriately operates a vehicle-mounted electronic device according to the situation.

[0002]

2. Description of the Related Art Conventionally, various on-vehicle electronic devices mounted on an automobile have a structure in which necessary information and signals are obtained from sensors and the like provided for each on-vehicle electronic device when executing its function. It had been.

When image information is required,
Each of the on-vehicle electronic devices has a structure provided with an imaging unit such as a CCD camera. For example, JP-A-4-27493
No. 5, JP-A-5-238310, JP-A-5-238310
As disclosed in Japanese Patent Publication No. 238314.

[0004]

However, according to the above-mentioned conventional system, sensors and CCs for each on-vehicle electronic device are required.
Since a plurality of D cameras and the like are individually installed in the vehicle, there is a problem that it is necessary to secure a large installation area.

[0005] Further, according to the method of installing a CCD camera for each vehicle-mounted electronic device, an expensive camera and a plurality of image processing devices for processing image information captured by the camera are required, which is economically inefficient. Further, some on-vehicle electronic devices do not always require data, and have a drawback that utilization efficiency is poor.

Accordingly, an object of the present invention is to save space in an installation area and to improve economical efficiency and utilization efficiency by using image information captured by a single image capturing means in a plurality of vehicle-mounted electronic devices. It is an object of the present invention to provide an environment monitoring device inside and outside a vehicle.

[0007]

Technical means for solving the above-mentioned problems include a single image pickup means installed at the front of the ceiling of a vehicle and capable of picking up an image of a wide range, An image processing device that processes the processed image information, an image recording device that records the image information processed by the image processing device, and outputs the image information recorded according to a request, and an image processing device that processes the image information. A display for displaying the captured image information, a plurality of in-vehicle electronic devices mounted on the vehicle, and data required for each in-vehicle electronic device by the image processing device from the captured image information, respectively. In this case, each of the on-vehicle electronic devices is capable of executing its function by supplying the data corresponding to the respective functions.

[0008] The image information of the inside and outside of the vehicle may be constantly collected by the image pickup means, and the image information may be divided by the image processing device, and the image information may be corrected for distortion and displayed on the display. .

Further, an in-vehicle LAN for bidirectionally transmitting and receiving data between the image processing apparatus and the plurality of on-vehicle electronic devices.
May be configured so that the plurality of on-vehicle electronic devices can execute respective functions based on data processed by an image processing device from the captured image information.

Further, image information of the inside and outside of the vehicle is constantly collected by the image pickup means, and the image information is divided by the image processing device and distortion-corrected for each, and sequentially recorded in the image recording device. The image processing device may be configured to output the image data to a display device in response to a request.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. As shown in FIGS.
The environment monitor device 1 inside and outside the vehicle includes a CCD camera 4 equipped with an ultra-wide-angle lens 3 or a fish-eye lens as a single image pickup unit mounted on the vehicle 2 and capable of imaging a wide range,
An image processing device 5 that processes image information captured by the D camera 4, an image recording device 6 that records image information processed by the image processing device 5, and outputs image information recorded according to a request. Monitor display 7 as a display for displaying image information processed by image processing device 5
And a plurality of in-vehicle electronic devices 8.

The CCD camera 4 is housed at a position in front of the ceiling of the vehicle 2, for example, in an overhead console in the vehicle, in order to collect image information of the inside and outside of the vehicle over a wide range.

The image processing device 5 and the image recording device 6 are installed below a seat, and the on-vehicle electronic devices 8 are appropriately installed in a vehicle, a body, an engine room, and the like.

Further, the CCD camera 4 and the image processing device 5, and the image processing device 5 and the monitor display 7 respectively include a coaxial cable 11 for transmitting a video signal,
12, the image processing device 5 and the image recording device 6
Is connected by a communication cable 13 constituting a high-speed LAN, and the image processing apparatus 5 and the on-vehicle electronic devices 8a, 8b, 8
c, 8d and 8e are connected in a bus shape by a communication cable 14 constituting a low-speed LAN.

Then, image information over a wide area inside and outside the vehicle is constantly collected by the CCD camera 4 and transmitted to the image processing device 5.

The image processing device 5 performs image processing such as division, synthesis, distortion correction, and the like on the transmitted image information, and sends the image-processed image data to the image recording device 6 and the monitor display 7, In accordance with an event such as a timing or a data request from the electronic device 8, a necessary image area and comparison data in the image information, a processing method (correlation, etc.) are determined, and the processing is controlled so that required processing data is sent back. I have.

In the image recording device 6, the image processing device 5
Is controlled so that the image data sent from the printer is sequentially recorded and output to the image processing apparatus 5 side in response to a request.

The driver can operate the monitor display 7 to display desired image information on the monitor display 7 through the image processing device 5 and visually recognize the image information, if necessary. In this case, the image information recorded in the image recording device 6 can be appropriately called out to the monitor display 7 through the image processing device 5 in response to a request from the monitor display 7 side.

The priorities of the events from the on-vehicle electronic devices 8 are determined in advance. For example, in this embodiment, the priority is set higher for the one related to traveling safety. That is, as shown in FIG. 2, the priority order of each vehicle-mounted electronic device 8 is as follows: airbag control device 8a, vehicle operation monitoring device 8b, seat belt control device 8c, power window control device 8d, and wiper control device 8e. Have been.

The image data for comparison in each event is controlled so as to be updated at a predetermined cycle, and erroneous determination due to a change in environment inside and outside the vehicle 2 is effectively prevented.

Each of the on-vehicle electronic devices 8 performs an action (on / off of the actuator, permission / prohibition of the vehicle operation, etc.) for the vehicle 2 according to each function based on the processing data supplied from the image processing device 5. Is controlled to run.

Next, the operation of each event in the vehicle inside / outside environment monitoring device 1 will be described.

FIG. 3 shows an example of processing when functioning as an environmental image recorder. The entire image information captured by the CCD camera 4 is constantly sent to the image processing device 5.

In the image processing apparatus 5, a destination area designating section 5a
After that, area designation data for dividing the entire image information into a predetermined area at a predetermined cycle (for example, a cycle of several minutes) is sent to the target area selection unit 5b and the image database 5c.

The target area selecting section 5b divides the image information sent from the CCD camera 4 into areas based on the area designation data and corrects the distortion, and stores the respective target area image data in the image database 5c. It is sent to the comparison processing unit 5d and the image recording device 6.

In the image database 5c, each target area image data sent from the target area selecting section 5b is classified and stored based on the area specifying data from the target area specifying section 5a. The target area image data is sent to the image comparison processing unit 5d.

The image comparison processing section 5d compares the target area image data sent from the target area selection section 5b with the previous target area image data sent from the image database 5c. For example, there are a processing method of performing a region limitation to obtain a correlation, a processing method of performing a region limitation, extracting an edge, thinning, extracting a feature, and matching a feature.

Then, processing data such as correlation value data which has been subjected to comparison processing by the image comparison processing section 5d is sent to the judgment section 5e. Based on the processing data, the judgment section 5e determines in advance whether or not there is an environmental change in each target area image data. Judge from the set threshold value.

Thereafter, the determination unit 5e sends the presence or absence of an environmental change to each target area image data to the image recording apparatus 6 as recording valid / invalid data, and the image recording apparatus 6 sends the data from the target area selection unit 5b. Of the respective target area image data, the target area image data determined as having an environmental change is rewritten and recorded, and the target area image data determined as having no environmental change is canceled without being rewritten and recorded.

As a recording medium of the image recording device 6,
A nonvolatile memory, a magnetic tape, an optical disk device, or the like may be used.

Here, the image recording device 6 performs
Each recorded target area image data is updated or maintained. Then, the airbag operation signal or the like is used as a trigger so that each target area image data recorded at that time is controlled to be frozen.

Accordingly, past image information immediately before an accident or the like can be obtained, and the function as an environmental image recorder can be exhibited.

Next, an example of processing in the vehicle operation monitoring device 8b will be described.

As shown in FIG. 4, the vehicle operation monitoring device 8
b monitors the vehicle state such as the stop state and the operation state of the engine of the vehicle 2, and when the driver starts the engine by operating the ignition key, the vehicle operation monitoring device 8b performs image processing. A driver identification request is output to the target area selection unit 5b, the image database 5c, and the determination unit 5e of the device 5.

On the other hand, as described above, the entire image information picked up by the CCD camera 4 is constantly sent to the image processing device 5, and the target area selecting section 5b receiving the driver identification request sends the information from the CCD camera 4. The driver's face area is extracted from the transmitted image information, and the distortion is corrected.
The data is sent to the image comparison processing unit 5d as face image data.

The image database 5c of the image recording device 6 and the image processing device 5 pre-registers the driver's face image and face image data such as characteristic data, and receives the driver identification request. Image database 5c
Is used to store the registered face image data of the registrant in the image comparison processing unit 5
Send to d. At this time, a method may be used in which the registrant's face image data is called from the image recording device 6 and sent to the image comparison processing unit 5d.

The image comparison processing section 5d compares the face image data sent from the target area selection section 5b with the registered face image data sent from the image database 5c. For example, as the comparison processing, there is a comparison processing method as described above.

Then, processing data such as correlation data and the like subjected to comparison processing by the image comparison processing section 5d is sent to the judgment section 5e, and the judgment section 5e determines whether or not the driver matches the registrant based on the processing data. Judgment is made from a preset threshold.

Thereafter, the determination unit 5e sends a determination as to whether or not the driver matches, to the vehicle operation monitoring device 8b as match / mismatch data, and the vehicle operation monitoring device 8b sends the match / mismatch data. If it is determined that the data coincide with each other, a determination is made that the vehicle operation is permitted, and the subsequent driving operation is controlled to be possible. Further, if it is determined that the data does not match or the data does not match, the vehicle operation is prohibited, and control is performed so that subsequent driving operations such as stopping the engine are disabled.

Therefore, the driver other than the registrant is
Operation operation can be effectively prevented.

Next, an example of processing in the airbag control device 8a will be described.

As shown in FIG. 5, the airbag control device 8a monitors the state of the vehicle, and the airbag control device 8a is triggered by an opening / closing signal of a door and an on / off signal of an ignition key. A passenger seat occupant check request is output to the target area selection unit 5b, the image database 5c, and the determination unit 5e.

On the other hand, as described above, the entire image information picked up by the CCD camera 4 is always sent to the image processing device 5, and the target area selecting section 5b receiving the passenger seat occupant check request sends the CCD camera 4 Take out the passenger seat area from the image information sent from
The data is sent to the image comparison processing unit 5d as the passenger seat image data.

In the image recording device 6 and the image database 5c of the image processing device 5, the image data of the passenger at the normal time when the passenger is not in the passenger seat is registered in advance. The image database 5c that has received the check request sends the normal-time passenger seat image data to the image comparison processing unit 5d. At this time, a method may be used in which the image data of the passenger seat at the normal time is called from the image recording device 6 and sent to the image comparison processing unit 5d.

The image comparison processing section 5d compares the passenger seat image data sent from the target area selecting section 5b with the stationary passenger image data sent from the image database 5c. As the comparison processing method, there is the above-described comparison processing method and the like.

Then, processing data such as correlation value data and the like subjected to comparison processing by the image comparison processing section 5d is sent to the judgment section 5e, and the judgment section 5e determines the presence or absence of an occupant in the passenger seat based on the processing data based on a predetermined threshold value. Is determined from.

Thereafter, the determination section 5e sends a determination as to whether or not an occupant is present in the passenger seat to the airbag control device 8a as occupant presence / absence data, and the airbag control device 8a transmits the occupant presence / absence data based on the transmitted occupant presence / absence data. If it is determined that there is an occupant, the passenger airbag is activated based on the airbag activation signal in an emergency, and if it is determined that there is no occupant,
Even if there is an airbag activation signal in an emergency, the passenger airbag is not activated and the standby state is maintained.

Therefore, unnecessary activation of the passenger airbag can be effectively prevented.

The driver's seat airbag is operated based on the airbag operation signal as before, regardless of the control operation of the airbag control device 8a.

Next, an example of processing in the seat belt control device 8c will be described.

As shown in FIG. 6, the seatbelt control device 8c monitors the vehicle condition similarly to the airbag control device 8a, and the seatbelt control device 8c uses the seatbelt control signal as a trigger by a door open / close signal and an ignition key on / off signal. Control device 8
c outputs a seat occupant check request to the target area selection unit 5b, the image database 5c, and the determination unit 5e of the image processing apparatus 5.

On the other hand, as described above, the entire image information picked up by the CCD camera 4 is constantly sent to the image processing device 5, and the target area selecting section 5b receiving the seat occupant check request sends the information from the CCD camera 4. The driver's seat area, the passenger's seat area, and the rear seat area are extracted from the transmitted image information, corrected for distortion, and sent to the image comparison processing unit 5d as seat image data.

In the image database 5c of the image recording device 6 and the image processing device 5, steady-state seat image data in a state where no occupant is on the driver seat, the passenger seat, and the rear seat are registered in advance. The image database 5c that has received the seat occupant check request sends the steady state seat image data to the image comparison processing unit 5d. On this occasion,
Calling the normal-time seat image data from the image recording device 6,
A method of sending the image data to the image comparison processing unit 5d may be used.

The image comparison processing unit 5d compares the seat image data sent from the destination area selection unit 5b with the steady-state seat image data sent from the image database 5c. As the comparison processing method, there is the above-described comparison processing method and the like.

Then, processing data such as correlation value data and the like subjected to the comparison processing by the image comparison processing section 5d is sent to the judgment section 5e, and the judgment section 5e determines the presence or absence of an occupant in each seat based on the processing data based on a predetermined threshold value. Is determined from.

Thereafter, the determination unit 5e sends a determination as to whether or not there is an occupant in each seat to the seatbelt control device 8c as occupant presence / absence data, and the seatbelt control device 8c transmits the occupant presence / absence data based on the transmitted occupant presence / absence data. If the seat is determined to have an occupant in each seat, the seat belt is controlled so as to restrict the free extension of the seat belt, and if the seat is determined to be occupantless, the seat belt is controlled to allow the free extension of the seat belt. You.

Therefore, the extension of the seat belt in an emergency such as a collision can be effectively prevented.

Next, an example of processing in the power window control device 8d will be described.

As shown in FIG. 7, the power window control device 8d monitors a window operation, and when the window is opened or closed, the power window control device 8d operates the target area selection section of the image processing device 5. 5b,
The window part check request is output to the image database 5c and the determination part 5e.

On the other hand, as described above, the entire image information picked up by the CCD camera 4 is constantly sent to the image processing device 5, and the target area selecting section 5b, which has received the window portion check request, sends the information from the CCD camera 4. The window area is extracted from the transmitted image information and the distortion is corrected, and is sent to the image comparison processing unit 5d as window image data.

In the image database 5c of the image recording device 6 and the image processing device 5, the normal window image data in the open state where no human body or other objects exist in the window is registered in advance. The image database 5c that has received the window check request sends the normal window image data to the image comparison processing unit 5d. At this time, a method may be used in which the normal window portion image data is called from the image recording device 6 and sent to the image comparison processing portion 5d.

In the image comparison processing unit 5d, the window image data sent from the target area selection unit 5b is
A comparison process is performed with the normal window portion image data sent from the image database 5c. As the comparison processing method, there is the above-described comparison processing method and the like.

Then, processing data such as correlation value data which has been compared by the image comparison processing unit 5d is sent to the judgment unit 5e. Based on the processing data, the judgment unit 5e determines in advance whether or not there is a human body or an object in the window. Is determined from the threshold value.

Thereafter, the determination unit 5e sends a determination as to whether or not a human body or the like exists in the window to the power window control device 8d as in-window presence / absence data. Based on the presence / absence data, if it is determined that a human body or the like is present, the window driving motor is controlled to be off regardless of the window operation signal, and if it is determined that no human body or the like is present, the window is operated according to the window operation signal. The window is controlled to be driven up and down.

Accordingly, it is possible to effectively prevent a human body or the like from being pinched in the power window.

Next, an example of processing in the wiper control device 8e will be described.

As shown in FIG. 8, the wiper control device 8
e monitors the surface of the windshield, and outputs a rainfall check request to the target area selection unit 5b, the image database 5c, and the determination unit 5e of the image processing apparatus 5 at a predetermined cycle.

On the other hand, as described above, the entire image information picked up by the CCD camera 4 is always sent to the image processing device 5, and the target area selecting section 5b receiving the rainfall check request sends the information from the CCD camera 4. The windshield area is extracted from the received image information and the distortion is corrected, and is sent to the image comparison processing unit 5d as windshield image data.

The image database 5c of the image recording device 6 and the image processing device 5 has previously registered fine weather image data of the windshield portion in a non-rainfall state, and receives the rainfall check request. 5c sends the fine weather image data to the image comparison processing unit 5d. At this time, a method may be used in which fine weather image data is called from the image recording device 6 and sent to the image comparison processing unit 5d.

The image comparison processing section 5d compares the windshield image data sent from the target area selection section 5b with the fine weather image data sent from the image database 5c. As the comparison processing method, there is the above-described comparison processing method and the like.

Then, processing data such as correlation data and the like subjected to comparison processing by the image comparison processing unit 5d is sent to the judgment unit 5e, and the judgment unit 5e judges the presence or absence of rain from a preset threshold based on the processing data. .

Thereafter, the determination unit 5e sends a determination as to whether or not there is rain to the wiper controller 8e as rain / non-presence data, and the wiper controller 8e determines that there is rain based on the sent rain / presence data. If so, the wiper drive motor is turned on, and if it is determined that there is no rainfall, the wiper drive motor is turned off.

Therefore, an automatic wiper operation according to the presence or absence of rain can be performed.

Although a method of detecting rainfall from a change in image data on the windshield surface is shown, a method of controlling the wiper by detecting solar radiation and rainfall from a change in image data in a landscape image outside the vehicle may be used.

As described above, the image information captured in a wide range by the single CCD camera 4 and the single image processing device 5 is appropriately divided into necessary areas and used by a plurality of on-vehicle electronic devices 8. Therefore, it is possible to save space in the installation area as compared with a case where a plurality of imaging units, sensors, and the like are individually installed.

In addition, a single super-wide-angle lens 3, a CCD camera 4, and a single image processing device 5 are used for each of the on-vehicle electronic devices 8, and there is also an advantage that economy is excellent.

Further, by supplying image information to each of the on-vehicle electronic devices 8 as needed, utilization efficiency can be improved.

Further, by arranging the super wide-angle lens 3 and the CCD camera 4 at the front part of the ceiling of the vehicle 2, there is an advantage that image information inside and outside the vehicle can be effectively secured over a wide range.

Further, a method of collecting image information over a wide range using the ultra wide-angle lens 3 or the like can be displayed on the monitor display 7 without any discomfort by performing distortion correction.

In the above-described embodiment, each example of the on-vehicle electronic device 8 is shown. However, the present invention is not limited to these, and can be used for identifying obstacles, pedestrians, white lines, etc. on the road. Can also be used to control

[0081]

As described above, according to the vehicle inside / outside environment monitoring apparatus of the present invention, a single image pickup means installed at the front of the ceiling of the vehicle and capable of picking up an image of a wide area, and an image picked up by the image pickup means An image processing device that processes the processed image information, an image recording device that records the image information processed by the image processing device, and outputs the image information recorded according to a request, and an image processing device that processes the image information. A display that displays the image information, a plurality of in-vehicle electronic devices each mounted at a predetermined position of the vehicle, and extracts data necessary for each in-vehicle electronic device from the image information captured by the image processing device, Each of the on-vehicle electronic devices can execute its function by supplying data corresponding to each on-vehicle electronic device. More can be automated control operation associated with the various vehicle, the attained space saving of installation area, there is an advantage of excellent economy and efficiency.

[Brief description of the drawings]

FIG. 1 is a schematic explanatory view showing an embodiment of the present invention.

FIG. 2 is a block diagram of the same.

FIG. 3 is a block diagram in a processing example.

FIG. 4 is a block diagram in a processing example.

FIG. 5 is a block diagram in a processing example.

FIG. 6 is a block diagram in a processing example.

FIG. 7 is a block diagram in a processing example.

FIG. 8 is a block diagram in a processing example.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 vehicle inside and outside environment monitoring device 2 vehicle 3 super wide angle lens 4 CCD camera 5 image processing device 6 image recording device 7 monitor display 8 in-vehicle electronic device 8a airbag control device 8b vehicle operation monitoring device 8c seat belt control device 8d power window control device 8e Wiper control device 11 Coaxial cable 12 Coaxial cable 13 Communication cable 14 Communication cable

Claims (4)

[Claims] A single imaging unit installed at a front part of a ceiling of a vehicle and capable of imaging a wide area; an image processing apparatus for processing image information captured by the imaging unit; An image recording device that records the processed image information and outputs the recorded image information in response to a request, a display that displays the image information processed by the image processing device, and a plurality of devices mounted on the vehicle. The in-vehicle electronic device and the data necessary for each in-vehicle electronic device are respectively extracted from the captured image information by the image processing device, and the data corresponding to each in-vehicle electronic device is supplied. A vehicle interior and exterior environment monitoring device characterized in that the function can be executed. 2. The apparatus according to claim 1, wherein image information of the inside and outside of the vehicle is constantly collected by the image pickup means, and the image information is divided by the image processing device and distortion-corrected for each of the image information and displayed on the display. Claim 1
The environment monitoring device inside and outside the vehicle according to the above description. 3. An in-vehicle LAN for bidirectionally transmitting and receiving data between the image processing apparatus and the plurality of in-vehicle electronic devices, and based on data processed by the image processing apparatus from the captured image information. 3. The environment monitoring device according to claim 1, wherein the plurality of on-vehicle electronic devices are capable of executing their respective functions. 4. The image information inside and outside the vehicle is constantly collected by the image pickup means, and the image information is divided by the image processing device and the distortion is corrected for each of the image information. 4. The environment monitoring device according to claim 1, wherein the information is output to the image processing device and a display in accordance with the condition.