JPH1143009A - Vehicle optical recognizing and monitoring system making use of pattern recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent an injury caused by expansion of an air bag and monitor the safety of an occupant by a proper computer recognizing system adapting a pattern recognizing system by irradiating an electromagnetic radiation, particularly, infrared ray to the content in a vehicle and an object out of the vehicle, and forming the image of the content on a charge coupled element by use of a lens. SOLUTION: When a driver 101 sits in a seat 102 in the rear of a handle 103 including an air bag device 104, five transmitter assemblies receiver assemblies 110, 111, 112, 113 and 114 measure the relative positions to an air bag of the head, breast, and body of the driver. A control circuit 130 is connected to transmitter/receiver 110-114 to control the transmission from the transmitter. This system comprises a microprocessor including a proper software such as pattern recognizing algorithm and other proper driver, signal regulator, signal generator and the like. The presence of a person on the seat is recognized, and this information can be used to give an influence on the operation of the system.

Description

DETAILED DESCRIPTION OF THE INVENTION

CROSS REFERENCE TO RELATED APPLICATIONS [0001]
7 / 878,571 (currently abandoned), and 1993
U.S. Patent Application Serial No. 08/04, filed March 31, 2016
No. 0,978, U.S. patent application Ser. No. 08 / 247,760, filed May 23, 1994, and U.S. patent application Ser.
No. 978, which is a continuation-in-part application, the last three being incorporated herein by reference.

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical confirmation and monitoring system for vehicles using pattern recognition.

[0003]

[Prior art]

1. Prior Art Regarding Occupants Not Sitting in Place and Back Seats for Children While airbags have saved thousands of lives, deployment of airbags has injured many, some seriously injured. I have. Therefore, a significant improvement in this regard is desired. REFERENCES U.S. Patent Application Ser.
As discussed in detail in US Pat. No. 9,978, vehicle occupants may be too close to the airbag before the airbag deploys for various reasons, resulting in serious injury from the deployment of the airbag, Or death. In addition, the rear type (rear-facing) placed in the right front passenger seat
Children in child seats are at risk of severe damage if the passenger airbag is deployed. For these reasons and the 1993 International Conference on Seatbelts in Canada,
and Airbags)
d, DS) requires the detection of the occupant position and the detection of the rear child seat, as first announced by "How Airbags Work" presented by .

[0004] Originally, these systems solve the problems of unoccupied occupants and back seats associated with current airbag systems, and are unnecessary when there are no people in the front seats. This is for preventing the deployment of the airbag. However, at present, development of an airbag for protecting a rear seat occupant in the event of a vehicle collision and protecting all occupants in the event of a side impact is being pursued. Therefore, a system that detects the presence of an occupant, determines whether or not the occupant has moved from a predetermined position, and confirms whether or not a rear-type child seat is present in the rear seat will be needed. It is believed that future vehicles will be equipped with eight or more airbags due to the pursuit of protection for rear seat occupants and protection from side impacts. In addition to eliminating confusion and possible harm from unnecessary airbag deployment, the cost of replacing these airbags would be excessive if all airbags were unnecessarily deployed in the event of an accident. It will be.

At present, there is an inflator for adjusting the amount of gas flowing into an airbag in consideration of the size and position of an occupant and the degree of an accident. US Patent Application 08/2
A vehicle identification and monitoring system (VIMS), discussed in 39,978, controls an inflator depending on the presence and location of a vehicle occupant or rear-facing child seat. The present invention relates to such a VI
An improvement to the MS system, which uses an advanced optical system including one or more CCD arrays (charge coupled device arrays) and illumination sources in combination with a trained neural network based pattern recognition system.

Others have already pointed out the need for a sensor to detect that the occupant is moving from a predetermined position, and several methods for ascertaining the position of a vehicle occupant have already been disclosed in US patents. I have. However, each of these systems has significant limitations. For example, White et al. (U.S. Pat. No. 5,071,160)
Discloses a single acoustic sensor / detector, which is mounted below the handle as shown. White et al. Correctly recognize that if one of the occupants adjusts the control knob on the radio, those sensors could malfunction and the airbag could be deployed incorrectly, thus eliminating such sensors. We recommend using more than one.

[0007]

Mattes et al. (U.S. Pat. No. 5,118,134) disclose ultrasonic, active or passive infrared and ultrashort wave radar sensors as a method of measuring changes in occupant position. And various methods, such as an electric eye. They use these sensors to measure changes in the position of the occupant during a crash, to determine the extent of the crash from that information, and to determine whether to deploy the airbag. Therefore, they use the occupant's movement as a collision sensor. No mention is made of any other features relating to detecting the occupant's movement from a predetermined position and monitoring the occupant, as disclosed in the patent application cited by reference. It is also strange that Matthes et al. Did not mention at all how to use active or passive infrared to determine the position of the occupant. As pointed out in the referenced patent application, direct measurement of occupant position based on passive infrared is probably not possible, and expensive GaAs
Active infrared, which requires modulation of an infrared laser, has also been very difficult and expensive until very recently.
Since none of these issues were mentioned, the method of use considered by Matthes et al. Was to indirectly determine the position of the occupant when passing through a plurality of longitudinally spaced sensors. It would be similar to the concept of an electric eye to measure.

[0008] The purpose of a sensor that detects when an occupant has moved from a predetermined position is that the impact between the head or chest and the deploying airbag can cause serious injury,
It consists in measuring the position of the occupant's head and / or chest relative to the airbag. Both White et al. And Mattes et al. Disclose only low mounting locations where the sensors are mounted in front of the occupant, such as on a dashboard or under a steering wheel. All such low mounting locations are prone to false detections, especially due to the position of the occupant's hands, arms and legs. In this case, at least three, and preferably more, of such sensors and detectors and appropriate logic circuits, for example, some of the sensors, such as when the driver's arm is the closest object to the two sensors. When the readings don't match each other,
You would need logic to ignore those readings.

Also disclose the use of an error correction circuit to distinguish between the speed of one occupant's hand and the speed of another occupant's hand when the occupant adjusts the radio knob. However, the circuit is not defined and is not shown. Three ultrasonic sensors of the type disclosed by White et al. Would be able to distinguish them in some cases. That is, if two of the sensors indicate that the occupant is not moving, and the third sensor indicates that the occupant is moving, it will be possible to distinguish. However, in such a combination,
If both hands and arms are in the path of the ultrasound transmitter and are in a position that shields most of the occupant's head or chest, the occupant cannot be distinguished. The size and operating position of the occupant can vary dramatically, so if the occupant moves to an extreme position and does not guarantee a good view of the occupant without being disturbed, a neural network can be used. A well-trained pattern recognition system is required.

[0010] Fujita et al. In US Pat.
U.S. Pat. No. 4,583 describes another method of measuring the position of an occupant, but in that method, if the occupant moves out of position, the measurement information is used to prevent the deployment of the airbag. Not used. In fact, according to the Fujita patent, the closer the occupant is to the airbag, the faster the airbag deploys, thereby increasing the likelihood of injury to the occupant. Fujita et al. Do not directly measure the occupant, but indirectly determine the position of the occupant from the measured position of the seat and the height of the occupant with respect to the seat. The height of the occupant is determined using an ultrasonic displacement sensor mounted just above the occupant.

[0011] As discussed above, the optical system described herein employs pre-impact sensing as described in US patent application Ser. Applicable to many other detection applications used inside and outside the cabin, such as intelligent headlight adjustment systems and blind spot (blind spot) monitoring devices.

2. Definition The use of pattern recognition is central to the invention and to the patent applications cited as references. Nowhere in the prior art does the use of training-based pattern recognition, as exemplified by the use of neural networks, be used to monitor the environment inside and outside the car.
The term "pattern recognition" as used herein refers to a system that processes signals emitted from or modulated by interaction with an object to determine to which class the object belongs. Means Such a system may determine whether an object is a member of a particular classification, or may apply the object to a particular classification in a classification group containing multiple classifications, or It is only determined that the category does not correspond to any of the categories. The processed signal is usually
An electrical signal output from a transducer that is responsive to sound or electromagnetic radiation and, if electromagnetic, can be any of visible light, infrared light, ultraviolet light, or radar. As used herein, a trainable or trained pattern recognition system refers to a pattern recognition system that has been taught various patterns by applying the system to various examples. The system that gives the best results is a neural network.

As used herein, the term "verify" means:
This means determining that an object belongs to a specific classification or classification group. The classification is, for example, a classification that includes all back-type child seats, depending on the purpose.
The classification includes all occupants, the classification including all occupants not sitting on the back-type child seat, and the like. If a particular person is to be recognized, the classification or its parent classification will include only one element, the person to be recognized.

The "occupancy" of the seat is a living thing, such as a human or a dog, or a non-living object, such as a box or bag of other living things such as plants or vegetables.

In the description herein relating to predictive sensing, the term "approaching" as used in the description of an object or vehicle approaching another refers to an object approaching a vehicle having a predictive sensing system relatively. Means movement. Thus, in the case of a side impact on a tree, the tree is considered to approach the side of the vehicle and collide with the vehicle. In other words,
This means that the commonly used coordinate system applies to the coordinate system on the target vehicle. The "target" vehicle is the vehicle on the side of the collision. By this arrangement, in all cases, i.e. (i) a moving vehicle collides with the side of a stationary vehicle, (ii) both vehicles are moving at the time of the collision, or (iii) stationary A general description can cover, for example, the case where the vehicle moves laterally with respect to the vehicle, a tree, or a wall.

3. 2. Prior Art of Pattern Recognition Ueno, JP-A-3-42337, discloses an apparatus for detecting a driving state of a vehicle driver. The apparatus includes a light transmitter for illuminating the driver's face and means for receiving an image of the driver and storing it for later analysis. Means are provided to determine the position of the driver's eye, then the pupil of the eye, and to determine whether the driver is looking aside or sleeping. Ueno does not determine the position of the eye relative to the rest of the cabin,
Measuring eye condition. Such a system will not work if the driver is wearing eyeglasses, especially sunglasses, or other optical devices that obstruct the field of view for measuring the driver. No pattern recognition techniques such as neural networks are used.

US Pat. No. 5,008,946 granted to Ando uses a complex set of rules to separate the driver's eyes and mouth, and uses this information, for example, in the driver's eyes. Used to control radios and other systems by moving the eyes and / or mouth. Ando uses natural light to irradiate only the driver's head. He also does not use a trained pattern recognition system, such as a neural network, or attempts to identify the contents of the vehicle or their relative position to the cabin. Rather,
Ando's patent is exclusively limited to controlling vehicle equipment that responds to the driver's mouth and eye movements.

US Pat. No. 5,298,732, issued to Chen, also discloses an optical filter positioned between a light source, such as the sun or the light of an oncoming vehicle, and the driver's eyes. The emphasis is on detecting the position of the driver's eyes. The chain does not provide details on how to detect the position of the eye, but provides a calibration system whereby the driver can adjust the filter to the correct position for the eye. I can do it. The Chen reference mentions the use of an automatic device for measuring the position of the eye, but does not describe how this device works. In any case, except for monitoring the position of the eyes, monitor the position of the occupant, measure the position of the eyes relative to the cabin, or identify other objects in the vehicle other than the driver's eyes I haven't mentioned anything at all. Again, this does not mention the use of a trainable pattern recognition system.

[0019] US Patent No. 5,305,012, granted to Faris, also describes a system for reducing glare in headlamps of both oncoming vehicles. Faris determines the position of the occupant's eyes by using two spaced infrared cameras that utilize passive infrared radiation from the driver's eyes. Faris is also only interested in the driver's eye position relative to the sun or oncoming headlights, and can identify or monitor the occupant, or determine the occupant's relative position to the cabin or airbag. Not detected. Faris also
It does not use trainable pattern recognition techniques such as neural networks. In fact, Farris did not even show how to detect the position of the occupant's eye, and he described "Robot Vision" by Berthold Horn, published by the MIT Press in Cambridge, Mass. (1991
References to a book entitled "Year)", and Faris did not use active infrared radiation (more generally, electromagnetic radiation) as in the present invention, Uses passive infrared radiation.

The use of neural networks as a pattern recognition technique is central to the present invention, which makes the surveillance system robust, reliable and practical. Because. The algorithms produced by the neural network programs typically require only a few lines of code written in the C computer language, while the aforementioned Ando, Chen and Farris techniques typically involve hundreds of lines. Also. As a result, the resulting system is low cost and easy to implement, making it practical for automotive applications. For example, CCD arrays, until very recently, were extremely expensive, and their use in VIMS was impractical. Similarly, implementing the techniques of the above referenced patents requires expensive microprocessors, while implementing neural networks and similar trainable pattern recognition techniques typically requires less than $ 5 inexpensive microprocessors. Processor is available.

The present invention uses advanced trainable pattern recognition capabilities such as neural networks. As discussed below, the data is typically pre-processed using various feature extractions. Examples of pattern recognition systems using neural networks for sonar signals are discussed in the following two documents: Gorman, RP and TJ.
Sejnowski, TJ “Analysis of Hidden Units in a Layere Trained to Classify Sonar Targets
d Network Trained to Classify Sonar Targets) ”,
Neural Networks, Volume 1, 75-89
1988, and "Classification of Sonar Targets Trained Using Massive Parallel Networks (Learned Clas
sification of Sonar Targets Using a Massively Para
llel Network) ”, IEEE Transaction
son Acoustics Speech, and
Signal Processing, Vol. 36, No. 7
No., July 1988. An example of a feature extraction technique is “Process and Apparatus for the Automatic Detection and Extraction of Features in Images and Displays.
Detection and Extraction of Features in Images and
No. 4,906,940, issued to Green et al., Entitled "Displays"). Other more efficient and efficient pattern recognition techniques include:
“Artificial Neuron and Meth
U.S. Pat. No. 5,33 granted to Wang, ST, entitled "Od of Using Same".
No. 90,136 and "Neural Network and Method of Using Same
U.S. patent application Ser. No. 08 / 076,601, entitled "
Seen in the issue. As another example, see US Pat.
35,339 (Morrison et al.), No. 5,2
14,744 (Schweizer et al.),
No. 5,181,254 (Schweizer
No. 4,881,270 (Knech)
t) et al.). All of the above references are incorporated herein by reference.

4. Optical Devices Optical devices can be used in several configurations to monitor a passenger compartment. One known method uses GaAs to illuminate an object, occupant or child seat momentarily.
Infrared laser beams are used in laser optical systems. This GaAs infrared laser beam is disclosed in U.S. patent application Ser.
978 in the manner described and illustrated in FIG. In order to receive the reflected light, the light receiving part (receiver) can be a charge-coupled device, that is, a CCD (a type of TV camera). The laser can also be used in scanning mode,
The use of lenses can also create conical light that covers a significant portion of the object. In these configurations, it is possible to precisely control the light to irradiate only a specific position of an object in the vehicle. In the scanning mode, the receiver may consist of only one or a few active elements, but in the case of conical light, an array of active elements is required. The laser system is disclosed in US patent application Ser.
As disclosed in US Pat. No. 78, it also has the additional important advantage of being able to determine the distance to the illuminated object.

In a simple manner, light produced by incoherent light emitting diodes is used to illuminate a desired area. In this case, a large CCD array is required because the area covered is not precisely controlled. Recently, however, the cost of CCD arrays has dropped significantly, and this arrangement has become the most economical system for monitoring the passenger compartment when distance measurements from the transmitter to the object are not required. If this distance is required, either a laser system, a stereographic system, a focusing system, or a hybrid ultrasound and optical system is required. Mechanical focus systems, such as those used in some camera systems, can measure the occupant's initial position, but are too slow to monitor the occupant's position during a collision. A distance measurement system based on a focusing system is disclosed in US Pat.
No. 93,124 (Subbarao), which can be used in conjunction with either a mechanical focusing system or two cameras, and which is sufficiently fast in the latter case It is. The Subaruo patent provides a great discussion of camera focus technology, and thus is incorporated herein by reference, but is a much more complex system than is necessary to practice the present invention. In fact, it is also possible to train a neural network to measure distances based on separate camera settings, i.e. two images taken with two adjacent CCDs and lenses with different characteristics. This is because, within the scope of the object of the present invention, the above technique is put to practical use by the camera disclosed in Subaru. Distance can also be measured by the system disclosed in U.S. Pat. No. 5,003,166 (Girod) by diffusing or scattering a structured light pattern projected onto an object.

In any of the above cases, regardless of the distance measurement system used, a trained pattern recognition system as defined above will not only identify and classify the illuminated object and its components, but also some Is used in the present invention to perform position detection.

[5] Optical and Acoustic Devices The laser systems described above are expensive, for example, if the distance from the airbag to the occupant needs to be measured, due to the need to modulate at a high frequency. Both laser and non-laser optical systems are generally suitable for measuring the position of an object in a two-dimensional plane, and a modulated laser system is capable of receiving each part of an image in a scanning mode. The distance from the vessel can be measured. With non-laser systems, it is also possible to measure the distance by a focus method as discussed above, or stereoscopically if two spaced receivers are used, in some cases. Could use only the position in view, for example, to estimate the position relative to the airbag. Finally, recently developed pulsed quantum well diode lasers provide an inexpensive distance measurement method, as discussed below.

Acoustic systems are also very effective at measuring distance, but because of the relatively slow speed of sound, simple electronic circuits need to be designed and the ability of the microprocessor is minimal. Because it is good. If a coordinate system is used where the z-axis is the distance from the transducer to the occupant, the sound is suitable for measurement in the z-direction, while a single C
A simple optical system using a CD would be suitable for measuring in the xy plane. Therefore, the combination of the acoustic system and the optical system makes it possible to perform all three-dimensional measurements with low-cost components.

An example of a system using such an idea is an optical system in which a passenger seat is irradiated with infrared light and a reflected light is displayed and a lens and a CCD array are combined, and the output of the CCD is digitized and analyzed. Combined with an analog-to-digital converter (ADC) that feeds into an artificial neural network (ANN) or other pattern recognition system. This system uses an ultrasonic transmitter and receiver to measure the distance to an object on the seat. The receiving converter supplies the data to the ADC and from there to the ANN. The same ANN can be used for both systems, so that all three-dimensional data to be analyzed by the ANN can be provided. A system using this low-cost component can perform accurate confirmation and distance measurement that cannot be achieved by a single system. If the acoustic portion of the system is supplemented with a phased array system as disclosed in the co-pending U.S. Patent Application (ATI-102), the optics may be used, for example, to determine the position of the driver's ear. Can be
And the phased array can direct a narrow beam to the occupant's ear and measure its distance.

6. Applications The applications of this technology are numerous, as described in the above-mentioned U.S. patent applications. For them: (i)
Monitoring the occupants for safety purposes to prevent injuries caused by airbag deployment, (ii) measuring the position of the occupants' eyes to automatically adjust the rearview mirror, (iii) night vision systems. Detecting the position of the seat for placing the eye in an appropriate position in order to eliminate parallax (parallax) of the head-up display, (iv)
Detecting the position of the occupant's ear to optimally adjust the entertainment system; (v) identifying the occupant for security reasons; (v
i) measurement of obstacles on closed paths of doors or windows, (vi
i) measuring the position of the occupant's shoulder so that the seat belt fastening position can be adjusted to provide the occupant with the best protection;
(Viii) Measuring the position of the back of the occupant so that the headrest can be adjusted in order to reduce whiplash injury during a rear-end collision, (ix) collision prediction detection, (x) blind spot (blind spot) detection, (xi) Intelligent headlight dimming, etc. In fact, more than 40 products have been identified based solely on the ability to identify and monitor objects and their parts in the interior of a car or truck cabin.

[0029]

SUMMARY OF THE INVENTION The present invention is directed to an occupant or a portion thereof in the interior of a vehicle such as an automobile or truck,
This is a system for confirming, detecting, and monitoring other objects in the vehicle interior and objects outside the vehicle. The system illuminates the contents of the vehicle and objects outside the vehicle with electromagnetic radiation, particularly infrared light, and uses one or more lenses to image the contents using one or more charge-coupled devices (CCDs).
Array). The output from the CCD array is
Classify, verify or locate content or external objects by appropriate computerized means employing a trained pattern recognition system. Typically, the information obtained from this verification and monitoring system is used to affect the operation of other systems in the vehicle.

The in-vehicle monitoring system of the present invention is installed in the interior of a vehicle equipped with an occupant protection device such as an inflatable airbag, and the vehicle is subjected to a considerable degree of collision, that is, the collision sensor uses the protection device. When exposed to a collision that determines that it should, the system according to the invention measures the position of the vehicle occupant relative to the airbag, and the occupant is likely to be injured by the deployment of the airbag. In such a case, it is impossible to deploy the airbag.

In some embodiments of the present invention,
One or more ultrasonic transmitters and receivers are added to make a distance measurement from the transmitter / receiver to the object. In some of these embodiments, a phased array system is used to cause the ultrasound from the ultrasound transmitter to be narrowly focused at a particular location on the object. In another embodiment, a modulated laser is used as an infrared light source to provide an accurate measurement of the distance to the point of reflection. In yet another embodiment, a focus system is used to measure the distance to the object.
Finally, in yet another embodiment, a GaAs pulsed quantum well laser system is used to directly measure the distance to the point of interest.

The basic objects of the optical sensing system according to the present invention are as follows. 1. It recognizes the presence of a person on a particular seat in a vehicle and uses this information to affect the operation of other systems in the vehicle, such as airbags, heating and air conditioning, or entertainment systems. 2. It recognizes the presence of a person on a particular seat in a vehicle and determines its position, and uses this position information to affect the operation of other systems in the vehicle. 3. The position, speed or size of the occupant in the vehicle is measured and this information is used to control the rate or amount of gas generated by the airbag inflation system. 4. The presence or location of the occupant in the rear seat of the vehicle is determined and this information is used to affect the operation of the rear seat protection airbag in the event of a head-on collision. 5. It recognizes the presence of a rear-facing child seat on a particular seat in a car and uses this information to affect the operation of other systems in the vehicle, such as airbag systems. 6. The approximate position of the driver's eye is measured and this information is used to control the position of the car's rearview mirror. 7. Monitors the position of the vehicle driver's head to determine if the driver is asleep or otherwise ill and unable to control the car, and uses this information to affect other systems in the vehicle Use to 8. A reliable and timely determination that the occupant is out of or about to move out of position and could be injured by the deployment of the airbag, and to suppress deployment of the airbag. An occupant position sensor that outputs a signal for restraining the occupant is provided. 9. Provided is a predictive sensor capable of accurately confirming an object which is likely to collide with a sensor arranged in a vehicle when snow and / or fog is present. 10. Detects the headlight of an oncoming vehicle or the taillight of a preceding vehicle,
An intelligent headlight dimming system is provided that distinguishes these lights from signal lights or reflections from the road surface and sends a signal to dimming the headlight. 11. Detects objects in the driver's blind spot
A blind spot detector is provided that classifies and alerts the driver, for example, when the driver is changing lanes, or continuously informs the driver of the condition of the blind spot occupancy.

These and other objects and advantages are:
The following description of a preferred embodiment of the vehicle identification and monitoring system of the present invention will become more apparent.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to the drawings, a portion of a cabin of an automobile is shown as 100 in FIGS. A driver 101 of a car is sitting on a seat 102 behind a handle 103 including an airbag device 104. Five transmitter and / or receiver assemblies 110, 111,
112, 113 and 114 are located at various locations in the cabin to determine the relative position of the head, chest and torso of the driving car with respect to the airbag, or to monitor the interior of the cabin. A control circuit 120 is coupled to the transmitters / receivers 110-114 to control the transmission from the transmitter and capture the return signal from the receiver. The control circuit 120 typically includes an analog-to-digital converter (ADC), a microprocessor including appropriate software such as sufficient memory and pattern recognition algorithms, and other suitable drivers;
Includes signal conditioners, signal generators, etc. In any embodiment, typically only one or two transmitters and receivers will be used, depending on the mounting location, as described below.

FIG. 1 shows a typical wave pattern of an ultrasonic wave from the transmitter / receiver 114. In this embodiment, transmitter / receiver 114 includes an ultrasonic transducer, which is typically used with an optical transmitter and CCD array as shown at 110, 112 and 113.
This optical system, the optical transmitter and the CCD array, locates occupants, objects and their features,
It is mapped as a two-dimensional image by the CD array, and the ultrasonic transmitter / receiver 114 measures the distance from the sensor to the occupant. When used to monitor a passenger seat, the optical system 110 determines that the seat is occupied, identifies what is occupied, and an ultrasonic system, such as 114, controls the occupant's access to the airbag. Determine the relative position. This optical system determines what the ultrasound system is measuring and distinguishes reflections from the occupant's head or chest from those from other objects. This transmitter / receiver 114 can be mounted on the rearview mirror 105.

In the case shown in FIG. 1, the ultrasonic wave emitted from the transmitter / receiver 114 bounces off the driver's head and chest and returns to the transmitter / receiver 114. This device has a control circuit 1
According to the 20 instructions, a burst of ultrasound, for example about 50 kilohertz, is transmitted periodically and the reflected signal is detected by the same or a different device. An algorithm in the associated electronics or control circuit 120 measures the time between transmission and reception of the ultrasound, thereby determining the distance from the transmitter / receiver to the driver, occupant or other occupant. Measure based on the speed of sound. This information is then sent to a crash sensor and diagnostic circuit, also located in circuit 120, where the occupant is too close to the airbag so that the injury caused by the deployment of the airbag is greater than the injury caused by the accident itself. Determine if it is not. In such a case, this circuit would render the airbag system inoperable, thereby preventing its deployment. In another case, a sensor algorithm calculates the likelihood of a collision requiring an airbag,
Wait until the probability exceeds a value that changes depending on the position of the occupant. In this way, for example, if the occupant is approaching the airbag and must make an immediate decision, this sensor will decide to deploy the airbag with a 50% likelihood assessment. For more distant occupants, wait until the probability rises to 95%. Although a system for the driver is shown, the front and rear occupant systems would be similar.

In another embodiment, a sensor algorithm may determine the rate of gas generation that affects the rate at which the airbag is inflated. In all these cases, the position of the occupant is used to influence the deployment of the airbag. That is, whether or not the airbag should be completely deployed, or the deployment timing or inflation speed is determined according to the position of the occupant.

The entire optical infrared transmitter / receiver assembly is shown at 112 in FIG. 2 and is mounted on an instrument panel facing the front window. The device 112 shown in enlarged form includes an infrared radiation source, or other form of electromagnetic radiation source, and a typically 160 pixel by 160 pixel charge coupled device array (CCD array). In this embodiment, a front window is used to reflect the illumination light provided by the infrared illumination as well as the light reflected by objects in the cabin, which is currently provided in some car models. It is similar in shape to a "heads-up" display. This “heads-up” display, of course, is currently only used to present information to the driver,
It is not used to reflect light from the driver to the receiver. Here again, unless one of the distance measurement systems described below is used, this system alone cannot measure the distance from the object to the sensor. Its main purpose is to identify and monitor objects.

Although the device 112 is actually about 2 cm in diameter, it is shown in FIG. Also, the reflective area on the front window is much smaller than shown, so that this area of the front window is flat and reflective, as is usually done when a head-up display is used. Special ingenuity has been applied so that

The system shown in FIG. 2 has a single CC
Due to the use of a D-array and the small size of this device, it cannot generally be used to create stereoscopic images, and therefore some other way to determine the distance to an object Is required. However, if two spaced CCD arrays are used, the distance to various objects in the cabin can be determined by a simple algorithm that looks for similar features on the two images and finds their relative positions. Can be. An alternative is to use a short focus lens. In this case, the lens is mechanically focused so that the clearest image can be measured, thereby determining the distance to the object. This is similar to the autofocus system for certain cameras, such as those manufactured by Fujifilm Corporation of Japan. of course,
Other methods described in the patents mentioned as references can also be used.

When an advanced pattern recognition system such as a neural network is adopted in the vehicle interior monitoring system,
The movement of the driver over time and the response to various stimuli can be monitored, and it can be determined whether the driving car has fallen asleep or otherwise has become inoperable. . In such a case,
You can adapt the car in various ways. One such system is shown in FIG. 1B, which comprises a converter 112 and a control circuit 120 incorporating a microprocessor.
The microprocessor is programmed to compare the driver's movement over time and is trained to recognize changes in behavior indicative of loss of service. If the system determines that the driver is likely to fall asleep, for example,
Turn on the warning light or send out a warning sound. If the driver does not respond to this warning, for example pressing the button 122, he activates a horn or light in such a way as to warn other vehicles,
In addition, the vehicle can be stopped. Of course, other responses can also be programmed, and other testing methods for assessing the driver's attention can be used without resorting to monitoring the driver's eye movements.

A more sophisticated system for monitoring driver behavior is described in "Eye Tracker Communication System" by Freidman et al.
U.S. Patent No. 4,648, entitled "Communication System".
No. 052, U.S. Pat. No. 4, entitled "Eye Position Sensor" by Heyner et al.
No. 720,189, U.S. Pat. No. 4,836,670 entitled "Eye Movement Detector" by Hutchinson, and "Eye Movement with Improved Calibration and Speed" by Hutchinson. Eye Movement Detector With Improve
d. Calibration and Speed), using various techniques described in U.S. Pat. No. 4,950,069, which track driver's eye movements, which were previously incorporated by reference. US Patent No. 5,008,94
No. 6 and US Pat. No. 5,305,012, all of which are incorporated herein by reference. The detection of a faulty driver can be best determined in particular by these techniques. These systems make use of advanced pattern recognition techniques, but in addition, often transmit signals so that the reflection of the cornea of the driver's eye can be detected, as discussed in the referenced patents. It is necessary to arrange the device and the CCD receiver at appropriate positions. The location may have to be a bit difficult to detect this corneal reflection due to the size of the CCD array used here, and sometimes to the reflection front window. It goes without saying that sunglasses hinder this approach.

The eye tracking system discussed above is facilitated by the present invention, which either locates the driver's eye directly using trained pattern recognition techniques, or uses the driver's head position. One of the main purposes of determining, either by inferring the position of the eye from the eye, is to be able to automatically position the seat so that the driver's eye is within the "eye oval" Because. The oval of the eye is an appropriate position for the driver's eyes for optimal driving of the car and for the position of the mirror. Thus, if it is possible to know where the driver's eyes are, it is possible to bring the driver's position so that the driver's eyes fall exactly within the oval of the eyes, and to observe the reflection of the eyes. It can be monitored using a small eye tracker (eye tracker). Further, by confirming the position of the driver's eye, the rearview mirror positioning device can be controlled so as to adjust the rearview mirror to an optimum position.

A more accurate acoustic system for making distance measurements to certain objects, or parts thereof, in a vehicle compartment is illustrated by the transducer 111 in FIG. In this case, three spaced-apart ultrasound transmitters / receivers are shown mounted on the A-pillar of the car. The A-pillar is the frontmost roof support pillar and also supports the front window. Due to wavelength, it is difficult to obtain a narrow beam using ultrasound unless a limited range of high frequencies or large transducers are used. For example, a commonly available 40 kHz transducer has a diameter of about 1 mm.
In cm, it emits sound waves that spread at an angle of about 60 degrees. To reduce this angle, the diameter of the transducer must be increased. An alternative solution uses several transducers to phase the emission so that the intended part of the target arrives at the same phase. This enhances reflections from selected portions of the target, while making reflections from adjacent sites interfere, so that the distance to the strongest portion within the intended range can be measured. By changing the phase of the transmission from the three converters 111,
The position of the reflection source on the curve can be specified. Identifying a reflector in space requires at least one additional transmitter / receiver, but not in a linear relationship with the others. The measurement accuracy can be determined by those skilled in the field of phased array radar by applying a relational expression. The waves coming out of the three transducers 111 shown in FIG. 3 are actually just a fraction of the waves that arrive at the desired point in space with the same phase. The effective propagation direction of these waves can be changed by changing the transmission phases of the three transmitters. Measurement of the approximate position of the point of interest on the occupant can be performed by a CCD array and appropriate analysis, and the phasing of the ultrasound transmitter is determined so that the distance to the required point can be measured.

FIG. 4 shows two optical systems with infrared radiation and a CCD array receiver, respectively. The price of CCD arrays has dropped dramatically in recent years, making them practical for in-car surveillance. Transducers 110 and 113 are CCD arrays of 160 × 160 pixels, which are almost covered by a spherical lens. This creates a “fisheye” effect, which allows light from different angles to be captured. A sensor, such as a ceiling light, indicated at 113 located at the center of the roof of the car can monitor the entire interior of the car with sufficient resolution, for example, to determine the state of occupancy in the car. CCDs such as those used here are in Culver City, California
er City, CA)
all Electronics Inc.). A fisheye lens is a "wide-angle photographic lens that covers an angle of about 180 degrees, and produces a circular image in which the depth of the center is excessively reduced and the distortion increases as the distance to the periphery increases." (Houghton Mifflin Mifflin Compa
ny), published by The American Heritage Dictionary of the En
glish Language, 3rd edition, 1992). The distortion of the fisheye lens can be significantly changed by changing the shape of the lens, so that certain parts of the cabin can be observed. Also, in many cases, a full 180 degree range is not required, and lenses that capture smaller angles will be used. Although a spherical lens is shown primarily herein, a particular lens design will be used depending on the location in the vehicle and the purpose of the particular receiver.

CCD arrays are commonly used in television cameras, for example, to convert video into electrical signals. CCDs used for the purposes of the present invention include:
All devices capable of converting optical frequencies, including infrared and ultraviolet light, into electrical signals are included. Certain CCD arrays used for many of the applications disclosed herein are fabricated on a single chip that is 2 cm or less in diameter. The data from the CCD array is digitized and an electronic circuit 1 including a microprocessor for analyzing the digitized data.
Sent to 20 serially. To reduce the amount of data that needs to be stored, the first processing of the video data is CC
This is performed while receiving from the D array.

One method of directly measuring the distance to an object without resorting to a distance finder (range finder) is to use a mechanical focusing system. However, the use of such devices is cumbersome, expensive, slow, and unreliable. An alternative is to use the focusing system described in US Pat. Nos. 5,193,124 and 5,003,166, which are cited, but such systems are expensive. Requires hardware and / or complex algorithms. Another alternative is shown in FIG. 5, where the transducer 116 is an infrared light source having a wide emission angle to illuminate all objects present in the front driver's seat. Receiving CCD converters 117 and 118 are provided at a distance so that stereoscopic image analysis can be performed by the control circuit 120. The circuit 120 includes a microprocessor including an appropriate pattern recognition algorithm, as well as other circuits described above. Including. In this case, the desired feature for detecting the position is first selected from one of the two images sent back from the CCD converters 117 and 118. And the software measures the position of the same feature on another image,
The distance of the feature from the transducer is determined by analysis known to those skilled in the art.

Transducers 116-118 are shown mounted on the A-pillars of the car. Since these transducers are typically very small, about 2 cm or less in diameter, they can be mounted on the front window itself or other convenient location that provides a clear view of the cabin part to be monitored. It is possible.

In particular, a new type of laser distance meter can be applied to the present invention. This product is powered by Power Spectra Inc. of Sunnyvale, California
Is a GaAs pulsed laser device, manufactured by the company, which can measure up to 30 m with an accuracy of <2 cm and a resolution of <1 cm. This system can be used in combination with transducer 116, thereby eliminating one of the receiving transducers 117 or 118. After the location of a particular feature of the vehicle occupant has been detected, the device is used with an appropriate projection mechanism that directs a laser beam at that particular feature. As a result, the distance to the characteristic portion can be adjusted within 2 cm, and more accurate measurement can be performed by performing calibration. This device has particular application not only for in-car measurements, but also for predictive sensing and blind spot monitoring applications outside the vehicle.

FIG. 6 is a partially cutaway side view of the cabin, showing a preferred mounting position of the optical in-vehicle monitoring sensors 110, 113, 210-214. Each of these devices is shown to have a fisheye lens and is shown enlarged for clarity. A typical practical device has a lens diameter of about 2 cm and protruding about 1 cm from the mounting surface. Due to their small size, these devices can be made almost invisible to the occupants of the vehicle. It should be noted that since these devices are optical, it is important to keep the lens surface relatively clean. The control circuit 120 has a self-diagnostic characteristic and compares the image returned from one of the transducers with the stored image to confirm the presence of key features. If the receiver fails to confirm the presence of the feature, a warning is displayed to the driver indicating that the lens surface needs to be cleaned. The technique shown in FIG. 6 can be used for many purposes, in addition to the monitoring functions described herein, such as:
(I) Confirmation of the existence of the back type child seat 230, (i)
i) monitoring of the back of the occupant 242, (iii) occupant 240
(Iv) confirmation of the position of the knee 241 of the occupant;
And (v) occupant height measurement using transducer 113.

The occupant position sensor, whatever its shape, is incorporated into the circuit of the airbag system shown schematically in FIG. In this embodiment, the occupant position sensor is used as an input to an intelligent electronic sensor and diagnostic system. The electronic sensor determines whether to deploy the airbag based on the vehicle's accelerating collision pulse or a collision sensor mounted on the collision area, and the occupant position sensor determines whether the occupant is too close to the airbag, Therefore, it is determined whether or not the airbag should be deployed. In FIG. 7, an electronic crash sensor in the sensor and diagnostic unit determines whether the crash is severe enough to require deployment of the airbag. The occupant position sensor measures the position of the vehicle occupant relative to the airbag and sends this information to a sensor and diagnostic unit, which determines whether it is safe to deploy the airbag. The ready sensor also checks to see if a vehicle collision has occurred there. If both the sensor / diagnosis unit and the arming sensor ready for operation are encountering a collision requiring an airbag and the position sensor determines that the occupant has left the airbag in a safe position, the airbag or The inflation restraint system is deployed.

FIG. 8 shows a detailed example of the occupant position sensor using infrared rays in the range of 0 to 2 m (corresponding to the occupant position of 0 to 1 m since the signal has to reciprocate with the occupant).
Is shown in the block circuit diagram of FIG. The operation is as follows. 48 MHz signal f by the crystal oscillator 401
1 is made and sent to the frequency tripler 402 where 14
A 4 MHz output signal is created. This 144MHz
The signal is then sent to infrared diode driver 403, which causes infrared diode 404 to operate at 144 MHz.
Is driven to emit infrared light modulated to the reference phase angle of 0 degree. Infrared diode 404 is aimed at the occupant of the car. Slightly higher frequency than signal f1 (48.0
The second signal f2 of 5 MHz) is sent to the frequency tripler 406 in the same manner as described above, and a frequency of 144.15 MHz is created. This signal is then fed to mixer 407, where 144 MHz from frequency tripler 402
Mixed with the signal. The mixed signal output from the mixer 407 is sent to the filter 408, where the difference between the frequency of three times f1 and the frequency of three times f2, that is, all signals other than the beat frequency of 150 kHz are removed. You. The infrared signal reflected from the occupant is received by the receiver 409 and sent to the preamplifier 411. Receiver 40
9 and the amplifier 411 are connected to a bias resistor 41.
0 is connected. This signal has the same modulation frequency, 144 MHz, as the outgoing signal, but is out of phase by an angle x with respect to the outgoing signal as the signal travels from the transmitter back to the occupant and to the receiver. The output from the preamplifier 411 is sent to the second mixer 412 together with the 144.15 MHz signal from the frequency tripler 406. Mixer 412
Are then amplified by automatic gain control amplifier 413 and fed to filter 414. Filter 414
Removes the difference frequency of 150 kHz, that is, all frequencies other than the beat frequency, similarly to the filter 408.
However, the frequency thus obtained is 150k
The Hz signal has a phase angle x with respect to the signal from filter 408. Both 150 kHz signals are sent to a phase detector 415, where the magnitude of the phase angle x is measured. The distance from the transmitting diode to the occupant can be expressed by the formula x / 345.6 using the above values.
Where x is the measured angle and the distance is meters.

The application described herein has been described by taking as an example the case of detecting the driver of a car. Of course, the same system for measuring the occupant's relative position with respect to the airbag is applicable to passengers in the passenger or rear seats, although some modifications may be required. The trigger timing of the sensor depending on the position of the occupant may be different between the driver and other occupants. Current systems are basically driver-based, which can result in airbags being deployed too late or injuries to occupants due to the lack of airbag deployment. Get higher. This is the case when the driver is not in a position where deployment of the airbag is required but another occupant is in a position where it is required. By using occupant position sensors for both the occupant and the driver, the airbag system can be optimized for each individual occupant, resulting in a significant reduction in injury. Especially,
If either the driver or the occupant is out of position, only the driver or the occupant system may not be deployed.

In an accident requiring an airbag, there is almost always a driver in the car. However, only about 30% of these vehicles carry passengers. If there is no passenger, there is usually no need to deploy the passenger's airbag. When the occupant monitoring system is used for a fellow passenger in combination with an appropriate pattern recognition circuit, it is possible to reliably know whether or not the seat is occupied. When there is no passenger, the deployment of the airbag on the passenger side can be disabled, thereby reducing the cost of replacing the airbag. This same strategy can be applied to monitoring the rear seats of a car. A trainable pattern recognition system, as used herein, can also distinguish between occupants and vegetable bags, for example. Finally, much has been discussed about children who are standing up by a brake operation or the like before a collision or have moved to a position close to the airbag and are out of a predetermined position. It should be understood that the occupant position sensor described herein can prevent the deployment of the airbag in such a state, as well as in the state of the back-type child seat as described above.

The use of trainable pattern recognition techniques, such as neural networks, is an important part of the present invention. These techniques are implemented to determine the differences between different types of objects using sophisticated computer programs that analyze example patterns. These computer programs are trained using representative data collected during the training period, data called a training set. After training, the computer program outputs computer algorithms, including rules that allow classification of objects based on data obtained after installation in the vehicle. This algorithmic rule is built into the system mounted on the car. Formulation of such rules is central to the pattern recognition technique used in the present invention. To date, artificial neural networks have been the most successful rule decision method. However, new systems are now being developed that incorporate many of the benefits of neural networks, such as learning by training, and are free from imperfections such as the inability to understand the network and the possibility of converging to the best conclusions.

In some embodiments of the present invention, such as to determine the presence of an obstacle in the path of a closing window, as described below, a trained researcher may return. The rules are simple enough so that you can devise algorithms that look at the incoming optical or acoustic signal and make the necessary decisions. In other cases, for example, when determining the presence of a rear-type child seat or an occupant, an artificial neural network is used to determine rules. One such set of neural network software for determining pattern recognition rules is NeuralWare Corporati, Pittsburgh, PA.
on). Another network pattern recognition technique is disclosed in the patents cited by Motorola. Numerous documents, including more than 500 U.S. patents, detail neural networks, so the theory and application of this technology is well known and will not be repeated here. Except for a few special cases where neural networks have been used to solve special problems, neural networks have not been applied to cars and trucks to date.

The system used in the present invention to determine the presence of a backside child seat, occupant, or vacancy is an artificial neural network. In this case, the network is activated by signals returning from the CCD array when detected, for example, by the converters 521 and 522 of FIG. For example, in the case of a front seat, throughout the training period the system is taught to distinguish between three cases. This training includes a child seat in the front seat of the car,
Performed by a number of experiments at various positions and orientations. Similarly, a sufficient number of experiments are repeated with occupants and vegetable boxes, bags, and other objects. By the time the neural network is fully trained, such experiments have been performed about a thousand times, so that a distinction can be made in three cases and that the correct decision can be output with very high accuracy. Become.

Once the network is determined, the results can be examined using, for example, tools supplied by NeuralWare, and the final rule is determined as a result of the trial and error. . If so, those rules can then be programmed into the microprocessor. Alternatively, a neural computer can be used to create the net directly. In either case, they can be implemented by those skilled in the art of pattern recognition technology using neural networks. If a microprocessor is used, a memory device for storing data from the analog-to-digital converter that digitizes data from the receiving converter is also required. On the other hand, if a neural network computer is used, the analog signals can be fed directly from the converter to the input nodes of the neural network, and no intermediate memory is required. In the case of microprocessor systems, some memory is required to store computer programs, and if a neural computer is used for more than one task, the singular value of the network associated with each task is determined. Memory is required to store.

There are several ways to measure the height of the driver and automatically adjust the seat or adjust the seat belt fixing point. Several alternatives are shown in the side plan view of FIG. 9, where two height-measuring sensors are mounted, i.
21 and another sensor 520 mounted on the A-pillar. These transducers have already been used in the other vehicle interior identification and monitoring system embodiments described herein.

In the patent application cited as a reference, ultrasound was the main technique for measuring occupant height. This technique usually requires at least two transducers, for example, if transducer 521 is used alone, this transducer will have a distance to the head regardless of which direction the head is facing. The exact position of the head is unclear. Knowing the distance from the head to the transducer 520 significantly reduces this ambiguity.

Optical converters using CCD arrays are now intensely price competitive and will eventually become the technology of choice for vehicle interior monitoring as described above. For example 160x1
The combination of a single 60 pixel CCD array with appropriate trained pattern recognition software can be used to create an image of the occupant's head and to accurately detect head position for the purposes of the present invention.

The occipital region detector 534 is also shown in FIG. This detector measures the distance from the headrest to the rearmost position of the occupant's head, and also ensures that the headrest is properly positioned behind the occupant's head to provide optimal support in the event of a rear impact. Used to control position. While most vehicles have adjustable headrests, occupants are rarely, if not completely, in the wrong position. Each year, there are more than 400,000 whip injuries due to car collisions, of which about 90,000 are due to rear collisions (Source: National Highway Traffic Safet)
y Administration (NHTSA)). A properly positioned headrest can significantly reduce the frequency of such injuries, but this could be achieved with the head detection of the present invention. A head detector 534 is shown schematically connected to a headrest control mechanism and circuit 540. This mechanism allows the headrest to be turned up and down, and in some cases back and forth. An occupant position sensor for side impact used with a door mounted airbag system is shown at 530 in FIG.

[0063] The seat belt is most effective when the vertical position of the upper attachment point to the vehicle is close to the occupant's shoulder. If the attachment point is too low, the belt will rub the shoulders, causing occupant discomfort. If it is too high, the belt will rub the neck and cause discomfort to the occupant, and will move significantly forward in the event of a collision, which will result in the occupant's head being hit against the steering wheel. For these reasons, it is desirable that the upper attachment point of the seat belt be slightly above the occupant's shoulder. To achieve this for occupants of various sizes, the position of the occupant's shoulder must be known, which can be achieved by the cabin monitoring system described herein.

Such a system is shown in FIG. 10, which is a side plan view of the seat belt securing and adjusting system. In this system, the infrared transmitter / C
A CD array receiver 620 is conveniently located on the headliner located above and outside the occupant's shoulders. A suitable pattern recognition system as described above is then used to measure shoulder position. This information is sent to the seat belt fixed height adjustment system 632, shown schematically, which moves the attachment point 631 to an optimal vertical position to position the seat bell and 630 in place.

FIG. 11 is a perspective view of a vehicle 710, which is about to be collided with the side by an adjacent vehicle 720, as viewed obliquely from above. The vehicle 710 is provided with a predictive sensor system, and The transmission of the infrared wave toward the car 720 is shown. The transmitter 730 is connected to the electronic module 740. Module 7
40 is a circuit 742 for driving the transmitter 730 and the receiver 7
34 and includes circuitry 744 for processing the signals returning from 736. Circuit 744 includes a neural computer 745, which includes receivers 734 and 734.
A pattern recognition decision is made on the basis of the signal from. Receivers 734 and 736 are mounted on the car's B-pillar and are covered with a protective clear cover. Another mounting location is shown as 738, which is the trim panel of the door window, where the rearview mirror (not shown) is often mounted. Another advantage of this system is that infrared light can pass through the fog, making the technology particularly suitable for predictive sensing applications.

In addition, the system detects an object in a blind spot of a car and displays the image of the object so that the driver can see it when, for example, the driver tries to change the driving lane. Or to activate a warning system. In this case, the car 71
In order to catch a car trying to pass through the side of 0, the mounting position must be selected so that good visibility can be provided along the side of the car. The locations 734, 736 and 730 can each provide sufficient visibility for this application, but the space in the vicinity of the car is not imaged. Alternative locations include mounting the unit on the outer rearview mirror or on the rear window or C-pillar. However, when mounted on a mirror, the disadvantage is that the device is covered with ice, snow or dust.

In both the case of the predictive sensor and the blind spot detector, the infrared transmitter and the CCD array system provide image information primarily for recognizing objects near the vehicle 710. To complete this process, distance information as well as velocity information is also required, which is usually obtained by differentiating position data. This can be achieved by any of the methods described above as well as radar systems. Although radar systems are not suitable for use inside vehicles, they are commonly used for sensing applications outside vehicles and are a well-known example of police radar. Currently, small radar systems are available that are inexpensive and fit into limited space. Another advantage of radar in this application is that
It is easy to obtain a transmitter with the desired divergence angle, so that the device does not have to be aimed at the target. In these cases, the best embodiment of the invention is to use radar, and the sub-optimal method is to use a pulsed GaAs laser system with a CCD array. However, using two CCD arrays or acoustic systems is also a good choice. Both acoustic systems and stereo systems using two CCD arrays have the disadvantage that they are slower than GaAs devices, and also have additional acoustic systems outside the vehicle, where deposits can accumulate and be affected on the working surface. That is something that must be done.

In a preferred embodiment, transmitter 730
Is an infrared transmitter and receivers 734, 736 and 7
Reference numeral 38 denotes a CCD converter which receives reflected infrared waves from the car 720. In the implementation shown in FIG. 11, an external airbag 790 is shown, which is described in U.S. patent application Ser.
Deploy when a side impact is about to occur, as described in the issue.

FIG. 12 shows a headlight or a vehicle 81 of an oncoming vehicle.
An external monitoring system used to detect the taillight of a car in front of zero is depicted. In this embodiment,
The CCD array is sensitive to visible light, and no separate illumination source is used. Again, the key to this technique lies in the use of trained pattern recognition algorithms and, in particular, artificial neural networks. As in the other cases mentioned above and in the pending U.S. patent application cited above, the pattern recognition system is adapted to recognize the pattern of the headlights of an oncoming vehicle or the pattern of the taillights of the preceding vehicle of car 810. The headlight is dimmed when trained and any of these conditions are sensed. It is also trained to not dimming advertising billboards or other reflected light from the road. One problem is to distinguish taillights that require dimming from distant headlights that do not require dimming. Three techniques are used for this: (i) measuring the spacing of the light sources, (ii) determining the relative positions of the light sources with respect to the car, and (iii) filtering and filtering the light sources. Use a red filter to compare the brightness without passing through. In the case of the taillight, the brightness through the red filter and the brightness without the filter are almost the same, whereas in the case of the headlight, there is a significant difference. In this case, two CCD arrays are used, one with a filter, or a filter that can be removed electrically or mechanically using something like liquid crystal.

The foregoing has shown and described an optical system that uses one or more CCD arrays and other related devices that meet the objects and advantages of the present invention as a monitoring system for monitoring the interior and exterior of a vehicle. However, many modifications, variations and variations of the present invention, as well as other uses and uses, will become apparent to those skilled in the art from a review of this specification and the accompanying drawings, which disclose the preferred embodiments described above. Will. All changes, modifications and variations that do not depart from the spirit and scope of the invention, as well as methods of use and uses, are intended to be covered by the invention as defined in the appended claims.

[0071]

The present invention has the following effects. 1. Recognizing the presence of a person on a particular seat in a car, this information can be used to affect the operation of other systems in the vehicle, such as airbags, heating and air conditioning, or entertainment systems. 2. It recognizes the presence of a person on a particular seat in a vehicle and determines its position, and this position information can be used to affect the operation of other systems in the vehicle. 3. The position, speed or size of the occupant in the vehicle can be measured and this information can be used to control the rate or amount of gas generated by the airbag inflation system. 4. This information can be used to determine the presence or location of the occupant in the backseat of the vehicle and to affect the operation of the backseat protection airbag in the event of a head-on collision. 5. Recognizing the presence of a rear-facing child seat on a particular seat in an automobile, this information can be used to affect the operation of other systems in the vehicle, such as airbag systems. 6. The approximate position of the driver's eye can be measured and this information can be used to control the position of the car's rearview mirror. 7. Monitors the position of the vehicle driver's head to determine if the driver is asleep or otherwise ill and unable to control the car, and uses this information to affect other systems in the vehicle Can be used for 8. A reliable and timely determination that the occupant is out of or about to move out of position and could be injured by the deployment of the airbag, and to suppress deployment of the airbag. An occupant position sensor that outputs a signal for suppressing the occupant position can be provided. 9. When snow and / or fog are present, it is possible to provide a predictive sensor capable of accurately confirming an object that is likely to collide with a sensor disposed in a vehicle. 10. Detects the headlight of an oncoming vehicle or the taillight of a preceding vehicle,
An intelligent headlight dimming system can be provided which distinguishes these lights from signal lights or reflections from the road surface and sends out a signal for dimming the headlights. 11. Detects objects in the driver's blind spot
A blind spot detector can be provided that categorizes, for example, alerts the driver when the driver attempts to change lanes, or continuously informs the driver of the condition of the blind spot occupancy.

[Brief description of the drawings]

FIG. 1 is a cutaway side plan view of a portion of a passenger compartment, showing a preferred mounting position of a monitoring sensor, in particular a preferred position for detecting an automobile driver, and a view from an ultrasonic position sensor mounted on a mirror; It shows a wave pattern.

FIG. 2 is a view similar to FIG. 1, showing a wave pattern from an optical system using an infrared light source, and a CCD array receiver using a front window as a reflective surface; Figure 3 schematically illustrates the interface between a careless light or buzzer mounted on the instrument panel and a reset button.

FIG. 3 is a view similar to FIG. 1 showing the wave pattern from a set of ultrasound transmitters / receivers, where the accurate spacing of the ultrasound beam by the spaced transducers and the phase of the signal; Focus and thus a specific point on the driver's surface can be measured accurately.

FIG. 4 is a view similar to FIG. 1, but showing an optical system wave pattern using an infrared light source and a CCD array receiver using a front window as a reflective surface;
In this case, the CCD array receiver is covered with a fisheye lens, and can capture the contents of the cabin in a wide-angle view.

FIG. 5 is a view similar to FIG. 1, showing the wave pattern from a pair of small CCD array receivers and one infrared transmitter, in which case spaced CCD arrays to the occupant features; The distance can be measured accurately.

FIG. 6 is a partially cutaway side view of the cabin, showing a preferred mounting position of the optical in-vehicle monitoring sensor.

FIG. 7 schematically shows an in-vehicle monitoring sensor used as an occupant position sensor together with the rest of the deployment suppression system.

FIG. 8 is a schematic diagram showing a circuit of an occupant position sensing device using a modulated infrared signal, beat frequency and phase sensing system.

FIG. 9 is a partially cut-away side plan view of the vehicle, showing a cabin of the vehicle including the driver, a preferable mounting position of an occupant position sensor for side collision, and reducing whiplash injury due to rear collision. Figure 2 shows an occupant back head position detector for use with a headrest adjustment system.

FIG. 10 is a partially cut-away side plan view showing the interior of an automobile with two occupant height measurement sensors, one sensor attached to a headliner above the occupant, and the other sensor A Mounted on pillars. Also shown is a seat belt attached to the seat, the seat belt having an adjustable upper fastening point that is automatically adjusted according to the occupant's height.

FIG. 11 is a perspective view of a vehicle that is about to collide with a side surface of another vehicle, showing the positions of various components of the predictive sensing system of the present invention.

FIG. 12 is a partially cut-away side plan view of a cabin showing a sensor for sensing oncoming headlights and / or taillights of a preceding vehicle used with an automatic headlight dimming system. .

[Explanation of symbols]

110, 112, 113, 114 Transmitter / Receiver 116 Infrared Light Source 117, 118 CCD Converter 120 Control Circuit 520, 521 Sensor 530 Occupant Position Sensor 620 CCD Array Receiver 632 Seat Belt Fixed Point Adjustment System 730 Transmitter 734 Receiver 745 Neural computer

 ──────────────────────────────────────────────────の Continuing on the front page (72) Inventor Wendell Sea Johnson 90165 Torrance Fujita Street, California, USA 3165

Claims (36)

[Claims] 1. An internal monitoring system in a motor vehicle having a passenger compartment that includes a passive occupant protection system and at least one occupant having a surface, wherein: a) said at least one occupancy may be present. Irradiating means for irradiating a portion of the cabin with electromagnetic radiation; b) receiver means including a CCD array for receiving electromagnetic radiation reflected from a surface of the at least one occupant in the cabin; c) Processing means coupled to said receiver means for processing said received radiation and generating based thereon an electronic signal representative of said surface of said at least one occupant in said cabin; d) said processing means Classification means coupled to and comprising a trained pattern recognition algorithm for classifying the electronic signal to identify the at least one occupant; ) The classification means coupled to a said at least one influencing output means to another system of the vehicle in response to the confirmation of the occupiers system. 2. The system of claim 1, wherein said passive protection means comprises an inflatable airbag, said at least one occupant being an occupant, and wherein said electronic signal is transmitted from said surface. A system for indicating a position of an occupant of a vehicle to be determined, the system including means for modifying a timing at which the output means starts inflation of the inflatable airbag according to the position of the occupant. 3. The system of claim 1, wherein the passive protection means comprises an inflatable airbag, and wherein the at least one occupant is an occupant, and wherein the electronic signal is determined from the surface. A system for indicating the position of an occupant of the vehicle to be operated, the output means including means for modifying the rate at which the inflatable airbag is inflated in response to the position of the occupant. 4. The passive protection means comprises an inflatable airbag, and said at least one occupant is an occupant, and wherein the electronic signal is determined from the surface by a vehicle occupant. And wherein the output means includes means for inhibiting deployment of the inflatable airbag in the absence of the occupant. 5. The system of claim 1, wherein the at least one occupant is an occupant's head, and wherein the electronic signal is indicative of a position of the occupant's head determined from the surface. The trained pattern recognition means includes means for classifying occupant head movement over time, and the responsive means coupled to the pattern recognition means is responsive to the classification of occupant head movement over time. System that affects another system of the vehicle. 6. The system of claim 5, wherein said another system includes an alarm. 7. The system of claim 5, wherein said another system includes limiting means for limiting the speed of said vehicle. 8. The system of claim 1, wherein said trained pattern recognition means comprises a neural network. 9. The system of claim 1, wherein the car further includes a seat, and wherein the at least one occupant is a rear-facing child seat located in the seat of the car. A system in which another system in the vehicle includes an airbag. 10. A system for monitoring at least one object outside a vehicle in a vehicle having an interior and an exterior, comprising: a) emission means for emitting an electromagnetic wave to irradiate an external object; and b) CCD. Receiving means including an array for receiving reflected electromagnetic radiation from the external object; c) coupled to the receiving means for processing the received radiation and generating an electronic signal representative of the external object based thereon. E) a classifying means coupled to said computing means and including a pattern recognition algorithm for classifying said electronic signal; and e) a classifying means coupled to said classifying means for producing said at least one occupant. Output means for affecting another system of the vehicle in response to said confirmation. 11. The system according to claim 10, further comprising measuring means for measuring a distance from said external object to said vehicle, said measuring means including radar. 12. The system according to claim 10, further comprising measuring means for measuring a distance from said external object to said vehicle, said measuring means comprising a pulsed laser. 13. The system according to claim 10, wherein said another system is arranged such that said external object is within a specific range for said vehicle and said driver of said vehicle is in contact with said external object. A system that is a warning system that is activated when an action is taken to increase the likelihood of a collision. 14. The system according to claim 10, wherein the another system is arranged so that the external object enters a specific range with respect to the vehicle, and the arithmetic processing unit performs a communication with the external object. A system that is a side impact airbag system that is activated when it determines that there is a high potential for collision. 15. The system according to claim 10, wherein said arithmetic processing means includes a neural network algorithm. 16. A headlight dimming system in a vehicle having an interior and an exterior, comprising: a) receiving means including a CCD array for receiving reflected electromagnetic radiation from an external object; and b) receiving. Processing means coupled to the means for processing the received radiation to produce an electronic signal representative of the received radiation; c) coupled to the processing means and comprising a trained pattern recognition algorithm, the electronic signal comprising: And d) output means coupled to the classification means for dimming the headlights of the vehicle according to the classification. Including headlight dimming system. 17. The system according to claim 16, further comprising radiation from taillights of a preceding vehicle in said section. 18. The system according to claim 16, wherein the trained pattern recognition algorithm comprises a neural network. 19. An internal monitoring system in a motor vehicle having a passenger occupant including a passive occupant protection system and at least one occupant having a surface, wherein: a) said at least one occupancy may be present. Irradiating means for irradiating a portion of the cabin with electromagnetic radiation; b) receiver means including a CCD array for receiving electromagnetic radiation reflected from a surface of the at least one occupant in the cabin; c) Processing means coupled to said receiver means for processing said received radiation and for generating electronic signals representative thereof of said surface of said at least one occupant in said cabin; d) from said CCD array Means for measuring a distance to said at least one occupant; e) a trained pattern recognition algorithm coupled to said processing means. Look, said at least 1
Classification means for classifying said electronic signal to identify two occupants; f) coupled to said classification means and said distance measurement means;
Output means for affecting another system of the vehicle in response to the confirmation of the at least one occupant. 20. The system according to claim 19, wherein said position measuring means comprises at least two charge coupled devices. 21. The system according to claim 19, wherein said another system is an inflatable passive protector, and wherein said analyzing means disables deployment of said inflatable passive protector. A system that includes circuits for: 22. The system of claim 19, further comprising means for utilizing waves reflected from a front window of the vehicle. 23. An internal monitoring system in a motor vehicle having a passenger occupant protection system and a cabin including at least one occupant having a surface, the system comprising: a) emitting an infrared wave toward the at least one occupancy. B) modulating means for modulating said oscillating wave with a first signal of a first frequency; c) converting a second signal having a second frequency slightly different from said first signal. Generating means for generating; d) receiving infrared waves emitted from said at least one transmitter and reflected from said at least one occupant;
One infrared receiver for outputting a third signal having the same frequency as the first signal and having a different phase; e) first mixing means for mixing the first signal with the second signal to generate a first beat signal; F) second mixing means for mixing the third signal with the second signal to form a second beat signal; and g) transmitting the phase difference between the first beat signal and the second beat signal to the at least one signal. Detecting means as a measure of the distance from a vessel to the at least one occupant. 24. A system for measuring the position of said at least one occupant with respect to said cabin in a motor vehicle having a cabin including at least one occupancy having a surface, said system comprising: Irradiating means for irradiating with electromagnetic radiation; b) at least two spaced receiver means, each comprising a CCD array, for receiving electromagnetic radiation reflected from said surface in said cabin; c) said at least two receiving means. Processing means coupled to the at least two receiver means for processing the received radiation received from each of the at least two receiver means to produce an electronic signal representative of the contents of the vehicle interior; d) the at least two receivers Means for comparing the signals received from the devices to determine the position of the at least one occupant relative to the at least two receivers; e) coupled to the comparing means. Is, the system including at least one influencing output means to another system of the vehicle according to the position of the occupiers. 25. An automobile having a cabin including at least one occupant having a surface, wherein the at least one occupant has a surface.
A system for measuring the position of two occupants, comprising: a) irradiating means for irradiating a part of the cabin with electromagnetic radiation; b) electromagnetic irradiation comprising a CCD array and reflected from the surface in the cabin. C) means for combining with the CCD array to obtain distance information from the CCD array to the at least one surface; and d) coupled to the receiver means and from the receiver means. Processing means for processing the received radiation to produce an electronic signal representative of the distance from the CCD array to the at least one surface; e) differentiating the vehicle according to the position of the at least one occupant. Output means for influencing the system. 26. The system according to claim 25, wherein the CCD array further comprises a fisheye lens. 27. The system of claim 25, wherein said means for obtaining distance information further comprises a focusable lens. 28. The system of claim 25, wherein said means for obtaining distance information further comprises a pulsed laser. 29. The system of claim 25, wherein the means for obtaining distance information further comprises structured light. 30. The system according to claim 25, wherein said means for obtaining distance information further comprises an ultrasonic transmitter and a receiver. 31. The system according to claim 30, wherein said ultrasonic transmitter means comprises a phased array. 32. The system according to claim 25, wherein said arithmetic processing means further includes pattern recognition means. 33. The system of claim 25, wherein said at least one occupant is a human part selected from the group consisting of an eye, an ear, a head, and a chest. 34. The system according to claim 32, wherein the vehicle has a driver and a rearview mirror, and wherein the another vehicle system positions the rearview mirror with respect to the driver's eye. Including system. 35. The system according to claim 25, wherein the occupant is a human, the vehicle has at least one airbag, and the another vehicle system is a vehicle for the at least one airbag. A system that includes means for controlling inflation. 36. An occupant position sensor in a motor vehicle having an occupant and a passive occupant protection system, comprising: a) at least one radiating electromagnetic radiation towards said occupant to illuminate most of said occupant. A transmitter; b) at least one receiver for receiving radiation reflected from said occupant, comprising measuring means for measuring the position of said occupant having a charge coupled device and pattern recognition means; C) means for mounting the at least one transmitter and the at least one receiver on the vehicle; d) coupled to the position measuring means of the at least one receiver and at the position of the occupant Analysis means responsive to another system of the vehicle.