JPH07329639A - Interior human body detecting device and vehicle equipment control device - Google Patents

Abstract

PURPOSE:To obtain a human body detecting device which detects a human body and a human body position accurately inside a vehicle regardless of the movement and the still condition of the human body, and an vehicle equipment control device using this human body detecting device. CONSTITUTION:A camera 1 to convert a photographed image in a vehicle into an electric signale, and a human body detector 2 to detect a human body and a human body position in the vehicle from the image signal obtained from the camera 1 by using an image processing means, are provided. Furthermore, a means to control the operating condition of a vehicle equipment such as an audio device to set the sound field production in the car room, to make a comfortable indoor circumstance, depending on the detecting result of the human body detecting device, is provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for automatically detecting the number of persons in the vehicle and the position of a human body from a photographed image in the vehicle, and an in-vehicle apparatus control apparatus for controlling an in-vehicle apparatus based on the detection result.

[0002]

2. Description of the Related Art The car, which is one of the essential items in modern society, is now the first high-tech product and its performance has improved significantly compared to before. On the other hand, comfort is required in the car. On the other hand, recently, a lot of luxurious equipment and abundant options are prepared, and efforts are being made to create a comfortable running space.

However, even in the inside of a vehicle which is said to be a comfortable space, for example, an air conditioner (hereinafter referred to as "air conditioner")
Currently, the operation of audio equipment and audio equipment (hereinafter referred to as "car audio") is left up to the driver, who is a typical occupant. In other words, the environment is set to make the driver most comfortable. Therefore, when a plurality of people board such a vehicle, a person close to the audio speaker may feel noisy, and depending on the location, the wind of the air conditioner may keep hitting and impair health. On the other hand, the volume is decreased and the wind direction of the air conditioner is changed, but these only change the state temporarily, and the original acoustic space and a comfortable air conditioning environment cannot be obtained.

From such an example, consider a car audio system, for example. In a general car audio system, sound output parts such as a front speaker and a rear speaker are fixedly installed at a specific position in the vehicle. Therefore, if the number of passengers on board changes or the seating position changes,
It is necessary to appropriately adjust the output of each speaker to generate an appropriate sound field. Otherwise the particular person close to the speaker is noisy and unavoidable. Also, depending on the position, the sound may be heard from only one direction, and it may be far from a comfortable acoustic space.

Therefore, as shown in FIG. 9, such adjustment is usually performed by the driver every time the seating position changes, by switching the positioning keys among various control keys installed in the DSP (Digital Signal Processor) unit 90 of the audio system. Switch 91 and, as shown in FIG.
The present situation is that a simple method of operating the front and rear fader knobs 101 and the left and right balance knobs 102 of the head unit 100 is used. Therefore, we cannot expect a satisfactory acoustic space.

Therefore, as an example of a method for solving such a problem, a human body detection system using an infrared sensor is used as a practical method.
No. 60075 is disclosed. FIG. 11 is a block diagram of this human body detection device. In the figure, 5 is a car body,
6FR is a front right seat, 6FL is a front left seat, 6R is a rear seat, and 110 is an infrared sensor for each seat.

[0007]

Generally, an infrared sensor for detecting a human body is a pyroelectric type that detects a change in the amount of infrared rays from the human body and cannot detect a stationary human body. Therefore, this type of infrared sensor detects a moving human body. However, since the movement of the human body in the vehicle is relatively small, it is difficult to detect and the detection accuracy itself is not good. Therefore, it is conceivable to increase the sensitivity of the infrared sensor to detect a slight movement of the human body, but there is a problem that it also causes an erroneous operation by detecting an infrared ray change of a heated object other than the human body.

Further, there is a problem that the same person is detected by a plurality of infrared sensors by detecting the infrared changes other than the corresponding seats. Furthermore, according to the infrared sensor,
There is a problem that only the human body position corresponding to the seat can be detected, and more detailed position information in the seat cannot be obtained.
Further, there is a problem that the infrared sensor here has no use other than for detecting a human body and the added value of the sensor is small.

The present invention has been made to solve the above problems, and a first object thereof is to accurately detect the human body and the position of the human body in the vehicle regardless of whether the human body is moving or stationary. Get the detector. And the second purpose is
An object of the present invention is to obtain an in-vehicle device control device that optimizes the operation of the in-vehicle device based on the detection result of the human body detection device.

[0010]

In order to solve the above-mentioned problems, the invention of claim 1 is directed to an image pickup means for picking up an image inside a vehicle, and a human body detection means for detecting a human body and a human body position from an image obtained from the image pickup means. Was set up.

According to the second aspect of the present invention, the human body position information from the human body detecting means is output to the electronic zoom control means. The electronic zoom control means is provided with display means for controlling the electronic zoom means to automatically enlarge the image and selectively switching between the enlarged image and the image captured by the image capturing means for display.

According to a third aspect of the present invention, there is provided a human body detecting means for detecting a human body and a human body position from an image signal from the image pickup means, a skin color extracting means for extracting a skin color area from the image signal, and the skin color extracting means. Noise removal means for removing or filling in small areas of the extracted skin color area, labeling means for sequentially numbering the skin color areas obtained by the noise removal means, and feature quantities of the areas labeled by the labeling means A feature amount calculation means for calculating, a face determination means for determining whether or not the area is a face from the feature amount obtained from the feature amount calculation means, and a human body based on position information of the area determined to be a face by the face determination means. And a human body position determining means for determining the position of.

According to the invention of claim 4, an image pickup means for picking up an image of the inside of the vehicle and converting the optical image thereof into an electric signal is installed above the front center of the inside of the vehicle.

According to a fifth aspect of the present invention, an image pickup means for picking up an image of the inside of the vehicle, a human body detection means for detecting a human body and a human body position from an image obtained from the image pickup means, and a vehicle mounted on the basis of the result of the human body detection means. In-vehicle device control means for setting the operation mode of the device is provided.

[0015]

According to the invention of claim 1, the human body and the human body position in the vehicle can be detected by the image pickup means.

According to the second aspect of the present invention, the visibility can be enhanced by enlarging and displaying the area including the human body at a predetermined enlargement ratio based on the human body position information.

According to the third aspect of the present invention, the human body and the human body position can be accurately detected from the skin color of the human body face.

According to the fourth aspect of the present invention, a wide area inside the vehicle can be imaged and the human body can be imaged from the front.

According to the fifth aspect of the invention, the in-vehicle device can be optimally controlled based on the information of the human body detecting means by the image pickup means.

[0020]

【Example】

Example 1. 1 is a schematic block diagram showing the configuration of a human body detection device in a vehicle according to a first embodiment of the present invention. In the figure, 1 is a camera unit that converts a captured image inside the vehicle into an electric signal, and 2 is a human body detection unit that detects the human body position in the vehicle by image processing the image signal from the camera unit 1,
3 is an audio control circuit which is an example of an in-vehicle device control device that controls the sound field generation in the vehicle based on the human body position detection result of the human body detection unit 2. 4FR is a front right speaker installed on the front right door, and 4FL is a front. A front left speaker installed on the left side door, 4RR is a rear right speaker installed on the rear right door, and 4RL is a rear left speaker installed on the rear left door.

FIG. 2 is a simplified layout view of a human body detecting device in a vehicle according to the first embodiment of the present invention. The same reference numerals as those in FIG. 11 denote the same or corresponding portions, and FIG.
2A is a perspective view from above the vehicle body, and FIG. 2B is a perspective view from the side of the vehicle body.

Next, the operation will be described. As shown in FIG. 2, by arranging the camera unit 1 above the center of the front part in the vehicle, a person sitting in each seat is imaged from the front.
At this time, if a relatively wide-angle lens is used, depending on the lens angle of view, the entire face and most of the upper body can be imaged.
The in-vehicle image captured by the camera unit 1 is converted into an electric signal and input to the human body detection unit 2. Human body detector 2
Then, the human body region is detected from the image signal input from the camera unit 1, and the number of people and the position are determined.

Here, a specific operation example of the human body detecting section 2 will be described with reference to FIG. The analog image signal a input from the camera unit 1 is input to the A / D converter 201 and converted into a digital image signal. Then, the skin color extraction circuit 2
In 02, the skin color portion is extracted. In this skin color extraction circuit 202, as shown in FIG. 4, the color difference signal R-Y is taken on the y-axis and BY is taken on the x-axis, and the hue is from .theta.1 to .theta.2 and the color saturation is from r1 to r2. Detects a thing as a skin color.
In FIG. 4, the shaded area is the skin color range. Although the range of colors to be detected is fan-shaped here, it may be a range of other shapes such as a square or a circle. Next, a binary image having a value of "1" for the part where the skin color is detected and "0" for the other parts is generated.

Next, noise removal circuit 203 removes noise or fills in holes. An expansion / contraction process is performed to remove noise from the binarized data. For example, contraction processing is 5
When the expansion process is performed five times in the order of five times, the small area of 5 × 5 pixels or less is removed. Conversely, expansion processing 5 times, contraction processing 5
When it is performed in the order of times, holes of 5 × 5 pixels or less are filled.

Next, labeling is performed by the labeling circuit 204. This is a process of assigning a number to each extracted area.

Next, the characteristic amount calculation circuit 205 calculates the characteristic amount. First, the barycentric position Gi, the area Ai, the maximum width Wi, the maximum height Hi, and the peripheral length Li of each region are obtained. Next, the aspect ratio Ri and the complexity Ci are calculated as the feature quantities by the following equations. Ri = Hi / Wi Ci = Li ² / Ai

Next, the face determination circuit 206 determines whether or not it is a face area. The judgment criteria are the following three points. 1) Ai> Amin Extract a region having a certain area or more. 2) Rmin <Ri <Rmax Extract the circumscribing rectangle whose aspect ratio is close to 1, that is, close to a square. 3) Ci <Cmax Extract a circle close to a circle. An area that satisfies the above three conditions is defined as a face image area. According to these three conditions, the circuit can be simple and the face image can be accurately extracted. The face determination circuit 206 outputs the barycentric position Gi of the area extracted as the face image.

Next, the human body position determination circuit 207 determines the human body position. For the determination, the seat on which the person is sitting and the position in the seat where the person is sitting are determined from the center of gravity position Gi of the area obtained by the face determining circuit. The human body position determination circuit 207 outputs the human body position inside the vehicle.

Regarding the human body detecting section 2, the skin color of the subject is referred to in the above. This skin color detection method is effective whether the subject is moving or stationary. However, in addition to this, for subjects with movement, instead of extracting the skin color, the difference between the images between frames is taken,
You may use this motion detection result. In addition, actually, the area of the human body can be specified without being limited to the method described above,
Any method can be used as long as the human body and position can be obtained.

Next, the audio control circuit 3 controls the human body position information obtained from the human body detecting section 2 so as to generate a sound field in an optimum mode for the human body position. For example, if you are sitting only on the front seats 6FR, 6FL, increase the output of the rear speakers 4RR, 4RL,
Control is performed to balance the front-rear volume with respect to the occupants seated in the front seats 6FR and 6FL. Furthermore, if the left and right positions of the human body sitting on the front seats 6FR and 6FL are close to the right side, the left speakers 4FL and 4FL
Control such that the output of RL is increased to balance the left and right sound volumes with respect to the occupants seated in the front seats 6FR and 6FL.

Although the first embodiment of the present invention has been described above, the present invention is not limited to the above-described first embodiment, and various modifications can be made. For example, in addition to the first embodiment, the human body position information from the camera unit 1 and the human body detection unit 2 is used to control the wind direction, the air volume, and the temperature switching of the air outlet of the in-vehicle air conditioner to perform the optimum air conditioning control. It is also possible.

Further, although the first embodiment is described on the premise that the camera unit 1 is exclusively used for detecting a human body,
It can also be used for other purposes. For example, as shown in FIG. 5, an image captured by the camera unit 1 is displayed on a monitor 7 installed in front of the front seat, and is also used as a surveillance camera system when a child or infant is placed in the rear seat. It is possible.

Furthermore, the camera unit 1 is made detachable from the vehicle body, and a video tape recorder (hereinafter referred to as "VTR").
If it can be connected to a recording device such as
It can be used even when the car is not used as a TR or a camera-separated VTR. Since the camera has various uses in this way, not only for detecting the human body position,
It is possible to have a system configuration with increased added value.

Example 2. Next, a combined system of the human body detection device shown in the first embodiment and a rear seat monitoring device capable of monitoring a person riding in a rear seat will be described. The rear seat monitoring device
This is a device for checking the state of the rear seat on the monitor in a normal sitting posture with the occupant sitting in the front seat looking at the image of the rear seat, and especially for the child or infant in the rear seat. It is used for the purpose of monitoring movements when a passenger is seated, and for supporting the communication between the front and rear seats by displaying the facial expressions and gestures of the person in the rear seat.

FIG. 6 is a block diagram showing the structure of the human body detecting device in the vehicle according to the second embodiment. In the figure,
3, 201-207, 4FR, 4FL, 4RR, 4RL
1 is the same as FIG. 1 and FIG. 3 shown as the first embodiment, and the description thereof will be omitted. Reference numeral 8 is an electronic zoom circuit for electronically enlarging the input image, 9 is an electronic zoom control circuit for controlling the enlargement ratio and enlargement position of the electronic zoom circuit 8 based on the determination results of the face determination circuit 206 and the human body position determination circuit 207, and 10 is A D / A converter for converting the digital enlarged image from the electronic zoom circuit 8 into an analog signal, 11 is a changeover switch for selecting the image signal from the camera section 1 and the image from the D / A converter 10, and 7 is a changeover switch. 11 is a monitor for outputting the selected image. Further, as shown in FIG. 7, the monitor 7 is installed in front of the front seat in the vehicle.

Next, the operation will be described. First, by arranging the camera unit 1 above the center of the front part of the vehicle as shown in FIG. 7, most of the upper body can be imaged from the front of a person seated in the front seat and the rear seat. The in-vehicle image captured by the camera unit 1 is converted into an electric signal,
The digital image signal is input to the electronic zoom circuit 8 by the / D converter 201.

The output signal of the A / D converter 201 is processed by the skin color extraction circuit 202, the noise removal circuit 203, the labeling circuit 204, the feature amount calculation circuit 205, and the face determination circuit 206, respectively. Judge your face,
Center of gravity position Gi and area Ai of the area extracted as the face image
Is output. The center of gravity position Gi is input to the human body position determination circuit 207 and the electronic zoom control circuit 9, and the area Ai is input to the electronic zoom control circuit 9.

The human body position determination circuit 207 determines the center of gravity position Gi.
Determine from which seat the person is sitting,
The information on the human body position is output to the audio control circuit 3 and the electronic zoom control circuit 9.

In the electronic zoom control circuit 9, the face determination circuit 20
6 and the position of the center of gravity Gi, the area Ai and the human body position information of the area extracted as a face from the human body position determination circuit 207, and the electronic zoom circuit 8 to enlarge the face image when the human body position is the rear seat. To control.

The operation of the electronic zoom control circuit 9 will be described with reference to FIG. First, it is assumed that an image of the inside of a vehicle in which a child is seated in the rear seat is obtained as shown in 801. The face determination circuit 206 recognizes the portion indicated by 802 as a face from the image of 801 and obtains its size and position. The electronic zoom control circuit 9 that has obtained the size and position information of the face enlarges the area 803 by the electronic zoom circuit 9 in order to obtain a bust shot image based on the information of the face area 802, and Image 804 is obtained. Here, the information on the size and the position of the face is always obtained from the human body detection circuit 2, and the control is performed so as to be close to the information on the size and the position of the face at the time of the bust shot given in advance. In summary, based on the information of the face area 802 in the image 801, the area 803 is electronically zoomed.
To obtain a bust shot image 804.

The bust shot image thus obtained is converted into an analog signal by the D / A converter 10 and input to the changeover switch 11. Changeover switch 1
The output signal of the camera unit 1 is input to the other input of 1. By switching the changeover switch 11, the input image itself or the bust shot of the person in the back seat can be selected.

The output signal of the changeover switch 11 is input to the monitor 7 to display a photographed image of the inside of the vehicle. Since the monitor 7 is installed in front of the front seat as shown in FIG. 7,
A person seated in the front seat visually recognizes the image on the monitor 7 and freely selects the switching of the changeover switch 11 to monitor the motion of the child or infant in the rear seat, and to check the operation with the occupant in the rear seat. You will be able to fully communicate.

On the other hand, the audio control circuit 3 controls the human body position information obtained from the human body detecting section 2 so as to generate a sound field in an optimum mode for the human body position. For example, if you are sitting only on the front seats 6FR, 6FL, increase the output of the rear speakers 4RR, 4RL,
Control is performed to balance the front-rear volume with respect to the occupants seated in the front seats 6FR and 6FL.

Thus, the camera unit 1 and the human body detecting unit 2 can be used together to realize the human body position detecting function and the rear seat detecting function.

Although the second embodiment of the present invention has been described above, the present invention is not limited to the above-described second embodiment, and various modifications can be made. For example, in addition to the second embodiment, the human body position information from the camera unit 1 and the human body detection unit 2 is used to control the wind direction, the air volume, and the temperature switching of the air outlet of the in-vehicle air conditioner to perform the optimum air conditioning control. It is also possible.

Although the method of detecting the face image area is used as the human body detection circuit 2, if the area of the human body can be specified and its position and size can be obtained, a bust shot image can be obtained from the information. Obtainable. In addition, although the bust shot is assumed as the final composition, it is not limited to this, and if it is a composition related to a person, such as a shot of the whole body, placing only the face in the lower left of the screen,
Anything will do. It is also possible to preset these plural compositions and allow the user to select and use them.

[0047]

As described above, according to the first aspect of the present invention, since the human body detecting means by the image pickup means is provided, the human body can be accurately detected whether the human body is moving or stationary.

Further, according to the invention of claim 2, the electronic zoom means is controlled by the information from the human body detecting means to display an image in which the human body region is automatically enlarged on the display means.
No special operation is required to enlarge the size, which improves visibility and enables monitoring of the rear seats.

According to the third aspect of the present invention, the still human body can be accurately detected by detecting the face area based on the skin color.

According to the fourth aspect of the present invention, by installing the image pickup means above the center of the front part, it is possible to detect a human body in a wide range in the vehicle, so that the detection accuracy can be improved.

According to the fifth aspect of the invention, the in-vehicle device is controlled based on the detected information about the human body and the human body position, so that the optimum environment can be easily created regardless of the passenger's riding position.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing the configuration of an in-vehicle human body detection device according to a first embodiment of the present invention.

FIG. 2 is a simplified layout view of a human body detection device in a vehicle according to the first embodiment.

FIG. 3 is a block diagram showing details of a human body detecting unit of the human body detecting device in the vehicle according to the first embodiment.

FIG. 4 is a skin color range diagram showing an extraction range of a skin color extraction circuit of the human body detection device in the vehicle according to the first embodiment.

FIG. 5 is a block diagram showing a modification of the human body detection device in the vehicle according to the first embodiment.

FIG. 6 is a block diagram showing a configuration of an in-vehicle human body detection device according to a second embodiment of the present invention.

FIG. 7 is a simplified layout view of a human body detection device in a vehicle according to a second embodiment.

FIG. 8 is a diagram for explaining the operation of the human body detection device in the vehicle according to the second embodiment.

FIG. 9 is a diagram showing a configuration of a conventional vehicle-mounted device control device.

FIG. 10 is a diagram showing a configuration of a conventional vehicle-mounted device control device.

FIG. 11 is a diagram showing a configuration of a conventional human body detection device in a vehicle.

[Explanation of symbols]

1 camera section, 2 human body detection section, 3 audio equipment control circuit, 7 monitor, 8 electronic zoom circuit, 9 electronic zoom control circuit, 202 skin color extraction circuit, 203 noise removal circuit, 204 labeling circuit, 205 feature amount calculation circuit, 206 Face discrimination circuit, 207 Human body discrimination circuit.

Claims (5)

[Claims] 1. An in-vehicle human body detecting device comprising: an image pickup means for picking up an image of the inside of a vehicle and converting an optical image thereof into an electric signal; and a human body detecting means for detecting a human body and a position of the human body based on an image signal from the image pickup means. . 2. An image pickup means for picking up an image of the inside of a vehicle and converting an optical image thereof into an electric signal, a human body detecting means for detecting a human body and a position of the human body based on an image signal from the image pickup means, and an enlargement processing of the image signal. Electronic zoom means, electronic zoom control means for controlling the electronic zoom means based on human body position information from the human body detecting means, image picked up by the image pickup means and image enlarged by the electronic zoom means A human body detection device in a vehicle, comprising: a display unit for selectively switching and displaying. 3. A human body detecting means for detecting a human body position based on an image signal from an image pickup means, a skin color extracting means for extracting a skin color area from the image signal, and a small area of the skin color area extracted by the skin color extracting means. Then, the noise removing means for filling in the holes, the labeling means for sequentially numbering the skin color areas obtained by the noise removing means, and the feature quantity calculating means for calculating the feature quantity of the area labeled by the labeling means,
A face determination unit that determines whether or not the region is a face based on the feature amount obtained from the feature amount calculation unit, and a human body position determination unit that determines the position of the human body from the region determined to be the face by the face determination unit The human body detection device in the vehicle according to claim 1 or 2, characterized in that. 4. An image pickup means for picking up an image of the inside of a vehicle and converting an optical image of the inside into an electric signal is installed above the center of the front part of the inside of the vehicle. The in-vehicle human body detection device described in. 5. The in-vehicle device control means according to claim 1, further comprising: an in-vehicle device control means for setting a predetermined operation mode of the in-vehicle device based on a result of the human body detection means. In-vehicle equipment control device.