JP2009107527A - Occupant detection device of vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To normally and exactly detect an occupant without being affected by brightness in a cabin in an image pickup and incident light, in a device for processing imaged image in the cabin to detect the occupant. <P>SOLUTION: When opening of a door is detected (or predicted) (S110-YES), the imaged image in the cabin is taken in and is stored as a reference image (S120). An object detecting processing (S140) for detecting a region changed from the reference image at a seat position in the vicinity of the opened door is executed by comparing the imaged image in the cabin with the reference image till the door is closed, and the presence or absence of an object is decided based on the detection result when the door is closed (S160). When the presence of the object is determined, an object detecting processing (S180) for detecting the region changed from the reference image at the seat position in the vicinity of the opened and closed door in the same procedure as the above for a fixed time and determining whether the region is moving is executed. When the fixed time elapses, whether the occupant sits on the seat position is determined based on the determination result (S200). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a vehicle occupant detection device that detects an occupant in a vehicle interior by processing an image obtained by imaging the vehicle interior.

Conventionally, as this kind of passenger detection device,
(1) The interior of the vehicle is imaged, the seat in the vehicle interior and the outline of the object on the seat are detected from the captured image, spot light is irradiated to each part of the seat, and the seat is based on the position where the reflected light is received. Measuring the distance to each part, based on the shape of the detected contour and the distance to the object specified by the contour, to determine whether there is an occupant in the passenger compartment,
(2) The interior of the vehicle is imaged, the edge of the image formed by the seat in the vehicle interior and the object on the seat is detected from the captured image, and whether there is a passenger in the vehicle interior based on the detected edge shape Configured to determine whether or not
(3) An image of the interior of the vehicle, and the captured image is compared with a pre-stored reference image to determine whether or not the passenger is in the passenger compartment.
Etc. are known (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-0778657

  However, in the conventional occupant detection device, the captured image in the passenger compartment changes greatly depending on the imaging conditions such as the brightness of the passenger compartment and the incident direction of light into the passenger compartment. Depending on the time of day, the presence or absence of a passenger may not be detected accurately.

  That is, in the occupant detection devices of (1) and (2) above, the captured image is processed to extract the contours and edges of the image, thereby recognizing the shape of the object including the seat of the vehicle, and the shape becomes the seat. The presence or absence of an occupant is determined based on whether or not the shape of the seated occupant is present.

  Therefore, in this apparatus, it is necessary to image the passenger compartment so that the seat and the occupant can be identified only by image processing. However, depending on the imaging conditions described above, the seat and the occupant can be identified on the captured image. In some cases, the presence or absence of an occupant may be erroneously determined.

  In order to prevent such an erroneous determination, the apparatus of (1) above irradiates the seat with spot light and measures the distance from the reflected light receiving position to the seat or object hit by the spot light. However, for example, when direct sunlight is incident on the passenger compartment, distance measurement cannot be performed using spot light, and the time zone during which erroneous determination can be prevented by distance measurement is limited. There is a problem that it ends up.

  On the other hand, in the apparatus of (3) above, the presence / absence of an occupant is determined by comparing the captured image obtained by imaging the passenger compartment with the reference image, and therefore the reference image is captured under the same conditions as the captured image at the time of detection. If so, the presence or absence of a passenger can be accurately determined.

  However, since occupant detection must be performed frequently when the door is opened and closed, the imaging conditions for imaging the passenger compartment for occupant detection vary depending on the season, time zone, weather, etc. There is little probability of matching with the imaging conditions at the time of imaging. For this reason, even in the apparatus of (3), the presence or absence of an occupant may be erroneously determined due to a difference in imaging conditions between the reference image and the captured image at the time of occupant detection.

  The present invention has been made in view of such problems, and in a vehicle occupant detection device that determines whether or not there is a passenger in a vehicle by processing a captured image in the vehicle, the brightness and incidence of the vehicle at the time of imaging. It is an object to make it possible to always accurately determine the presence or absence of an occupant without being affected by light.

  In the occupant detection device according to claim 1, which has been made to achieve the above object, it is detected by the door state detection means that the door is opened, or the door is opened based on the operating state of each part of the vehicle. If this is predicted, the reference image storage means captures the captured image in the vehicle interior from the imaging means, and stores the captured image as the reference image.

  Further, when the door state detecting means detects that the door is opened and closed, the occupant detecting means captures the captured image in the vehicle interior from the imaging means, and stores the captured image and the reference image storage means. An occupant in the passenger compartment is detected by comparing with the reference image.

  That is, in the present invention, when the door of the vehicle is opened or when the door is predicted to be opened, the vehicle interior is imaged, and the captured image is stored as a reference image used for determining the presence or absence of an occupant. Therefore, the reference image is an image obtained by imaging the passenger compartment immediately before the occupant opens the door and gets on (or gets off) the vehicle.

  Therefore, the reference image and the image captured thereafter are images captured under substantially the same imaging conditions, and the occupant is detected depending on the imaging conditions of these images as in the conventional device of (3) above. The accuracy is not reduced.

  Further, in the present invention, when the door of the vehicle is opened or before the door is opened, the interior of the vehicle is imaged, and this is used as a reference image, and then the presence or absence of a passenger is determined by comparing the image with the captured image. Therefore, unlike the conventional apparatuses (1) and (2) (that is, apparatuses that do not use the reference image), the imaging conditions that can detect the occupant are not limited.

  Therefore, according to the present invention, an occupant in a vehicle compartment can be detected accurately at all times, and an on-vehicle device (air conditioner, audio device, airbag) provided in the vehicle compartment is utilized by using the detection result. Etc.) can be optimally controlled.

  By the way, since the vehicle is usually provided with a plurality of doors, and the seats on which the passengers who get on and off by opening the doors are substantially determined, the passenger detection device according to the present invention further includes: It may be configured as described.

  That is, in the occupant detection device according to claim 2, the door state detection unit is configured to detect an open / close state for each of a plurality of doors provided in the vehicle, and the occupant detection unit is a door state detection unit. The position of the seat to be detected is specified based on the position of the door that is detected to be opened and closed, and the images in the region corresponding to the seat position are compared among the captured image and the reference image captured from the imaging unit. Thus, an occupant in the seat position is detected.

  Therefore, according to the occupant detection device of the second aspect, when detecting the occupant, it is not necessary to compare the reference image and the captured image in the entire imaging region captured by the imaging means in the vehicle interior. Thus, the image processing required for the comparison (and thus detection of the occupant) can be performed in a short time.

  In addition, since the image area for comparing the captured image and the reference image can be limited to the seat position where the occupant may get on or get off, the influence of noise included in the image area that is not necessary for occupant detection is compared during image comparison. As a result, it is difficult to receive the occupant, and the detection accuracy of the passenger can be improved.

  On the other hand, the occupant detection means obtains a captured image of the vehicle interior from the imaging means not after the door is closed but between the time when the door is opened and closed (that is, when the passenger gets in or out of the vehicle interior). However, it may be compared with a reference image.

  However, while the door is open, the image changes greatly due to passengers getting in and out of the passenger compartment, loading and unloading of luggage, and the amount of light incident from outside the vehicle also increases. In some cases, the presence or absence of an occupant cannot be determined accurately.

For this reason, the occupant detection means may be configured to acquire a captured image from the imaging means and compare it with a reference image after the door state detection means detects that the door is closed.
In this case, the occupant detection unit compares the captured image acquired from the imaging unit with the reference image to identify an image region that has changed from the reference image in the captured image, and the shape and size of the region. It may be determined whether or not the passenger is seated from, but simply by the shape and size of the image area changed from the reference image, the luggage placed on the seat and the passenger cannot be distinguished, It may be possible to erroneously determine whether there is a passenger.

  Therefore, as described in claim 3, the occupant detection means captures a plurality of captured images from the imaging means until a predetermined period elapses after the door state detection means detects that the door is closed. The captured plurality of captured images and the reference image are respectively compared, and an occupant is detected based on a change in the comparison result.

  In other words, in this way, it is determined whether or not there is an occupant by detecting not only the size and shape of the object on the seat, but also the movement of the object from the comparison result between the captured image in the passenger compartment and the reference image. As a result, the occupant detection accuracy can be further improved.

  The occupant detection means according to claim 3 determines the seating state only for a predetermined period after the door is closed, so that the seating state is always determined while the door is closed. In comparison, it is possible to reduce the processing load of the image processing apparatus or computer that functions as the occupant detection means.

  Next, the reference image storage means acquires a captured image from the imaging means when the door state detection means detects that the door is opened (when the door is opened), and stores it as a reference image. Alternatively, when it can be predicted that the door will open from the operating state of each part of the vehicle, the captured image may be acquired from the imaging means and stored as a reference image, or when the door can be predicted to open When it is detected that the door is opened without being predicted, a captured image may be acquired from the imaging unit and stored as a reference image.

  When the reference image storage means is configured to store the captured image in the passenger compartment as the reference image when it is predicted that the door will open, the electronic device for operating the host vehicle as described in claim 4. If the receiving means for receiving the transmission radio wave wirelessly transmitted from the key is mounted on the vehicle, when the receiving means receives the transmission radio wave from the electronic key in the reference image storage means, the door changes to the open state. It is preferable to predict and acquire a captured image from the imaging means and store it as a reference image.

  According to a fifth aspect of the present invention, there is provided stop detection means for detecting stop of the vehicle, and when the stop of the vehicle is detected by the stop detection means, the reference image storage means receives the captured image from the image pickup means. It may be acquired and stored as a reference image.

  In this way, if the vehicle interior is imaged by predicting that the door will open and the captured image is stored as a reference image, the captured image when the door is completely closed is used as the reference image. Therefore, it is possible to detect the occupant more accurately than when the captured image captured when the door is opened is used as the reference image.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration of the entire occupant detection device of the embodiment.
As shown in FIG. 1, an occupant detection device 2 according to the present embodiment includes a vehicle interior camera 4 that is arranged on the ceiling in front of a vehicle interior so as to be able to image the entire seat including a driver seat, a passenger seat and a rear seat. And door switches (hereinafter referred to as “door SW”) 6FL, 6FR, 6RL, 6RR, which are provided on each of the four entrance doors of the vehicle, and detect the open / closed state of each door, and these parts 4, 6FL-6RR. An electronic control device (hereinafter referred to as an occupant detection ECU) 10 for detecting an occupant is connected.

  Of the doors SW6FL to 6RR, the door SW6FL is provided on the left front seat door (FL) which is the door on the passenger seat side, and the door SW6FR is provided on the right front seat door (FR) which is the door on the driver seat side. The door SW6RL is provided on the left rear seat door (RL) that is the left door on the rear seat, and the door SW6RR is provided on the right rear seat door (RR) that is the door on the right side of the rear seat.

  Next, the occupant detection ECU 10 determines the open / closed state of each door from the on / off states of the doors SW6FL to 6RR, and images (hereinafter referred to as seat images) obtained by capturing all seats in the vehicle interior from the vehicle interior camera 4. By capturing and processing the captured seat image, it is determined whether or not an occupant is seated in each seat, and the determination result is output to the air conditioner ECU 30, the audio ECU 40, the airbag ECU 50, and the like. The microcomputer is composed mainly of a CPU 12, a ROM 14, a RAM 16, a bus 18 that connects these parts, and the like.

  The bus 18 of the occupant detection ECU 10 includes an image processing unit 20 that processes a captured image captured by the vehicle interior camera 4, a switch input unit 22 that detects an on / off state of each of the doors SW6FL to 6RR, and A communication unit 24 that performs data communication with other ECUs (air conditioner ECU 30, audio ECU 40, airbag ECU 50,...) Mounted on the vehicle is connected via a communication line 26 wired in the passenger compartment. Yes.

  The air conditioner ECU 30 is for controlling an air conditioner (air conditioner) mounted on the vehicle. For example, when an occupant detection result is received from the occupant detection ECU 10, the air conditioner ECU 30 corresponds to the seating position of the occupant in the passenger compartment. Adjust the blowing direction of conditioned air.

  The audio ECU 40 is for controlling an audio device mounted on the vehicle. For example, when receiving an occupant detection result from the occupant detection ECU 10, the audio ECU 40 has a plurality of speakers according to the seating position of the occupant in the passenger compartment. Adjust the audio output balance, volume, etc.

  Further, the airbag ECU 50 controls the operation state of the airbag of each seat in the vehicle interior. For example, when the detection result of the occupant is received from the occupant detection ECU 10, the airbag ECU 50 is disposed in the seat on which the occupant is seated. The airbag is set in an operable state, and the operation of the airbag disposed in a seat where no occupant is seated is stopped.

  Each of the ECUs 40, 50, 60 is configured with a microcomputer as the center, like the occupant detection ECU 10, and is capable of data communication with other ECUs via the communication line 26.

Next, the seating state determination process executed by the CPU 12 of the occupant detection ECU 10 to determine the seating state of the occupant in each seat of the vehicle will be described with reference to the flowchart shown in FIG.
As shown in FIG. 2, in this seating state determination process, first, in S110 (S represents a step), it is determined whether any of the doors SW6FL to 6RR is in an on state. It is determined whether any of the two doors is opened.

  If all of the doors SW6FL to 6RR are in an off state and all four doors of the vehicle are closed, by executing S110 again, it waits for any of the four doors to open, and the doors SW6FL to 6RR are opened. If any of the four doors of the vehicle is opened by changing from the off state to the on state, the process proceeds to S120.

  In S120, a captured image (for example, the image shown in FIG. 5A) is acquired from the vehicle interior camera 4 via the image processing unit 20, and the RAM 16 (or the image processing unit 20 is used with the acquired captured image as a reference image). In the image memory).

Next, in S130, the object detection score is set to an initial value “0”, and in subsequent S140, an object detection process for updating the object detection score is executed.
In this object detection process, a captured image is acquired from the in-vehicle camera 4 and compared with a reference image, thereby detecting an image region that has changed from the reference image in the captured image, and an object detection score is calculated based on the detection result. This is an update process and is executed according to the procedure shown in FIG.

  That is, as shown in FIG. 3, in the object detection process, first, in S310, a captured image (for example, an image shown in FIG. 5B) is acquired from the vehicle interior camera 4 via the image processing unit 20, In subsequent S320, the difference image illustrated in FIG. 5C is generated by detecting the difference between the acquired captured image and the reference image stored in S120.

  Note that when generating the difference image in S320, the seat position of the occupant who opens and gets off the door based on the position of the door that is open at that time is specified, and the captured image and the previously acquired image The image data of an area corresponding to the specified seat position (hereinafter referred to as an extraction area) is extracted from the reference image stored in the above.

  The difference image is obtained by extracting the pixel value for each pixel from the image data in the extraction region extracted from the captured image and the reference image, and obtaining the absolute value of the difference between the pixel values. The pixel value of the pixel is generated, and the pixel value of the pixel other than the extraction region is set to zero (or the maximum value).

  For this reason, in the present embodiment, when generating a difference image, a seat position for setting an extraction region for extracting image data from a captured image and a reference image is set in advance for each door of the vehicle. As illustrated in FIG. 5, when the door on the passenger seat side (the left front seat door) is opened, the passenger seat is selected as a seat position from which image data is to be extracted, and an image area corresponding to the passenger seat is extracted from the image data. It will be set as an area.

In FIG. 5, the frame of the extraction area is a quadrangle, but the shape of this frame may correspond to the shape of each seat, or may be set to the shape of a person seated in each seat .
Next, when a difference image is generated in S320, a binarized image illustrated in FIG. 5D is generated by binarizing the generated difference image in S330. In subsequent S340, the binarized image is subjected to a labeling process to extract a pixel region having pixel values different from the background as an object region as illustrated in FIG. 5E.

  In the subsequent S350, as illustrated in FIG. 5F, the object region having an area smaller than the noise determination threshold is removed as noise from the object regions extracted in S340. A region is determined, and in S360, the area ratio of the determined object region in the extraction region is calculated.

  Finally, in S370, it is determined whether or not the area ratio of the object region is equal to or greater than a preset size determination threshold value. The object detection score is updated by adding the value “1” to the object detection score, and the object detection process is terminated. If the area ratio of the object region is less than the threshold value, the object detection process is terminated without updating the object detection score.

  When the object detection process of S140 is executed in this way, this time, the process proceeds to S150, and whether or not all of the doors SW6FL to 6RR are in an off state (in other words, all four doors of the vehicle are closed). If all the doors are not closed, the process proceeds to S140 again, and the object detection process of S140 is repeatedly executed until all four doors of the vehicle are closed.

  On the other hand, when it is determined in S150 that all four doors of the vehicle are closed, the process proceeds to S160. In S160, the object detection score updated by repetitively executing the object detection processing is opened or closed this time by determining whether or not the value of the object detection score is greater than or equal to a preset object determination threshold value. It is determined whether there is an object in the seating area corresponding to the door.

  If it is determined in S160 that the value of the object detection score is equal to or greater than the threshold value and there is an object, the process proceeds to S170, and conversely, the value of the object detection score is less than the threshold value and it is determined that there is no object. Then, the seating state determination process is temporarily terminated, and thereafter, the processes after S110 are executed again.

Next, in S170, the occupant detection score is set to an initial value “0”, and in subsequent S180, an occupant detection process for updating the occupant detection score is executed.
In this occupant detection process, a captured image is acquired from the in-vehicle camera 4 and compared with a reference image to detect an image area that has changed from the reference image in the captured image, and whether or not the detection result has changed. Is a process for updating the occupant detection score, and is executed according to the procedure shown in FIG.

  That is, as shown in FIG. 4, in the occupant detection process, first, in S410, a captured image is acquired from the vehicle interior camera 4 via the image processing unit 20, and in the subsequent S420, the acquired captured image and A difference image is generated by detecting a difference from the reference image stored in S120.

  When generating the difference image in S420, the image is opened and closed after it is determined in S110 that any of the four doors has been opened until it is determined in S150 that all the doors have been closed. Based on the position of the opened door, the seat position of the occupant who opens and gets off the door is specified, and then the identification is performed from the captured image acquired this time and the previously stored reference image in the same procedure as S320. Image data of an area corresponding to the seat position (hereinafter referred to as an extraction area) is extracted.

  Next, when a differential image is generated in S420, in subsequent S430 to S450, an image area (extraction area) of the seat position corresponding to the door that is opened and closed this time in the same procedure as S330 to S350 in FIG. The image data of the extraction region divided into the object region and the background region is generated as shown in FIG. 5F by detecting the image region (object region) that has changed from the reference image. The process proceeds to S460.

  In S460, using the image data of the extraction area generated as described above and the image data of the extraction area generated when the occupant detection process is executed n times before (n is a positive integer), By calculating an exclusive OR (exclusive or: XOR) for each pixel of each image data, one of the detection results of the object region detected this time and the object region detected n times before is the object region. Identify pixels that are not.

  In the subsequent S470, the number of pixels specified in S460 (that is, the number of pixels having the value “1” as a result of the XOR calculation) is equal to or greater than a threshold value set in advance to determine the change in the object area. If the number of pixels is equal to or greater than the threshold value, it is determined that the object region has changed, and the value “1” is added to the occupant detection score to update the occupant detection score. And the said passenger | crew detection process is complete | finished. If the number of pixels is less than the threshold value, the occupant detection process is terminated without updating the occupant detection score.

  In calculating the XOR of the image data in S460, when all the doors are closed and the occupant detection process has not been executed n times or more (that is, there is no image data of the extraction area generated n times before). In the case), the occupant detection process is terminated without executing the processes of S460 and S470.

  When the occupant detection process of S180 is executed in this manner, the process proceeds to S190, and it is determined in S150 that all the doors have been closed, and whether or not a predetermined time has elapsed. If the predetermined time has not elapsed, the process proceeds to S180 again. As a result, the occupant detection process in S180 is repeatedly executed until a predetermined time elapses after all doors are closed.

  On the other hand, if it is determined in S190 that a certain time has elapsed since all the doors were closed, the process proceeds to S200, and the value of the occupant detection score updated by repeatedly executing the occupant detection process is as follows. It is determined whether or not the object placed at the seat position corresponding to the door that has been opened / closed this time is moving by determining whether or not it is equal to or greater than a preset threshold value for determining the occupant. Determine if you are seated.

  In the subsequent S210, the determination result in S200, that is, the seating state of the occupant (in other words, the presence or absence of the occupant) in the seat to be detected is transmitted to another ECU (air conditioner ECU 30, audio ECU 40, air ECU) via the communication unit 24. To the bag ECU 50 etc.), and the seating state determination process is temporarily terminated.

  As described above, in the occupant detection device 2 of the present embodiment, when any of the doors of the vehicle is detected to be opened by the doors SW6FL to 6RR, the occupant detection ECU 10 detects that fact. Then, a seat image obtained by capturing all seats in the vehicle interior is captured from the vehicle interior camera 4 and stored as a reference image.

  Then, in the occupant detection process executed until the vehicle door is closed and a predetermined time elapses, the seat image captured by the vehicle interior camera 4 is compared with the reference image, thereby obtaining a seat image. When the object area changed from the reference image is identified and the occupant detection score for occupant presence / absence determination is updated based on the change in the object area, and the occupant detection score exceeds a threshold value, the vehicle seat Is determined to be seated.

  Therefore, according to the present embodiment, the seat image captured by the in-vehicle camera 4 for occupant detection and the reference image used to determine the presence or absence of the occupant based on the seat image are substantially the same. Images are captured under the conditions, and the detection accuracy of the occupant is not lowered due to the difference in the imaging conditions of these images.

  In the present embodiment, the presence or absence of an occupant is determined by comparing the seat image captured by the in-vehicle camera 4 and the reference image. Therefore, the occupant is determined only by the image captured by the camera without using the reference image. The imaging conditions that can detect the occupant are not limited as in the conventional apparatus that determines the presence or absence of the vehicle.

  Therefore, according to the occupant detection device 2 of the present embodiment, it becomes possible to always detect the occupant in the vehicle compartment at all times, and on the basis of the detection result, an in-vehicle device (air conditioner, Audio equipment, airbags, etc.) can be optimally controlled.

  Further, in the present embodiment, when the object region is specified by comparing the seat image and the reference image in the occupant detection process, the seat position to be detected is specified from the position of the door in the open state. Thus, only the image data in the image area (extraction area) corresponding to the seat position is extracted.

  For this reason, when comparing the seat image and the reference image, it is not necessary to compare the images in the entire imaging region by the vehicle interior camera 4, and image processing (specifically, the detection of the occupant) is necessary. Can perform the processing of S320 to S350 in a short time.

  In addition, since the image area for comparing the seat image and the reference image can be limited to the seat position where the seating state is predicted to change depending on the occupant's getting on (or getting off), the occupant can perform the processing of S320 to S350. It becomes difficult to be influenced by noise included in an image area unnecessary for detection, and this can also improve the detection accuracy of the occupant.

  Next, in the present embodiment, by comparing the seat image captured by the vehicle interior camera 4 and the reference image until the vehicle door is once opened and closed, the reference image in the seat image is compared. When an image detection area (object area) that has changed is detected, an object detection process is performed to determine whether there is an object in the seat from the area of the object area, and when it is determined that there is an object in the seat Only the occupant detection process is executed.

  For this reason, according to this embodiment, an occupant seated in a vehicle seat is detected by performing the object detection process and the occupant detection process in stages, and the occupant is detected only by the occupant detection process. Compared with the case where it does in this way, the detection accuracy of a passenger | crew can be improved.

  In the present embodiment, the vehicle interior camera 4 corresponds to the imaging means of the present invention, and the doors SW6FL to 6RR correspond to the door state detection means of the present invention. Of the seating state determination processing (FIG. 2) executed by the occupant detection ECU 10, the processing of S110 and S120 corresponds to the reference image storage means of the present invention, and includes the occupant detection processing shown in FIG. The process of S200 corresponds to the occupant detection means of the present invention.

As mentioned above, although one Embodiment of this invention was described, this invention is not limited to the said embodiment, A various aspect can be taken in the range which does not deviate from the summary of this invention.
For example, in the above embodiment, until the vehicle door is once opened and closed, the image captured by the camera is compared with the reference image, and the object region changed from the reference image is detected in the captured image. Although it has been described that the object detection process for determining the presence / absence of an object from the area is performed, it is not always necessary to execute this process. Even if the passenger detection process is performed after the door is closed, The seating state of the passenger in each seat can be detected.

  In the above embodiment, the object detection process is described as being executed after the door is opened until the door is closed. However, both the object detection process and the occupant detection process are performed after the door is closed. You may make it perform in order. In this way, since the object detection process can be executed with the door closed, the presence or absence of an object is erroneously determined by the light entering the vehicle interior or the movement of the object when the door is open. Can be prevented.

  On the other hand, in the above embodiment, when it is detected that any of the doors of the vehicle is opened via the doors SW6FL to 6RR, the seat image is captured from the vehicle interior camera 4 and stored as a reference image. explained.

  However, since this reference image is a seat image when the door is opened by the occupant, depending on the operation (particularly speed) when the occupant opens the door, the door is already large when the vehicle interior camera 4 captures the image. It is also conceivable that a seat image that is opened and picked up with light incident from the opening is set as a reference image. In such a case, it is conceivable that the presence or absence of an occupant cannot be accurately determined by comparing the seat image captured after the door is closed with the reference image.

  For this reason, when it can be predicted that the door will open before it is detected by the doors SW6FL to 6RR, a captured image (seat image) is acquired from the vehicle interior camera 4 immediately after the prediction, You may make it memorize | store this as a reference | standard image.

  For example, in an automobile, when a radio wave transmitted from an electronic key (smart key) possessed by an occupant (driver) is received, the door lock is ready to be released. After that, when the occupant touches the operation part of the door, the door is locked. Smart entry systems configured to release are known.

  For this reason, when the ECU for the smart entry system (see the smart ECU 60 indicated by the dotted line in FIG. 1) is mounted on the vehicle, the occupant detection ECU 10 executes the reference image acquisition process shown in FIG. It is good to do.

  That is, in this reference image acquisition process, in S510, data communication is performed with the smart ECU 60 to determine whether the smart key is currently approaching the vehicle. (S510-YES), one of the vehicle doors is predicted to be opened, and the process proceeds to S520, where a captured image (seat image) is acquired from the vehicle interior camera 4, and the image is used as a reference image for a certain period of time. Remember. In this case, the smart ECU 60 corresponds to the receiving means described in claim 4.

  In this case, in the seating state determination process, as shown in FIG. 6B, when it is detected in S110 that any of the four doors is open, the process proceeds to S115. It is determined whether or not the reference image is already stored by the reference image acquisition process. If the reference image is stored, the process proceeds to S130 without executing the process of S120, and conversely, the reference image is stored. If not, the reference image is stored in S120, and then the process proceeds to S130.

  If the reference image acquisition process and the seating state determination process are executed in this way, a certain time elapses after the smart ECU 60 detects the approach of the smart key (and thus the occupant) to the vehicle. In the meantime, when the door of the vehicle is opened, the image in the passenger compartment imaged when the approach of the smart key is detected is used as the reference image, and the reference image is displayed immediately before the door is opened. An indoor seat can be imaged.

  When a certain time has elapsed after detecting the approach of the smart key, the stored reference image is erased. However, even if the approach of the smart key is detected, the occupant does not always open the door. This is because if the reference image is continuously stored as it is, the captured image may be different from the captured image when the door is opened and closed next time.

  In addition, in order to predict that the door will open, it is not always necessary to use the smart ECU. For example, as shown in FIG. 6C, in the reference image acquisition process, for example, via an electronic key or an operation switch. By performing data communication with the body ECU that controls the locking / unlocking of the door according to the command from the occupant, it is determined whether or not the door lock has been released (S610). When released (S610-YES), it is predicted that one of the doors of the vehicle will be opened, the process proceeds to S620, a captured image (seat image) is acquired from the vehicle interior camera 4, and the image is displayed for a certain period of time. Only the reference image may be stored.

  For example, as shown in FIG. 6D, in the reference image acquisition process, for example, when the ignition switch is on, the occupant switches the shift position of the automatic transmission to the parking position, or the parking brake. Is operated to determine whether or not the vehicle has been stopped (S710). If it is determined that the occupant has stopped the vehicle (S710-YES), it is predicted that one of the vehicle doors will be opened, and the process proceeds to S720. Then, the captured image (seat image) may be acquired from the vehicle interior camera 4 and the image may be stored as a reference image for a certain period of time.

  If the release of the door lock or the stop of the vehicle is detected and the reference image is stored in this way, the door can be used not only when the occupant gets on the vehicle but also when the occupant gets out of the vehicle. An image of the passenger compartment can be taken before opening and stored as a reference image.

  In the reference image acquisition process of FIG. 6D, when detecting the stop operation of the vehicle by the occupant, the shift position of the automatic transmission is determined by data communication with the A / TECU that controls the automatic transmission. Is switched to the parking position, or the operation state of the parking brake is determined based on the on / off state of a switch provided on the parking brake or the lighting state of the display lamp indicating the operation state of the parking brake. What is necessary is just to make it detect.

  In this case, the A / TECU or the parking brake operation detection switch corresponds to the stop detection means of the present invention. However, it is not always necessary to detect the parking brake or the shift position of the automatic transmission in order to detect the stop of the vehicle. For example, when the driver operates the braking device of the vehicle and the vehicle speed becomes zero. In addition, it may be determined that the vehicle has stopped.

Further, the reference image acquisition processing shown in FIGS. 6A, 6 </ b> C, and 6 </ b> D may be executed in parallel in the occupant detection ECU 10 or may be executed in combination. You may do it,
Next, in the above-described embodiment, in the seating state determination process, the occupant detection process has been described as being repeatedly executed until a predetermined time has elapsed after all the doors of the vehicle are closed. For example, the ignition switch of the vehicle When the engine is stopped and the door is opened and closed, when all the doors of the vehicle are closed until the ignition switch is turned on (that is, until the engine is started) During this time, the occupant detection process may be executed. The occupant detection process may be repeatedly executed after all the doors of the vehicle are closed until the shift position of the automatic transmission is switched to the drive position and the vehicle starts to travel. .

It is a block diagram showing the structure of the whole passenger | crew detection apparatus of embodiment. It is a flowchart showing the seating state determination process performed in passenger | crew detection ECU. It is a flowchart showing the detail of the object detection process in FIG. It is a flowchart showing the detail of the passenger | crew detection process in FIG. It is explanatory drawing showing the process sequence of the captured image in an object detection process and a passenger | crew detection process. It is a flowchart showing the reference | standard image acquisition process and seating determination process as a modification of embodiment.

Explanation of symbols

  2 ... occupant detection device, 4 ... vehicle interior camera, 6FL, 6FR, 6RL, 6RR ... door switch, 10 ... occupant detection ECU, 12 ... CPU, 14 ... ROM, 16 ... RAM, 18 ... bus, 20 ... image processing unit DESCRIPTION OF SYMBOLS 22 ... Switch input part 24 ... Communication part 26 ... Communication line 30 ... Air-conditioner ECU, 40 ... Audio ECU, 50 ... Airbag ECU, 60 ... Smart ECU.

Claims (5)

Imaging means for imaging the passenger compartment;
Door state detecting means for detecting an open / closed state of a vehicle door;
When it is detected by the door state detection means that the door is opened, or when the door is predicted to open based on the operation state of each part of the vehicle, a captured image of the vehicle interior is captured from the imaging means, Reference image storage means for storing the captured image as a reference image;
When it is detected by the door state detection means that the door is opened and closed, a captured image in the vehicle interior is captured from the imaging means, and the captured image and the reference image stored in the reference image storage means Occupant detection means for detecting occupants in the passenger compartment by comparing
An occupant detection device for a vehicle, comprising: The door state detection means is configured to detect an open / close state for each of a plurality of doors provided in the vehicle,
The occupant detection means identifies a seat position to be detected based on the position of the door detected to be opened and closed by the door state detection means, and includes the captured image and the reference image captured from the imaging means. The vehicle occupant detection device according to claim 1, wherein an occupant at the seat position is detected by comparing images in an area corresponding to the seat position.   The occupant detection means captures a plurality of captured images from the imaging means until a predetermined period elapses after the door state detection means detects that the door is closed, and the captured plurality of captured images. The vehicle occupant detection device according to claim 1, wherein the vehicle occupant is detected based on a change in the comparison result. A receiving means for receiving a transmission radio wave wirelessly transmitted from an electronic key for operating the own vehicle;
The reference image storage means predicts that the door will open when the receiving means receives a transmission radio wave from the electronic key, captures a captured image of the vehicle interior from the imaging means, and uses the reference image as the reference image. The vehicle occupant detection device according to any one of claims 1 to 3, wherein the vehicle occupant detection device is stored. Comprising stop detection means for detecting stop of the vehicle,
The reference image storage means predicts that the door will open when the stop of the vehicle is detected by the stop detection means, captures a captured image of the vehicle interior from the imaging means, and stores it as the reference image The vehicle occupant detection device according to any one of claims 1 to 4, wherein:

Images