JP2012044404A - Cabin monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a cabin monitoring device capable of recognizing an occupant's face at a high speed and with high precision without influencing the occupant and/or operation environment.SOLUTION: The cabin monitoring device, monitoring an occupant in a cabin of an own vehicle, includes: occupant determination means that determines whether or not the occupant exists in the cabin; after-boarding image acquisition means that acquires an after-boarding image which is an image of the inside of the cabin where the occupant exists when the occupant determination means determines that the occupant exists in the cabin; differential image acquisition means that acquires a differential image of a difference between a before-boarding image which is an image of the inside of the cabin where no occupant exists and the after-boarding image; detection target region setting means that sets a detection target region of a detection target of the occupant to be monitored based on the differential image; face detection means that detects the occupant's face from the detection target region set by the detection target region setting means; and monitoring means that monitors the face detected by the face detection means.

Description

  The present invention relates to a vehicle interior monitoring device that monitors the interior of a vehicle, and more specifically, a vehicle interior that monitors passengers in the vehicle based on an image captured by a camera or the like mounted on the vehicle. It relates to a monitoring device.

  2. Description of the Related Art Conventionally, a vehicle interior monitoring device has been developed that monitors passengers in a vehicle interior using a camera or the like mounted on the vehicle. More specifically, in the vehicle interior monitoring device, edge information is acquired from an image captured by a camera or the like by comparing the luminance (shading) of each pixel, and a candidate for a face-like shape is obtained based on the edge information. To detect. Then, the vehicle interior monitoring device recognizes the face of the passenger in the vehicle interior from the face-like shape candidates and monitors the position and state of the passenger's eyes to prevent so-called dozing and side-by-side driving. Therefore, such a vehicle interior monitoring device is required to recognize the occupant's face at high speed and accurately.

  For example, Patent Document 1 discloses a technique for automatically or manually selecting a face area from an image captured by a camera or the like, and further narrowing down a necessary area including a face from the selected face area. . In other words, in Patent Document 1, a face area and a necessary area are narrowed down step by step from an image captured by a camera or the like, and a background other than the face is excluded to accurately recognize the face.

JP 2005-352560 A JP 2009-176005 A

  However, in the conventional vehicle interior monitoring device, the face-like shape candidate is detected from the entire range of the image captured by the camera or the like, and the face of the passenger-like shape is recognized from the face-like shape candidate. Regarding the face recognition processing, there is a problem that the initial face detection processing takes a long time. In addition, since a plurality of face-like candidates are detected based on the edge information, there is a problem that, for example, a headrest similar to the face shape is erroneously recognized as a face.

  Further, the seating position in the seat is different for each occupant, or the seat position and / or the steering position is adjusted. In Patent Document 1 described above, since a face area is automatically or manually selected from an image captured by a camera or the like, the face area cannot be selected stably and accurately at an initial stage. There is a problem that the occupant's face cannot be recognized accurately.

  SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a vehicle interior monitoring device that can recognize a passenger's face at high speed and accurately without affecting the passenger and / or driving environment.

In order to achieve the above object, a vehicle interior monitoring device according to the present invention is a vehicle interior monitoring device that monitors an occupant in the vehicle interior of the host vehicle, and determines whether the occupant is present in the vehicle interior. And an after-boarding image acquisition unit that acquires an after-boarding image that is an image of the passenger compartment in which the occupant is present, and an image of the passenger compartment in which no occupant exists. A difference image acquisition means for acquiring a difference image that is a difference between a pre-boarding image and a post-boarding image, and a detection target region setting for setting a detection target region that is a target for detecting an occupant to be monitored based on the difference image Means, a face detection means for detecting the face of the occupant from the detection target area set by the detection target area setting means, and a monitoring means for monitoring the face detected by the face detection means.
With this configuration, the difference image acquisition unit acquires a difference image that is a difference between the pre-boarding image and the post-boarding image, and the detection target region setting unit is a target for detecting a passenger to be monitored based on the difference image. The detection target area is set, and the face detection means detects the occupant's face from the detection target area set by the detection target area setting means, so that the occupant can be detected quickly and accurately without affecting the occupant and / or driving environment. Can be recognized.

Preferably, the vehicle further includes pre-boarding image acquisition means for capturing a pre-boarding image by photographing the passenger compartment immediately before the occupant gets on the vehicle.
With this configuration, the pre-boarding image acquisition unit captures the passenger compartment immediately before boarding the occupant and acquires the pre-boarding image, so that the pre-boarding image and the post-boarding image approximate the portion other than the occupant, and the detection target The region setting means can easily set a detection target region that is a target for detecting the occupant to be monitored based on the difference image.

Preferably, the vehicle further includes door unlocking determining means for determining whether or not the door of the own vehicle is unlocked, and when the door unlocking determining means determines that the door of the own vehicle is unlocked, The acquisition means is characterized in that the vehicle interior is photographed and a pre-boarding image is acquired.
With this configuration, the pre-boarding image acquisition unit captures the interior of the vehicle and acquires the pre-boarding image when the door unlocking determination unit determines that the door of the host vehicle is unlocked. The pre-boarding image and the post-boarding image are further approximated with respect to each other, and the detection target region setting means can easily set the detection target region that is a target for detecting the occupant to be monitored based on the difference image.

Furthermore, a preferable after-boarding image acquisition means is characterized in that the passenger compartment is photographed at the timing when the occupant is seated in the driver's seat.
With this configuration, the after-boarding image acquisition means captures the interior of the vehicle at the timing when the occupant is seated on the driver's seat, and thus the captured after-boarding image approximates an image in a state in which the occupant is driving. In other words, the occupant's face detected from the detection target area set by the detection target area setting means approximates the face of the occupant in the driving state, and the monitoring means can easily monitor the occupant's face. Become.

Furthermore, a preferable after-boarding image acquisition means is characterized in that the interior of the vehicle is photographed at a timing at which the occupant is ready to drive.
With this configuration, the after-boarding image acquisition unit captures the interior of the vehicle at a timing when the occupant is ready to drive, and thus the captured after-boarding image is further approximated to an image in a state in which the occupant is driving. In other words, the occupant's face detected from the detection target area set by the detection target area setting means further approximates the face of the occupant in the driving state, and the monitoring means monitors the occupant's face. It becomes easy.

Furthermore, a preferable after-boarding image acquisition means is characterized in that the interior of the vehicle is photographed at a timing when the shift range is set to the drive range.
With this configuration, the after-boarding image acquisition means captures the interior of the vehicle at the timing when the shift range is set to the drive range, and thus the captured after-boarding image is further approximated to an image in a state where the occupant is driving. To do. In other words, the occupant's face detected from the detection target area set by the detection target area setting means further approximates the face of the occupant in the driving state, and the monitoring means monitors the occupant's face. It becomes easy.

Preferably, the movement determination means for determining whether or not the driver seat and / or the steering column has moved from a preset position, and the movement determination means has determined that the driver seat and / or the steering column has moved. In this case, the travel amount calculating means for calculating the amount of movement of the driver seat and / or the steering column from the preset position, and the boarding for correcting the pre-boarding image based on the movement amount calculated by the movement amount calculating means. The image processing apparatus further includes a front image correction unit, and the difference image acquisition unit acquires a difference image that is a difference between the pre-boarding image corrected by the pre-boarding image correction unit and the post-boarding image.
With this configuration, the movement amount calculation means calculates the movement amount of the driver seat and / or the steering column from a preset position, and the pre-boarding image correction means calculates the movement amount calculated by the movement amount calculation means. Based on this, the pre-boarding image is corrected. Thus, even when the driver's seat position and / or the steering position are adjusted by the occupant, the corrected pre-boarding image and post-boarding image are approximated for portions other than the occupant, and the detection target region setting means Based on the difference image, it becomes easy to set a detection target region that is a target for detecting a passenger to be monitored.

Preferably, the movement determination means for determining whether or not the driver seat and / or the steering column has moved from a preset position, and the movement determination means has determined that the driver seat and / or the steering column has moved. In this case, the face detection means detects the occupant's face from the entire range of the post-boarding image.
With this configuration, as described above, an enormous amount of processing is performed by calculating the amount of movement of the driver seat and / or steering column from a preset position and correcting the pre-boarding image based on the calculated amount of movement. When time is spent, the face detection means detects the occupant's face from the entire range of the post-boarding image and does not need to perform useless processing.

In order to achieve the above object, a vehicle interior monitoring method of the present invention is a vehicle interior monitoring method executed by a vehicle interior monitoring device that monitors an occupant in the vehicle interior of the host vehicle, and whether the occupant is present in the vehicle interior. An occupant determination step for determining whether or not, an occupant determination step for acquiring an after-boarding image that is an image of the interior of the vehicle in which the occupant is present, A difference image acquisition step for acquiring a difference image that is a difference between a pre-boarding image and a post-boarding image that are images of a vehicle interior in which no vehicle exists, and a detection target region that is a target for detecting a passenger to be monitored based on the difference image A detection target region setting step for setting the detection target, a face detection step for detecting the occupant's face from the detection target region set in the detection target region setting step, and a face detected in the face detection step And a monitoring step.
With this configuration, in the difference image acquisition step, a difference image that is a difference between the pre-boarding image and the post-boarding image is acquired, and in the detection target region setting step, an occupant to be monitored is detected based on the difference image. Since the detection target area is set and the face detection step detects the occupant's face from the detection target area set in the detection target area setting step, the occupant is quickly and accurately affected without affecting the occupant and / or driving environment. Can be recognized.

  Moreover, in order to achieve the said objective, each process which each structure of the vehicle interior monitoring apparatus of this invention mentioned above can be considered as a vehicle interior monitoring method which gives a series of processing procedures. This method is provided in the form of a program for causing a computer to execute a series of processing procedures. This program may be installed in a computer in a form recorded on a computer-readable recording medium.

  As described above, according to the vehicle interior monitoring device of the present invention, the occupant's face can be recognized quickly and accurately without affecting the occupant and / or the driving environment.

The block diagram which shows the hardware constitutions of the vehicle interior monitoring apparatus 100 which concerns on the 1st Embodiment of this invention. 1 is a functional block diagram showing a schematic configuration of a vehicle interior monitoring apparatus 100 according to the first embodiment of the present invention. The flowchart which shows the flow of a process of the vehicle interior monitoring method 300 which the vehicle interior monitoring apparatus 100 which concerns on the 1st Embodiment of this invention performs. The figure which shows the image 400 before boarding which is an image of the vehicle interior in which a passenger | crew does not exist. The figure which shows the after-boarding image 500 which is an image of the passenger compartment where a passenger | crew exists. The figure which shows the difference image 600 which is the difference of the image 400 before boarding, and the image 500 after boarding The figure which shows the detection target area | region 700 used as the object which detects a passenger | crew from the difference image 600 shown in FIG. The figure which shows a mode that a passenger | crew's face is detected from the detection object area | region 700 shown in FIG. The block diagram which shows the hardware constitutions of the vehicle interior monitoring apparatus 900 which concerns on the 2nd Embodiment of this invention. Functional block diagram which shows schematic structure of the vehicle interior monitoring apparatus 900 which concerns on the 2nd Embodiment of this invention. The flowchart which shows the flow of a process of the vehicle interior monitoring method 1100 which the vehicle interior monitoring apparatus 900 which concerns on the 2nd Embodiment of this invention performs.

Hereinafter, each embodiment of the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a block diagram showing a hardware configuration of a vehicle interior monitoring apparatus 100 according to the first embodiment of the present invention. In FIG. 1, the vehicle interior monitoring apparatus 100 includes a driver monitor camera 11, a door opening / closing detection sensor 12, an occupant detection sensor 13, and a driver monitor ECU (Electronic Control Unit) 14.

  The driver monitor camera 11 is installed in a steering column, for example, and photographs the interior of the vehicle. More specifically, the driver monitor camera 11 captures the interior of the vehicle based on a capturing timing control signal from the driver monitor ECU 14 and transmits the captured image data to the driver monitor ECU 14.

  The door open / close detection sensor 12 is, for example, a keyless entry system, and transmits door unlock detection information to the driver monitor ECU 14 when detecting that the door is unlocked.

  The occupant detection sensor 13 is, for example, a sensor that detects an occupant by a load applied to the seat. When the occupant is detected, the occupant detection sensor 13 transmits driver detection information to the driver monitor ECU 14.

  The driver monitor ECU 14 transmits an imaging timing control signal to the driver monitor camera 11 based on the door unlock detection information transmitted from the door opening / closing detection sensor 12 and the driver detection information transmitted from the occupant detection sensor 13. Further, the driver monitor ECU 14 processes the image captured by the driver monitor camera 11 to recognize the occupant's face in the passenger compartment, and monitors the recognized occupant's face. Details of the processing in the driver monitor ECU 14 will be described later.

  FIG. 2 is a functional block diagram showing a schematic configuration of the vehicle interior monitoring apparatus 100 according to the first embodiment of the present invention. In FIG. 2, the vehicle interior monitoring apparatus 100 includes a door unlock determination unit 101, a pre-boarding image acquisition unit 102, an occupant determination unit 103, an after-boarding image acquisition unit 104, a difference image acquisition unit 105, and a detection target. An area setting unit 106, a face detection unit 107, and a monitoring unit 108 are provided.

  The door unlock determination means 101 determines whether or not the door of the host vehicle has been unlocked. Specifically, the door unlocking determination unit 101 is, for example, a keyless entry system, in which the door is unlocked by communication between a remote key operated by an occupant and an ECU mounted on the host vehicle. Detect that.

  The pre-boarding image acquisition unit 102 acquires a pre-boarding image that is an image in the passenger compartment where no passenger is present. Specifically, for example, the pre-boarding image acquisition unit 102 is a driver monitor camera installed on the steering column, and immediately before the occupant gets into the vehicle, the vehicle interior where no occupant is present is photographed to obtain the pre-boarding image. What is necessary is just to acquire an image. In the present embodiment, the time immediately before the occupant gets into the own vehicle is the timing when the door of the own vehicle is unlocked.

  The occupant determination means 103 determines whether an occupant is present in the passenger compartment. Specifically, the occupant determination unit 103 detects that the occupant has boarded, and for example, a occupant load applied to the seat may be detected by a sensor.

  The after-boarding image acquisition unit 104 captures the interior of the vehicle in which the occupant is present, and acquires an after-boarding image that is an image of the interior of the vehicle in which the occupant is present. Specifically, the after-boarding image acquisition means 104 is, for example, a driver monitor camera installed on the steering column, and captures the after-boarding image by photographing the passenger compartment at the timing when the occupant is seated on the driver's seat.

  The difference image acquisition unit 105 acquires a difference image that is a difference between the pre-boarding image acquired by the pre-boarding image acquisition unit 102 and the post-boarding image acquired by the post-boarding image acquisition unit 104.

  The detection target area setting unit 106 sets a detection target area based on the difference image acquired by the difference image acquisition unit 105. The detection target area is an area to be searched for the occupant's face to be monitored from the post-ride image.

  The face detection means 107 detects the occupant's face from the detection target area set by the detection target area setting means 106.

  The monitoring unit 108 monitors the face detected by the face detection unit 107. Specifically, the monitoring means 108 monitors the occupant's face with a driver monitor camera installed on the steering column.

  In addition, the door unlocking determination unit 101 to the monitoring unit 108 described above are controlled by a control unit (not shown) which is a driver monitor ECU, for example.

  Next, the flow of processing will be described in detail for the vehicle interior monitoring method executed by the vehicle interior monitoring device 100 according to the first embodiment of the present invention. FIG. 3 is a flowchart showing a process flow of the vehicle interior monitoring method 300 executed by the vehicle interior monitoring device 100 according to the first embodiment of the present invention. 3, the vehicle interior monitoring method 300 includes a door unlocking determination step S301, a pre-boarding image acquisition step S302, an occupant determination step S303, a post-boarding image acquisition step S304, a difference image acquisition step S305, and a detection target. An area setting step S306, a face detection step S307, and a monitoring step S308 are included.

  In door unlock determination step S301, the door unlock determination means 101 determines whether or not the door of the host vehicle has been unlocked. Specifically, the door unlocking determination means 101 determines whether or not the door is unlocked by communication between a remote key operated by the occupant and the ECU mounted on the host vehicle in the keyless entry system. To do. When it is determined that the door of the host vehicle is unlocked, the door unlock determination unit 101 notifies the pre-boarding image acquisition unit 102 of the door unlock detection information, and proceeds to the processing before the boarding image acquisition step S302 (door). Unlocking determination step S301 (Yes). On the other hand, when the door unlocking determination means 101 determines that the door of the host vehicle is not unlocked, the door unlocking determination step S301 is repeated (No in the door unlocking determination step S301). In other words, the door unlock determination means 101 repeats the door unlock determination step S301 and monitors whether or not the door of the vehicle is unlocked.

  In the pre-boarding image acquisition step S302, the pre-boarding image acquisition unit 102 acquires a pre-boarding image that is an image of the passenger compartment where no passenger is present. FIG. 4 is a diagram illustrating a pre-boarding image 400 that is an image of a passenger compartment in which no passenger is present. The pre-boarding image acquisition unit 102 acquires the pre-boarding image 400 by photographing the interior of the vehicle with a driver monitor camera installed in the steering column based on the door unlocking detection information from the door unlocking determination unit 101.

  In occupant determination step S303, the occupant determination means 103 determines whether an occupant is present in the passenger compartment. Specifically, the occupant determination means 103 determines whether or not the occupant is seated by the load applied to the seat after it is determined in the door unlock determination step S301 that the door of the host vehicle is unlocked. When it is determined that the occupant is seated, the occupant determination unit 103 notifies the post-boarding image acquisition unit 104 of the driver detection information and proceeds to the processing after the boarding image acquisition step S304 (Yes in the occupant determination step S303). On the other hand, when it is determined that the occupant is not seated, the occupant determination unit 103 repeats the process of the occupant determination step S303 (No in the occupant determination step S303). In other words, the occupant determination unit 103 repeats the occupant determination step S303 and monitors whether or not the occupant is present in the passenger compartment.

  In the after-boarding image acquisition step S304, the after-boarding image acquisition unit 104 captures the interior of the vehicle in which the occupant is present and acquires an after-boarding image that is an image of the interior of the vehicle in which the occupant is present. FIG. 5 is a diagram illustrating an after-boarding image 500 that is an image of the passenger compartment where a passenger is present. Based on the driver detection information from the occupant determination unit 103, the after-boarding image acquisition unit 104 captures the interior of the vehicle with a driver monitor camera installed on the steering column, and acquires the after-boarding image 500.

  In the difference image acquisition step S305, the difference image acquisition means 105 is a difference that is a difference between the pre-boarding image 400 acquired in the pre-boarding image acquisition step S302 and the post-boarding image 500 acquired in the post-boarding image acquisition step S304. Get an image. Specifically, the difference image is generated by comparing the luminance value in each pixel for the pre-boarding image 400 and the post-boarding image 500. FIG. 6 is a diagram illustrating a difference image 600 that is a difference between the pre-boarding image 400 and the post-boarding image 500. For example, the pre-boarding image 400 shown in FIG. 4 and the post-boarding image 500 shown in FIG. 5 have the same background (rear seat, driver's seat, headrest, etc.) in the passenger compartment. The point is that the crew is seated in Therefore, as shown in FIG. 6, a differential image 600 of only the occupant seated in the seat is generated.

  In detection target region setting step S306, detection target region setting means 106 sets a detection target region based on difference image 600 acquired in difference image acquisition step S305. FIG. 7 is a diagram illustrating a detection target region 700 that is a target for detecting an occupant from the difference image 600 illustrated in FIG. 6. In FIG. 7, the detection target area setting unit 106 sets a detection target area 700 corresponding to the occupant range in the difference image 600. Here, in the difference image 600, since the background in the vehicle interior (for example, the rear seat, the driver's seat, the headrest, etc.) is deleted, the detection target area setting means 106 is photographed by a driver monitor camera installed on the steering column. The detection target area 700 can be set in a narrowed range as compared with the entire range in the vehicle interior.

  In face detection step S307, the face detection means 107 detects the occupant's face from the detection target area 700 set in the detection target area setting step S306. FIG. 8 is a diagram illustrating a state in which the occupant's face is detected from the detection target region 700 illustrated in FIG. 7. In FIG. 8, the face detection means 107 first recognizes the outline of the occupant's face from the detection target area 700 and sequentially detects the detailed portions of the face (eyes, nose, mouth, etc.). As a method for detecting a face, for example, there is a method in which a search range is set for each feature point of a face and a detailed portion is accurately detected (see Patent Document 2).

  In the monitoring step S308, the monitoring unit 108 monitors the face detected in the face detection step S307. More specifically, the monitoring means 108 monitors the occupant's face with a driver monitor camera installed on the steering column. For example, when the face is not facing the front and when the heel is closed, the driver asides and sleeps. It is determined that the vehicle is driving and alerts the occupant.

  As described above, according to the vehicle interior monitoring apparatus 100 and the vehicle interior monitoring method 300 according to the first embodiment of the present invention, the difference image 600 that is the difference between the pre-boarding image 400 and the post-boarding image 500 is acquired. Based on the difference image 600, a narrowed detection target area 700 is set as compared with the entire range of the passenger compartment photographed by the driver monitor camera installed in the steering column. And since a passenger | crew's face is detected from the said detection object area | region 700, a passenger | crew's face can be recognized rapidly and correctly, without affecting a passenger | crew and / or driving environment.

  In particular, in the present embodiment, in both scenes where the pre-boarding image 400 and the post-boarding image 500 are acquired, an environment (for example, brightness of the vehicle interior) in which the vehicle interior is photographed by a driver monitor camera installed on the steering column is set. Approximate. Specifically, with respect to the pre-boarding image 400, by photographing the interior of the vehicle at the timing when the vehicle door is unlocked just before the vehicle rides on the vehicle, the environment other than the vehicle occupant seated on the seat is obtained. It is approximated to the scene where the post-boarding image 500 is acquired. Thereby, the difference image 600 of only the passenger seated on the seat can be generated, and the detection target area 700 can be set based on the difference image 600.

  In the present embodiment, the timing for photographing the passenger compartment where no occupant is present is the timing at which the door of the host vehicle is unlocked, but the present invention is not limited to this. For example, it may be the timing when the occupant pulls the door knob of the own vehicle and the timing when the occupant actually opens the door of the own vehicle.

  Further, the pre-boarding image 400 may be acquired by photographing the vehicle interior with a driver monitor camera installed on the steering column immediately before the occupant gets on each time, but is recorded in advance by a recording means such as a memory. It doesn't matter. However, as described above, it is preferable that the vehicle interior environment is approximated in both scenes in which the pre-boarding image 400 and the post-boarding image 500 are acquired, so a plurality of pre-boarding images corresponding to environmental conditions are recorded in advance. It is more preferable. For example, assuming a scene that affects the interior environment (brightness of the interior), the time zone (morning, noon, night), weather (clear, cloudy, rain), season (spring, summer, autumn, A plurality of pre-boarding images may be recorded in advance at a stop position (in winter, in the shade, in the shade, on a road with lights, on a mountain road without lights), or a combination thereof. Thereby, the difference image 600 of only the passenger seated in the seat is generated by selecting the pre-boarding image that approximates the vehicle interior environment (such as the brightness of the vehicle interior) in the scene where the post-boarding image 500 is captured. The detection target area 700 can be set based on the difference image 600.

  In the present embodiment, the timing of photographing the passenger compartment where the occupant is present is the timing when the occupant is seated, but the present invention is not limited to this. For example, when the shift range is set to the drive range, when the occupant grasps the steering wheel, when the engine is started, and when the vehicle starts to move, it is possible to detect the situation that the occupant is ready to drive. Anything is fine.

<Second Embodiment>
FIG. 9 is a block diagram showing a hardware configuration of a vehicle interior monitoring apparatus 900 according to the second embodiment of the present invention. In FIG. 9, the vehicle interior monitoring apparatus 900 includes a driver monitor camera 11, a door opening / closing detection sensor 12, an occupant detection sensor 13, a driver monitor ECU 14, a seat adjustment mechanism 91, and a steering column adjustment mechanism 92. In FIG. 9, the same components as those in the vehicle interior monitoring apparatus 100 according to the first embodiment of the present invention shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. In the present embodiment, a configuration different from the first embodiment of the present invention will be mainly described.

  The seat adjustment mechanism 91 adjusts the driver seat position so that, for example, after the occupant is seated, the occupant can easily drive, the seat position information indicating the adjusted driver seat position is sent to the driver monitor ECU 14. Send.

  For example, the steering column adjustment mechanism 92 adjusts the handle position so that the occupant can easily drive after the occupant is seated. Thereby, the position of the driver monitor camera installed in the steering column is also moved. Therefore, the steering column adjustment mechanism 92 transmits tilt telescopic position information indicating the adjusted handle position (the moved driver monitor camera position) to the driver monitor ECU 14.

  The driver monitor ECU 14 obtains a pre-ride image by taking a picture of the interior of the vehicle where no occupant is present with the driver monitor camera 11 immediately before the occupant gets into the vehicle, and then the seat position information transmitted from the seat adjustment mechanism 91. Then, based on the tilt telescopic position information transmitted from the steering column adjustment mechanism 92, the pre-boarding image is corrected. Details of the processing in the driver monitor ECU 14 will be described later.

  FIG. 10 is a functional block diagram showing a schematic configuration of a vehicle interior monitoring apparatus 900 according to the second embodiment of the present invention. In FIG. 10, the vehicle interior monitoring apparatus 900 includes a door unlocking determination unit 101, a pre-boarding image acquisition unit 102, an occupant determination unit 103, an after-boarding image acquisition unit 104, a movement determination unit 901, and a movement amount calculation. Means 902, pre-boarding image correction means 903, difference image acquisition means 904, detection target area setting means 106, face detection means 107, and monitoring means 108 are provided. In FIG. 10, the same components as those in the vehicle interior monitoring apparatus 100 according to the first embodiment of the present invention shown in FIG. In the present embodiment, a configuration different from the first embodiment of the present invention will be mainly described.

  The movement determination unit 901 determines whether or not the driver seat and / or the steering column has moved from a preset position. Specifically, for example, the movement determination unit 901 may recognize that the driver seat and / or the steering column has moved from a preset position in cooperation with the seat adjustment mechanism and / or the steering column adjustment mechanism. .

  When the movement determination unit 901 determines that the driver seat and / or the steering column has moved, the movement amount calculator 902 calculates the amount of movement of the driver seat and / or the steering column from a preset position. Specifically, the movement amount calculation unit 902 obtains seat position information from the seat adjustment mechanism, and calculates how much the driver's seat has moved up, down, left, right, and back from a preset position. Further, the movement amount calculation means 902 obtains tilt telescopic position information from the steering column adjustment mechanism, and indicates how much the driver monitor camera installed on the steering column has moved up and down, left and right and back and forth from a preset position. calculate.

  The pre-boarding image correction unit 903 corrects the pre-boarding image based on the movement amount calculated by the movement amount calculation unit 902. For example, when the driver's seat has moved from the preset position to the front of the vehicle, the pre-boarding image correction unit 903 enlarges the driver's seat image in the pre-boarding image based on the amount of movement. Correct the pre-boarding image. Also, when the driver monitor camera has moved upward from a preset position, the pre-boarding image correction means 903 corrects the pre-boarding image by shifting the entire pre-boarding image downward based on the amount of movement. To do.

  The difference image acquisition unit 904 acquires a difference image that is a difference between the pre-boarding image corrected by the pre-boarding image correction unit 903 and the post-boarding image acquired by the post-boarding image acquisition unit 104.

  Note that the movement determination unit 901 to the difference image acquisition unit 904 described above are controlled by a control unit (not shown) that is a driver monitor ECU, for example.

  Next, the flow of processing will be described in detail for the vehicle interior monitoring method executed by the vehicle interior monitoring apparatus 900 according to the second embodiment of the present invention. FIG. 11 is a flowchart showing a process flow of the vehicle interior monitoring method 1100 executed by the vehicle interior monitoring apparatus 900 according to the second embodiment of the present invention. 11, the vehicle interior monitoring method 1100 includes a door unlocking determination step S301, a pre-boarding image acquisition step S302, an occupant determination step S303, a movement determination step S1101, a movement amount calculation step S1102, and a pre-boarding image correction. It includes step S1103, post-boarding image acquisition step S1104, difference image acquisition step S1105, detection target region setting step S306, face detection step S307, and monitoring step S308. In FIG. 11, the same components as those in the vehicle interior monitoring method 300 according to the first embodiment of the present invention shown in FIG. In the present embodiment, a configuration different from the first embodiment of the present invention will be mainly described.

  In the occupant determination step S303, when the occupant determination unit 103 determines that the occupant is seated, the occupant determination unit 103 notifies the post-boarding image acquisition unit 104 of the driver detection information and proceeds to the process of the movement determination step S1101 (in the occupant determination step S303). Yes). On the other hand, when it is determined that the occupant is not seated, the occupant determination unit 103 repeats the process of the occupant determination step S303 (No in the occupant determination step S303).

  In movement determination step S1101, the movement determination means 901 determines whether or not the driver's seat and / or the steering column has moved from a preset position. If the movement determination unit 901 determines that the driver seat and / or the steering column has moved from a preset position, the movement determination unit 901 proceeds to the process of the movement amount calculation step S1102 (Yes in the movement determination step S1101). On the other hand, if the movement determination means 901 determines that the driver's seat and / or the steering column have not moved from the preset position, the movement determination means 901 proceeds to the processing after the boarding image acquisition step S1104 (No in movement determination step S1101).

  Note that the movement determination means 901 does not determine that the driver's seat and / or the steering column have moved from a preset position until a predetermined time after determining that the occupant is seated in the occupant determination step S303. What is necessary is just to determine with a driver seat and / or a steering column not moving from the preset position. Further, the movement determination means 901 is a position where the driver's seat and / or the steering column are set in advance from the determination that the occupant is seated in the occupant determination step S303 until the engine starts or the vehicle starts to move. If it is not determined that the vehicle has moved, the driver's seat and / or the steering column may be determined not to move from a preset position.

  In the movement amount calculation step S1102, the movement amount calculation unit 902 moves the driver seat and / or the steering column from a preset position when it is determined in the movement determination step S1101 that the driver seat and / or the steering column has moved. The amount of movement performed is calculated. For example, the movement amount calculation means 902 includes the driver seat position (X, Y, Z) = (X1, Y1, Z1) when the pre-boarding image is captured, and the driver seat after movement when the driver seat is adjusted. Based on the position (X, Y, Z) = (X2, Y2, Z2), the movement amount (ΔX, ΔY, ΔZ) = (X2-X1, Y2-Y1, Z2-Z1) of the driver seat is calculated. . Similarly, the movement amount calculation means 902 is based on the position of the driver monitor camera installed on the steering column when the pre-boarding image is taken and the position of the driver monitor camera after movement when the steering column is adjusted. The movement amount of the driver monitor camera is calculated.

  In the pre-boarding image correction step S1103, the pre-boarding image correction unit 903 corrects the pre-boarding image based on the movement amount of the driver seat and / or the movement amount of the driver monitor camera calculated in the movement amount calculation step S1102. More specifically, the pre-boarding image correction unit 903 captures the pre-boarding image captured in the pre-boarding image acquisition step S302, and the post-boarding image capturing in the post-boarding image acquisition step S1104, which will be described later, of the environment other than the passenger sitting on the seat. It corrects so that it may approximate.

  Specifically, for example, the driver seat moves a predetermined distance (ΔZ) from the driver seat position when the pre-boarding image is captured in the pre-boarding image acquisition step S302 to the front of the vehicle (Z-axis positive direction). In that case, the pre-boarding image correction means 903 enlarges the driver seat image in the pre-boarding image according to ΔZ. If the driver's seat has moved a predetermined distance (ΔY) upward (Y-axis positive direction) from the driver's seat position when the pre-boarding image was captured in the pre-boarding image acquisition step S302, The image correction unit 903 shifts the driver seat image of the pre-boarding image upward according to ΔY. As for the method for correcting the pre-boarding image according to the movement amount (ΔX, ΔY, ΔZ) of the driver seat, for example, the enlargement ratio or reduction ratio of the driver seat image, and the correction amount in the vertical and horizontal directions are set in advance. May be.

  Also, for example, if the vehicle has moved a predetermined distance (ΔZ) in the occupant direction (Z-axis negative direction) from the driver monitor camera position when the pre-boarding image is captured in the pre-boarding image acquisition step S302, The image correcting unit 903 enlarges the entire pre-boarding image according to ΔZ. Further, when the driver monitor camera has moved a predetermined distance (ΔY) upward (Y-axis positive direction) from the driver monitor camera position when the pre-boarding image was captured in the pre-boarding image acquisition step S302, The pre-boarding image correction means 903 shifts the entire pre-boarding image downward according to ΔY. As for the correction method for the pre-boarding image according to the movement amount (ΔX, ΔY, ΔZ) of the driver monitor camera, for example, the enlargement ratio or the reduction ratio of the pre-boarding image and the correction amount in the vertical and horizontal directions are set in advance. It may be.

  In the after-boarding image acquisition step S1104, the after-boarding image acquisition unit 104 captures the interior of the vehicle in which the occupant is present and acquires an after-boarding image that is an image of the interior of the vehicle in which the occupant is present. Specifically, the after-boarding image acquisition means 104 is installed in the steering column in response to a movement determination result indicating whether or not the driver's seat and / or the steering column has moved from a preset position in the movement determination step S1101. The driver's monitor camera takes a picture of the interior of the vehicle and obtains an image after boarding. This is because, for example, if the driver monitor camera position is moved after the inside of the vehicle is photographed by the driver monitor camera, the environment other than the occupant seated in the seat (for example, the background, etc.) in the pre-boarding image and the post-boarding image. ) Is different.

  In the difference image acquisition step S1104, when the driver's seat and / or the steering column move from the preset position, the difference image acquisition unit 904 corrects the pre-boarding image and the post-boarding image corrected in the pre-boarding image correction step S1103. A difference image that is a difference from the post-boarding image acquired in acquisition step S1104 is acquired. On the other hand, when the driver's seat and / or the steering column has not moved from the preset position, the difference image acquisition unit 904 obtains the pre-boarding image acquired in the pre-boarding image acquisition step S302 and the post-boarding image acquisition step S1104. The difference image which is a difference with the after-boarding image acquired in (1) is acquired.

  In this way, even when the driver's seat and / or the steering column move from a preset position, the environment other than the occupant seated on the seat (for example, the background) in the pre-boarding image and the post-boarding image Is approximating. Thereby, the difference image generated in the difference image acquisition step S1104 can generate the difference image 600 of only the occupant seated in the seat, as shown in FIG. 6 of the first embodiment of the present invention. it can.

  As described above, according to the vehicle interior monitoring apparatus 900 and the vehicle interior monitoring method 1100 according to the second embodiment of the present invention, the driver's seat and / or the steering column is moved from a preset position. As in the first embodiment of the present invention, the difference image 600 of only the occupant seated in the seat is acquired, and is photographed by the driver monitor camera installed on the steering column based on the difference image 600. A narrowed detection target area 700 is set as compared with the entire range in the vehicle interior. And since a passenger | crew's face is detected from the said detection object area | region 700, a passenger | crew's face can be recognized rapidly and correctly, without affecting a passenger | crew and / or driving environment.

  In the present embodiment, for the pre-boarding image, the vehicle interior is photographed by the driver monitor camera installed on the steering column immediately before the occupant gets on the vehicle, and the amount of movement of the driver seat and / or the driver monitor camera Although the pre-boarding image is corrected based on the movement amount, the corrected pre-boarding image may be recorded in advance by a recording unit such as a memory. For example, assuming a driver's seat after movement and / or a driver monitor camera after movement, a plurality of pre-boarding images corresponding to the assumption are recorded in advance. In addition, assuming scenes that affect the interior environment (brightness of the interior), the time zone (morning, noon, night), weather (clear, cloudy, rain), season (spring, summer, autumn, A plurality of pre-boarding images may be recorded in advance at a stop position (in winter, in the shade, in the shade, on a road with lights, on a mountain road without lights), or a combination thereof. Thereby, the process which correct | amends the image before boarding can be simplified, and a passenger | crew's face can be recognized at higher speed.

  In addition, when the process for correcting the pre-boarding image becomes complicated and it takes time to correct the pre-boarding image, the boarding is performed without generating the difference image based on the pre-boarding image and the post-boarding image. The occupant's face may be detected from the entire range in the vehicle interior in the rear image.

  INDUSTRIAL APPLICABILITY The present invention can be applied to a vehicle interior monitoring device or the like that monitors passengers in a vehicle interior, and is useful for processing that detects a passenger's face at high speed and accurately at an initial stage.

11 Driver Monitor Camera 12 Door Open / Close Detection Sensor 13 Occupant Detection Sensor 14 Driver Monitor ECU
91 Seat adjustment mechanism 92 Steering column adjustment mechanism 100, 900 Car interior monitoring device 101 Door unlocking determination means 102 Pre-boarding image acquisition means 103 Passenger determination means 104 Post-boarding image acquisition means 105, 904 Difference image acquisition means 106 Detection target area setting Means 107 Face detection means 108 Monitoring means 300, 1100 Vehicle interior monitoring method 400 Pre-boarding image 500 Post-boarding image 600 Difference image 700 Detection target area 901 Movement determination means 902 Movement amount calculation means 903 Pre-boarding image correction means S301 Door unlocking determination Step S302 Pre-boarding image acquisition step S303 Passenger determination step S304, S1104 Post-boarding image acquisition step S305, S1105 Difference image acquisition step S306 Detection target area setting step S307 Face detection step S308 Monitoring step S1 01 movement determination step S1102 movement amount calculating step S1103 before boarding an image correction step

Claims (9)

A vehicle interior monitoring device for monitoring passengers in the vehicle interior of the vehicle,
Occupant determination means for determining whether the occupant is present in the passenger compartment;
When the occupant determination means determines that an occupant is present in the passenger compartment, an after-boarding image acquisition means for acquiring an after-boarding image that is an image of the passenger compartment in which the occupant exists;
Difference image acquisition means for acquiring a difference image that is a difference between an image before boarding and an image after boarding that are images of a passenger compartment in which the occupant does not exist;
A detection target area setting means for setting a detection target area to be detected based on the difference image;
Face detecting means for detecting the occupant's face from the detection target area set by the detection target area setting means;
A vehicle interior monitoring device comprising: monitoring means for monitoring the face detected by the face detection means.   The vehicle interior monitoring device according to claim 1, further comprising pre-boarding image acquisition means for capturing the pre-boarding image by photographing the vehicle interior immediately before the occupant gets in the vehicle. Door unlocking determining means for determining whether or not the door of the host vehicle is unlocked;
When it is determined by the door unlocking determination means that the door of the host vehicle is unlocked, the pre-boarding image acquisition means captures the vehicle interior and acquires the pre-boarding image, The vehicle interior monitoring device according to claim 2.   The vehicle interior monitoring device according to any one of claims 1 to 3, wherein the after-boarding image acquisition means images the vehicle interior at a timing when the occupant is seated on a driver's seat.   The vehicle interior monitoring device according to any one of claims 1 to 3, wherein the after-boarding image acquisition means images the vehicle interior at a timing when the occupant is ready to drive.   The vehicle interior monitoring device according to any one of claims 1 to 3, wherein the after-boarding image acquisition means images the vehicle interior at a timing when a shift range is set to a drive range. Movement determination means for determining whether the driver's seat and / or the steering column has moved from a preset position;
A movement amount calculating means for calculating a movement amount by which the driver seat and / or the steering column has moved from a preset position when it is determined by the movement determining means that the driver seat and / or the steering column has moved;
Pre-boarding image correction means for correcting the pre-boarding image based on the movement amount calculated by the movement amount calculation means;
The difference image acquisition unit acquires a difference image that is a difference between the pre-boarding image corrected by the pre-boarding image correction unit and the post-boarding image. The vehicle interior monitoring device described in 1. Movement determination means for determining whether the driver's seat and / or the steering column has moved from a preset position;
The face detection means detects the occupant's face from the entire range of the post-ride image when the movement determination means determines that the driver seat and / or the steering column has moved. The vehicle interior monitoring device according to any one of Items 1 to 6. A vehicle interior monitoring method executed by a vehicle interior monitoring device that monitors passengers in the vehicle interior of the host vehicle,
An occupant determination step of determining whether the occupant is present in the passenger compartment;
When it is determined in the occupant determination step that an occupant is present in the passenger compartment, an after-boarding image acquisition step of acquiring an after-boarding image that is an image of the passenger compartment in which the occupant exists;
A difference image acquisition step of acquiring a difference image that is a difference between an image before boarding and an image after boarding that are images of a passenger compartment in which the occupant does not exist;
Based on the difference image, a detection target area setting step for setting a detection target area that is a target for detecting an occupant to be monitored;
A face detection step of detecting the occupant's face from the detection target region set in the detection target region setting step;
And a monitoring step of monitoring the face detected in the face detection step.

Images