JP7204941B2 - Occupant monitoring device, occupant monitoring system, and occupant monitoring method - Google Patents

Description

The present invention relates to an occupant monitoring device, an occupant monitoring system, and an occupant monitoring method.

Patent Literature 1 describes an occupant monitoring device that captures an image of an occupant sitting in a front passenger seat with a camera housed in a rearview mirror case and monitors the occupant using the captured image. This occupant monitoring device is an image of the front pillar captured by the camera built into the rearview mirror when the rearview mirror is adjusted to an appropriate posture so that the occupant in the driver's seat can easily see the rear. Detect from Then, when the front pillar is detected at a predetermined position in the image, the occupant monitoring device determines that the image of the occupant sitting in the front passenger seat has been captured.

JP-A-2000-264128

However, since there is a certain permissible range for the posture of the rearview mirror that allows the passenger in the driver's seat to visually recognize the rear, the passenger in the passenger's seat is not always shown at a specific position in the image. Therefore, even if the occupant in the image is not the occupant to be monitored, the occupant monitoring device described in Patent Document 1 may erroneously detect the occupant in the image as the occupant to be monitored. was there.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to detect a driver to be monitored from an image captured by a camera built into a rearview mirror whose posture is adjustable . .

An occupant monitoring device according to the present invention includes an image acquisition unit that acquires an image of the interior of a vehicle captured by a camera built in a rearview mirror that can be adjusted in attitude, and a rearview mirror that allows the driver to easily view the rear. a facility detection unit that sets a facility detection area at a predetermined position of the image acquired by the image acquisition unit and detects a rear window from the facility detection area when the attitude is adjusted; When the rear window is detected, a monitoring area setting unit for setting an occupant monitoring area for the image based on the image position of the rear window ; and an occupant monitoring unit that monitors the driver using the image of the occupant monitoring area set by the area setting unit.

According to this invention, the driver to be monitored can be detected from the image taken by the camera built in the rearview mirror whose attitude is adjustable.

1 is a perspective view showing a configuration example of a rearview mirror according to Embodiment 1; FIG. FIG. 2 is a perspective view showing an example of a vehicle in which the room mirror of FIG. 1 is installed; 4 is a diagram showing an example of an image captured by an infrared camera built into the rearview mirror according to Embodiment 1; FIG. 1 is a block diagram showing a configuration example of an occupant monitoring system according to Embodiment 1; FIG. 5 is a flowchart showing an operation example of rearview mirror initial attitude adjustment by the passenger monitoring system according to Embodiment 1; FIG. 6A shows an example of an image in which the equipment detection unit cannot detect the rear window, and FIG. 6B shows an example of an image in which a part of the occupant monitoring area set by the monitoring area setting unit protrudes. 4 is a flow chart showing an operation example of occupant monitoring by the occupant monitoring system according to Embodiment 1; 8A and 8B are diagrams showing a hardware configuration example of the occupant monitoring device according to Embodiment 1. FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, in order to describe the present invention in more detail, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1 is a perspective view showing a configuration example of a rearview mirror 1 according to Embodiment 1. FIG. FIG. 2 is a perspective view showing an example of a vehicle 10 in which the rearview mirror 1 of FIG. 1 is installed. Here, the lateral direction of the vehicle 10 is the X axis, the longitudinal direction is the Y axis, and the vertical direction is the Z axis. The rearview mirror 1 is an in-vehicle device installed on the front window (not shown) of the vehicle 10 or on the ceiling near it, and is used by the driver 11 to visually recognize the rear. As indicated by the arrow in FIG. 2 , the driver 11 checks the rear of the vehicle 10 by looking at the rear window 12 through the rearview mirror 1 .

The rearview mirror 1 includes a mirror case 2 , an arm 3 , an infrared filter 4 , a half mirror 5 , infrared light emitters 6 a and 6 b , an infrared camera 7 and a partition member 8 . An arm 3 is provided on the rear surface of the mirror case 2 . The arm 3 fixes the mirror case 2 to the front window (not shown) or the ceiling near it. Also, this arm 3 changes the posture of the rearview mirror 1 in the X-axis direction and the Z-axis direction. The driver 11 may grasp the rearview mirror 1 by hand and adjust the posture of the rearview mirror 1 . Alternatively, if the arm 3 incorporates an actuator, the driver 11 may operate the actuator to adjust the posture of the rearview mirror 1 .

A half mirror 5 is attached to the front of the mirror case 2 . An infrared filter 4 is arranged on the back surface of the half mirror 5 and inside the mirror case 2 . Infrared light emitters 6 a and 6 b and an infrared camera 7 are arranged on the back surface of the infrared filter 4 and inside the mirror case 2 . In the example of FIG. 1, the infrared light emitting unit 6a and the infrared light emitting unit 6b are arranged on both sides of one infrared camera 7, but only one of the infrared light emitting unit 6a and the infrared light emitting unit 6b is arranged. It may be a configuration that is

The infrared light emitting units 6 a and 6 b irradiate the inside of the vehicle 10 with infrared rays. The infrared camera 7 receives infrared rays reflected in the vehicle 10 and transmitted through the half mirror 5 and the infrared filter 4 among the infrared rays emitted by the infrared light emitting units 6a and 6b, and outputs them as an image. The infrared light emitting section 6a, the infrared light emitting section 6b, and the infrared camera 7 are partitioned by a partition member 8. FIG. Therefore, the infrared light emitted from the infrared light emitting units 6 a and 6 b is reflected by the half mirror 5 and the infrared filter 4 and does not enter the infrared camera 7 .

Infrared light emitters 6 a and 6 b , an infrared camera 7 , and a partition member 8 are arranged on the back side of the infrared filter 4 . Therefore, the infrared light emitting units 6a and 6b, the infrared camera 7, and the partition member 8 cannot be seen from the outside of the rearview mirror 1. FIG. Therefore, the infrared camera 7 for occupant monitoring can be arranged in the vehicle 10 without making the driver 11 and fellow passengers, who are subjects, aware of the presence of the infrared camera 7 and the infrared light emitting units 6a and 6b.

FIG. 3 is a diagram showing an example of an image captured by the infrared camera 7 built into the rearview mirror 1 according to the first embodiment. The image shown in FIG. 3 is an image captured by the infrared camera 7 when the rearview mirror 1 is adjusted to an appropriate posture so that the driver 11 can easily see the rear. This image shows the driver 11 in the driver's seat, the passenger seat, the rear window 12, the left and right rear side windows 14, the left and right rear pillars 16, the roof lining 17, and the like. The imaging range of the infrared camera 7 covers the rear window 12 and the driver's seat where the driver 11, who is monitored by the occupant monitoring device 20, gets on when the posture of the rearview mirror 1 is adjusted for the original purpose. It is assumed that it is set to a range that includes .

FIG. 4 is a block diagram showing a configuration example of the occupant monitoring system 30 according to the first embodiment. The occupant monitoring system 30 includes the rearview mirror 1 and the occupant monitoring device 20 . An infrared camera 7 built in the rearview mirror 1 and a warning device 26 provided in the vehicle 10 are connected to the occupant monitoring device 20 . This warning device 26 has a speaker, a display, or the like, and warns the driver 11 by means of sound, display, or the like. The occupant monitoring device 20 includes an image acquisition section 21 , an equipment detection section 22 , a monitoring area setting section 23 , an occupant monitoring section 24 and a warning section 25 .

The image acquisition unit 21 acquires from the infrared camera 7 an image inside the vehicle 10 captured by the infrared camera 7 built in the rearview mirror 1 whose attitude is adjustable. The image acquisition section 21 outputs the acquired image to the facility detection section 22 and the occupant monitoring section 24 .

The equipment detection unit 22 receives the image from the image acquisition unit 21 and sets the equipment detection area 13 at a predetermined position of this image. Here, when the rearview mirror 1 is adjusted to an appropriate posture so that the driver 11 can easily visually recognize the rear, that is, when the posture of the rearview mirror 1, which is an in-vehicle device, is adjusted for its original purpose. First, the rear window 12 is assumed to be the first facility in the vehicle interior captured by the infrared camera 7 . The facility detection area 13 is an area for detecting the rear window 12, which is the first facility, and is set in advance at a fixed position within the image. The equipment detection unit 22 sets the equipment detection area 13 in the image captured by the infrared camera 7 and detects the rear window 12 from the equipment detection area 13 . If the rear window 12 can be detected from the equipment detection area 13, it means that the driver 11 can visually recognize the rear through the rearview mirror 1. If the rear window 12 cannot be detected from the equipment detection area 13, there is a high possibility that the driver 11 cannot visually recognize the rear through the rearview mirror 1. When the equipment detection unit 22 can detect the rear window 12 from the equipment detection area 13 , the equipment detection unit 22 outputs the image and the image position of the rear window 12 to the monitoring area setting unit 23 . On the other hand, when the image and the rear window 12 cannot be detected from the facility detection area 13, the facility detection section 22 notifies the warning section 25 to that effect.

The monitoring area setting unit 23 receives the image and the image position of the rear window 12 from the equipment detection unit 22 . The monitoring area setting unit 23 sets the occupant monitoring area 15 for the image based on the image position of the rear window 12 notified from the facility detection unit 22 . The monitoring area setting unit 23 holds in advance information regarding the position of the occupant monitoring area 15 based on the position of the rear window 12 . The positional relationship between the rear window 12 and the occupant monitoring area 15 when the occupant to be monitored is the driver 11 is illustrated in FIG. The monitoring area setting unit 23 notifies the crew monitoring unit 24 of the image position of the set crew monitoring area 15 . When part or all of the set occupant monitoring area 15 (hereinafter referred to as "at least part") protrudes from the image, the monitoring area setting unit 23 notifies the warning unit 25 of the fact.

The occupant monitoring unit 24 receives the image from the image acquiring unit 21 and sets the occupant monitoring area 15 notified from the monitoring area setting unit 23 to this image. The occupant monitoring unit 24 monitors the driver 11 using the image of the occupant monitoring area 15 among the images received from the image acquisition unit 21 . For example, the occupant monitoring unit 24 determines the state of the driver 11, such as drowsiness, distraction, or fatigue.

The warning unit 25 instructs the warning device 26 to issue a warning when receiving a notification from the equipment detection unit 22, the monitoring area setting unit 23, or the occupant monitoring unit 24. FIG. The warning device 26 issues a warning when instructed by the warning unit 25 .

Next, the operation of the occupant monitoring system 30 will be described.
FIG. 5 is a flow chart showing an operation example of rearview mirror initial posture adjustment by the occupant monitoring system 30 according to the first embodiment. For example, the occupant monitoring system 30 performs the operation shown in the flowchart of FIG. 5 each time the vehicle 10 starts.

In step ST1, the image acquisition unit 21 acquires from the infrared camera 7 an image captured by the infrared camera 7 while the infrared light emitting units 6a and 6b emit infrared rays.

In step ST<b>2 , the facility detection section 22 sets the facility detection area 13 at a predetermined position of the image acquired by the image acquisition section 21 and detects the rear window 12 from the facility detection area 13 .

When the equipment detection unit 22 can detect the rear window 12 from the equipment detection area 13 (step ST2 "YES"), the monitoring area setting unit 23 sets the image position of the rear window 12 detected by the equipment detection unit 22 as a reference. , the occupant monitoring area 15 is set for the image acquired in step ST1 (step ST3).

If the equipment detection unit 22 cannot detect the rear window 12 from the equipment detection area 13 (“NO” in step ST2), there is a high possibility that the driver 11 cannot visually recognize the rear through the rearview mirror 1 . Therefore, in step ST<b>5 , the equipment detection unit 22 instructs the warning unit 25 to issue a warning to prompt the driver 11 to adjust the posture of the rearview mirror 1 . The warning unit 25 issues a warning from the warning device 26 according to this instruction.

6A shows an example of an image in which the equipment detection unit 22 cannot detect the rear window 12 from the equipment detection area 13. FIG. Since the rearview mirror 1 faces downward, the rear window 12 is not shown in this image. For example, when the rear window 12 is within the facility detection area 13, the facility detection unit 22 detects the rear window 12 from the facility detection area 13, and at least part of the rear window 12 is located within the facility detection area 13. It is assumed that the rear window 12 could not be detected from the equipment detection area 13 when it protruded.

In step ST4, the monitoring area setting section 23 determines whether or not the set occupant monitoring area 15 fits within the image. If the occupant monitoring area 15 fits within the image (“YES” in step ST4), the monitoring area setting section 23 notifies the occupant monitoring section 24 of the image position of the occupant monitoring area 15 .

If at least part of the set occupant monitoring area 15 protrudes from the image (step ST4 "NO"), the monitoring area setting unit 23 is likely to be in a state where the occupant monitoring unit 24 cannot monitor the driver 11 . Therefore, in step ST<b>6 , the monitoring area setting unit 23 instructs the warning unit 25 to issue a warning prompting the driver 11 to adjust the posture of the rearview mirror 1 . The warning unit 25 issues a warning from the warning device 26 according to this instruction.

FIG. 6B shows an example of an image in which part of the occupant monitoring area 15 set by the monitoring area setting unit 23 protrudes. Half of the occupant monitoring area 15 set with reference to the image position of the rear window 12 protrudes from the image. In this example, there is a high possibility that the driver 11 cannot visually recognize the rear through the rearview mirror 1 .

After the warning device 26 issues the warning in step ST5 or step ST6, the occupant monitoring device 20 performs the operations shown in the flowchart of FIG. If the warning device 26 issues a warning in step ST5 or step ST6, it is highly likely that the face of the driver 11 will not fit within the occupant monitoring area 15. Difficult to monitor.

Since the occupant monitoring device 20 urges the driver 11 to adjust the attitude of the rearview mirror 1 before the vehicle 10 starts, the driver 11 forgets to adjust the attitude of the rearview mirror 1 and leaves the vehicle while the vehicle is running. Adjustment of the posture of the mirror 1 can be suppressed. Therefore, the driver 11 can concentrate on driving. Further, for example, when the vehicle 10 is shared by a plurality of people, the rearview mirror 1 can be adjusted to a proper posture for the driver after the change before departure.

FIG. 7 is a flow chart showing an operation example of occupant monitoring by the occupant monitoring system 30 according to the first embodiment. After performing the rearview mirror initial position adjustment operation shown in FIG. 5, the occupant monitoring system 30 starts the operation shown in the flowchart of FIG. The occupant monitoring unit 24 has already received the image position of the occupant monitoring area 15 from the monitoring area setting unit 23 before starting the operation shown in the flowchart of FIG.

In step ST11, the image acquisition unit 21 acquires from the infrared camera 7 an image captured by the infrared camera 7 while the infrared light emitting units 6a and 6b are emitting infrared rays.

In step ST<b>12 , the facility detection section 22 sets the facility detection area 13 at a predetermined position of the image acquired by the image acquisition section 21 and detects the rear window 12 from the facility detection area 13 .

If the facility detection unit 22 cannot detect the rear window 12 from the facility detection area 13 (step ST12 "NO"), there is a possibility that the driver 11 readjusted the posture of the rearview mirror 1 while the vehicle 10 was running. . Therefore, in step ST<b>16 , the equipment detection unit 22 instructs the warning unit 25 to issue a warning prompting the driver 11 to adjust the posture of the rearview mirror 1 . The warning unit 25 issues a warning from the warning device 26 according to this instruction.

When the equipment detection unit 22 can detect the rear window 12 from the equipment detection area 13 (“YES” in step ST12), the image position of the detected rear window 12 is the image position of the rear window 12 detected in the previous step ST12. (step ST13). It should be noted that in the initial step ST13, the facility detection unit 22 determines whether or not the image position of the rear window 12 detected in the initial step ST13 has changed from the image position of the rear window 12 detected in step ST2 of FIG. Judge.

If the image position of the rear window 12 has changed ("YES" in step ST13), there is a possibility that the driver 11 has readjusted the posture of the rearview mirror 1 while the vehicle 10 is running. Therefore, in step ST<b>17 , the equipment detection section 22 notifies the monitoring area setting section 23 to reset the occupant monitoring area 15 . In accordance with this notification, the monitoring area setting unit 23 resets the occupant monitoring area 15 for the image acquired in step ST11 based on the image position of the rear window 12 detected in step ST12. Then, the monitoring area setting unit 23 notifies the crew monitoring unit 24 of the reset image position of the crew monitoring area 15 .

If the image position of the rear window 12 has not changed (step ST13 "NO"), the occupant monitoring unit 24 monitors the driver 11 using the image of the occupant monitoring area 15 among the images acquired in step ST11. (step ST14). Even when the posture of the rearview mirror 1 with the infrared camera 7 built therein is changed, the position of the occupant monitoring area 15 set in the image picked up by the infrared camera 7 is changed according to the posture of the rearview mirror 1.例文帳に追加Therefore, the driver 11 can be detected from the image even if the infrared camera 7 for passenger monitoring is placed inside the rearview mirror 1 whose attitude changes.

If the abnormality of the driver 11 is recognized (step ST15 “YES”), the occupant monitoring section 24 instructs the warning section 25 to issue a warning to the driver 11 . The warning unit 25 issues a warning from the warning device 26 according to this instruction (step ST18). On the other hand, if the occupant monitoring unit 24 does not recognize the driver's abnormality (step ST15 "NO"), the occupant monitoring device 20 performs the operation shown in the flowchart of FIG.

Finally, the hardware configuration of the occupant monitoring device 20 according to Embodiment 1 will be described.
8A and 8B are diagrams showing a hardware configuration example of the occupant monitoring device 20 according to Embodiment 1. FIG. The functions of the image acquisition unit 21, the equipment detection unit 22, the monitoring area setting unit 23, the occupant monitoring unit 24, and the warning unit 25 in the occupant monitoring device 20 are realized by a processing circuit. That is, the occupant monitoring device 20 has a processing circuit for realizing the above functions. The processing circuit may be the processing circuit 100 as dedicated hardware, or the processor 101 that executes a program stored in the memory 102 .

As shown in FIG. 8A, when the processing circuit is dedicated hardware, the processing circuit 100 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an Application Specific Integrated Circuit (ASIC). ), FPGA (Field Programmable Gate Array), or a combination thereof. The functions of the image acquisition unit 21, the facility detection unit 22, the monitoring area setting unit 23, the occupant monitoring unit 24, and the warning unit 25 may be realized by a plurality of processing circuits 100, or the functions of each unit may be integrated into one process. It may be implemented in circuit 100 .

As shown in FIG. 8B, when the processing circuit is the processor 101, the functions of the image acquisition unit 21, the equipment detection unit 22, the monitoring area setting unit 23, the occupant monitoring unit 24, and the warning unit 25 are software, firmware, Alternatively, it is implemented by a combination of software and firmware. Software or firmware is written as a program and stored in memory 102 . The processor 101 reads out and executes programs stored in the memory 102 to achieve the functions of each unit. That is, the occupant monitoring system 20 includes a memory 102 for storing a program which, when executed by the processor 101, results in the steps shown in the flow charts of FIGS. 5 and 7 being executed. It can also be said that this program causes a computer to execute the procedures or methods of the image acquisition section 21, the facility detection section 22, the monitoring area setting section 23, the occupant monitoring section 24, and the warning section 25. FIG.

Here, the processor 101 is a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, or the like.
The memory 102 may be a nonvolatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), or flash memory, or a hard disk or flexible disk. or an optical disc such as a CD (Compact Disc) or a DVD (Digital Versatile Disc).

The functions of the image acquisition unit 21, the equipment detection unit 22, the monitoring area setting unit 23, the occupant monitoring unit 24, and the warning unit 25 are partly realized by dedicated hardware and partly by software or firmware. You may make it Thus, the processing circuitry in the occupant monitoring device 20 can implement the above-described functions by means of hardware, software, firmware, or a combination thereof.

As described above, the occupant monitoring device 20 according to Embodiment 1 includes the image acquisition unit 21, the equipment detection unit 22, the monitoring area setting unit 23, and the occupant monitoring unit 24. The image acquisition unit 21 acquires an image inside the vehicle 10 captured by the infrared camera 7 built into the room mirror 1 whose attitude is adjustable. The facility detection unit 22 uses the rear window 12 as the facility in the vehicle 10 imaged by the infrared camera 7 when the posture of the rearview mirror 1 is adjusted for use for the original purpose, and the image acquisition unit 21 acquires the facility. A facility detection area 13 is set at a predetermined position of the image obtained, and the rear window 12 is detected from the facility detection area 13 . When the equipment detection unit 22 detects the rear window 12, the monitoring region setting unit 23 sets the occupant monitoring region 15 for the image based on the image position of the rear window 12. FIG. The occupant monitoring unit 24 monitors the driver 11 using the image of the occupant monitoring area 15 set by the monitoring area setting unit 23 among the images acquired by the image acquisition unit 21 . With this configuration, the occupant monitoring device 20 can detect the driver 11 to be monitored from the image captured by the infrared camera 7 built into the room mirror 1 whose attitude is adjustable.

The occupant monitoring device 20 according to Embodiment 1 also includes a warning unit 25 . If at least one of the case where the equipment detection unit 22 cannot detect the rear window 12 or the case where at least a part of the occupant monitoring area 15 set by the monitoring area setting unit 23 protrudes from the image, the warning unit 25 to issue a warning to With this configuration, the occupant monitoring device 20 can be used when the driver 11 provided in the vehicle 10 cannot visually recognize the rear through the rearview mirror 1 or when the occupant monitoring unit 24 cannot monitor the driver 11. In addition, the driver 11 can be urged to readjust the posture of the rearview mirror 1 . Therefore, the occupant monitoring device 20 achieves both a posture in which the infrared camera 7 built in the rearview mirror 1 can image the driver 11 to be monitored and a posture suitable for using the rearview mirror 1 for its original purpose. be able to.

Note that if the rear window 12 is hidden due to the presence of luggage or the like in the vehicle 10, the facility detection unit 22 cannot detect the rear window 12 in steps ST2 and ST12. In this case, the facility detection unit 22 detects a predetermined second facility in the vehicle 10 from the image, and estimates the image position of the rear window 12, which is the first facility, based on the image position of the second facility. may For example, if the left and right rear side windows 14 are the second facility, the facility detection unit 22 detects at least one of the left rear side window 14 and the right rear side window 14 from the image. The facility detection unit 22 may determine whether the detected rear side window 14 is on the left side or the right side of the vehicle 10 based on the shape of the rear side window 14, the angle of the rear pillar 16, or the like. When at least one of the left rear side window 14 and the right rear side window 14 is detected from the image, the facility detection part 22 estimates the image position of the rear window 12 based on the detected image position of the rear side window 14.例文帳に追加The positional relationship between the left rear side window 14 and the rear window 12 and the positional relationship between the right rear side window 14 and the rear window 12 are preset in the equipment detection unit 22 . The monitoring area setting unit 23 sets the occupant monitoring area 15 for the image based on the image position of the rear window 12 estimated by the facility detection unit 22 .

Further, when the outside of the vehicle is dark, such as when the vehicle 10 is running at night, it may be difficult for the equipment detection unit 22 to detect the rear window 12 from the equipment detection area 13 . Therefore, it is desirable that the rear pillar 16 and the roof lining 17 around the rear window 12 are made of a material that does not absorb infrared rays. As a result, the infrared rays emitted by the infrared light emitting units 6a and 6b are transmitted through the rear window 12, but are reflected around the rear window 12, increasing the difference in pixel values between the rear window 12 and its surroundings. Therefore, detection of the rear window 12 by the equipment detection unit 22 is facilitated.

Further, in Embodiment 1, the occupant monitoring unit 24 monitors the driver 11, but an occupant other than the driver 11 may be monitored. In this case, the monitoring area setting unit 23 holds in advance information about the image position corresponding to the seat where the passenger to be monitored, such as the front passenger seat, is seated, as the position of the passenger monitoring area 15 with reference to the position of the rear window 12. . In addition, the imaging range of the infrared camera 7 is set to a range that includes the rear window 12 and the seat where the passenger to be monitored boards when the posture of the rearview mirror 1 is adjusted for the original purpose. be.
Note that the occupant monitoring unit 24 may monitor two or more occupants at the same time. In this case, the monitoring region setting unit 23 sets two or more occupant monitoring regions 15 on the image with the position of the rear window 12 as a reference. In addition, the imaging range of the infrared camera 7 is set to a range that includes the rear window 12 and all the seats on which the passengers to be monitored board when the posture of the rearview mirror 1 is adjusted for the original purpose. be done.

Further, in Embodiment 1, the infrared camera 7 for monitoring passengers is built in the rearview mirror 1, but it may be built in an in-vehicle device other than the rearview mirror 1. FIG. The in-vehicle equipment that incorporates the infrared camera 7 includes a display installed on the ceiling of the front seat whose posture is adjustable, an overhead console installed on the ceiling of the front seat that can be opened and closed, and the like. In that case, the equipment detection unit 22 detects the equipment in the vehicle interior imaged by the infrared camera 7 built in the display etc. when the posture of the display etc. is adjusted so as to be used for the original purpose. 1 facility. In addition, the imaging range of the infrared camera 7 is set to a range that includes the first facility and the seat on which the passenger to be monitored boards when the posture of the display or the like is adjusted so that it can be used for its original purpose. be.

Further, in Embodiment 1, the infrared camera 7 is used as the camera for passenger monitoring, but a visible light camera may be used.

Further, in Embodiment 1, the functions of the image acquisition unit 21, the equipment detection unit 22, the monitoring area setting unit 23, the occupant monitoring unit 24, and the warning unit 25 are integrated into the occupant monitoring device 20 mounted on the vehicle 10. However, these functions may be distributed to two or more devices among a server device on a network, a mobile terminal such as a smart phone, or the occupant monitoring device 20 . Also, the occupant monitoring device 20 may be constructed as a server device on a network.

It should be noted that, within the scope of the invention, any component of the embodiment can be modified, or any component of the embodiment can be omitted.

The occupant monitoring system according to the present invention monitors the occupants using the images taken by the cameras built in the on-vehicle equipment capable of adjusting the attitude. Suitable for use in occupant monitoring systems to monitor occupants.

1 room mirror (in-vehicle device), 2 mirror case, 3 arm, 4 infrared filter, 5 half mirror, 6a, 6b infrared light emitting unit, 7 infrared camera, 8 partition member, 10 vehicle, 11 driver, 12 rear window (second 1 facility), 13 facility detection area, 14 rear side window, 15 occupant monitoring area, 16 rear pillar, 17 roof lining, 20 occupant monitoring device, 21 image acquisition unit, 22 facility detection unit, 23 monitoring area setting unit, 24 occupant monitoring unit , 25 warning unit, 26 warning device, 30 occupant monitoring system, 100 processing circuit, 101 processor, 102 memory.

Claims (6)

an image acquisition unit that acquires an image of the interior of the vehicle captured by a camera incorporated in a rearview mirror that can be adjusted in posture;
When the posture of the rearview mirror is adjusted so that the driver can easily visually recognize the rear, the facility detection area is set at a predetermined position of the image acquired by the image acquisition unit, and the rear window is positioned in the facility detection area. A facility detection unit that detects from
a monitoring area setting unit for setting an occupant monitoring area for the image with reference to the image position of the rear window when the rear window is detected by the facility detection unit;
and an occupant monitoring unit that monitors the driver using the image of the occupant monitoring area set by the monitoring area setting unit among the images acquired by the image acquisition unit. If at least one of the case where the equipment detection unit cannot detect the rear window or the case where at least a part of the occupant monitoring area set by the monitoring area setting unit protrudes from the image, the 2. The occupant monitoring device according to claim 1, further comprising a warning unit for instructing a warning device to issue a warning. When the rear window cannot be detected, the equipment detection unit detects predetermined left and right rear side windows in the vehicle interior from the image, and detects images of the rear window based on image positions of the left and right rear side windows. 2. The occupant monitoring system of claim 1, further comprising estimating position. When the rear window cannot be detected, the facility detection unit detects at least one of the left rear side window and the right rear side window from the image, and the image position of the rear window is based on the image position of the detected rear side window. 2. The occupant monitoring device according to claim 1 , wherein the occupant monitoring device estimates A rearview mirror with a built-in camera and an adjustable posture,
An occupant monitoring system comprising the occupant monitoring device according to claim 1 . The image acquisition unit acquires an image of the interior of the vehicle captured by a camera built into the rearview mirror with adjustable posture,
When the equipment detection unit adjusts the attitude of the rearview mirror so that the driver can easily visually recognize the rear, the equipment detection area is set at a predetermined position of the image acquired by the image acquisition unit, and the rear view mirror is detected. detecting a window from the facility detection area;
a monitoring area setting unit, when the equipment detecting unit detects the rear window, sets an occupant monitoring area for the image based on the image position of the rear window ;
An occupant monitoring method, wherein an occupant monitoring unit monitors the driver using an image of the occupant monitoring area set by the monitoring area setting unit out of the images acquired by the image acquiring unit.

Images