WO2023242889A1

WO2023242889A1 - Image capture device, occupant monitoring system, and vehicle

Info

Publication number: WO2023242889A1
Application number: PCT/JP2022/023562
Authority: WO
Inventors: 一弘都築; 太郎熊谷
Original assignee: 三菱電機株式会社
Priority date: 2022-06-13
Filing date: 2022-06-13
Publication date: 2023-12-21

Abstract

The present invention comprises: a housing (11); an image capture unit (15) that is provided in the housing (11) and guides light incident from a subject area through a front panel (12) that forms one surface of the housing on the subject area side thereof; and a blocking part (13) that can partially block, on the front panel (12), light from entering the image capture unit (15) from the subject area.

Description

Imaging devices, occupant monitoring systems, and vehicles

The present disclosure relates to an imaging device, an occupant monitoring system, and a vehicle.

Conventionally, an imaging device is installed inside a vehicle to monitor the interior of the vehicle, and information based on images of the interior of the vehicle captured by the imaging device (hereinafter referred to as "monitoring information") is transmitted to devices inside or outside the vehicle. There are known technologies that provide various services by outputting data to the outside.
For example, Patent Document 1 discloses an in-vehicle device including a driver camera that transmits data used for driving evaluation of a vehicle driver to a driving evaluation device, and the driving evaluation device uses an image of the driver while driving as the data. An in-vehicle device that transmits data is disclosed.
Incidentally, in recent years, wide-angle cameras that can image not only the driver but also multiple occupants, such as the driver and a passenger in the front passenger seat (hereinafter referred to as "passenger seat occupant"), have been used as imaging devices for monitoring the inside of a vehicle. is widespread.

JP 2019-159642 Publication

Conventional technologies, such as the technology disclosed in Patent Document 1, have been applied to provide various services by outputting monitoring information based on captured images of occupants inside a vehicle to the outside. In a vehicle, if the imaging device installed inside the vehicle is a wide-angle camera, the imaging device can image an area where multiple occupants may be present. There has been a problem in that there is a risk that personal information of the occupant, such as a captured image of an occupant other than the occupant, for which the consent of the occupant has not been obtained may be provided to a third party.

The present disclosure has been made in order to solve the above-mentioned problems, and is an imaging device capable of imaging an area where occupants in at least two seats may exist in a vehicle interior, the imaging device comprising: It is an object of the present invention to provide an imaging device that can partially select an area to be imaged.

An imaging device according to the present disclosure is an imaging device capable of imaging a subject area in which occupants in at least two seats may be present in a vehicle interior of a vehicle, and includes a housing, a housing provided within the housing, and a subject area of the housing. An imaging unit that guides light incident from the subject area through a front panel forming one side of the camera, and a shielding unit that can partially block the light entering the imaging unit from the subject area on the front panel. It is prepared.

According to the present disclosure, it is possible to partially select the region to be imaged from among the regions that can be imaged.

1A, FIG. 1B, and FIG. 1C are diagrams for explaining a configuration example of an imaging device according to the first embodiment. FIG. 2A, FIG. 2B, and FIG. 2C are diagrams for explaining an example of attachment of a shielding portion included in the imaging device according to Embodiment 1 to the imaging device. To describe an example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging device according to the first embodiment. This is a diagram. Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for Another example of the shielding of light from the shielding target area by the shielding part and the imaging range of the imaging apparatus that changes when the shielding part blocks the light from the shielding target area in the imaging apparatus according to the first embodiment will be described. This is a diagram for 1 is a diagram showing a configuration example of an occupant monitoring device according to a first embodiment; FIG. 3 is a flowchart for explaining the operation of the occupant monitoring device according to the first embodiment. 13A and 13B are diagrams showing an example of the hardware configuration of the occupant monitoring device according to the first embodiment.

Embodiments of the present disclosure will be described in detail below with reference to the drawings.
Embodiment 1.
The imaging device according to Embodiment 1 is mounted on a vehicle and used to monitor the condition of an occupant, such as the direction of the occupant's face or the occupant's line of sight.
In the first embodiment, the imaging device is an infrared camera.
The imaging device is installed so as to be able to image an area (hereinafter referred to as a "subject area") where occupants in at least two seats may be present in the vehicle interior.
The image captured by the imaging device is output to an occupant monitoring device that together with the imaging device constitutes an occupant monitoring system. The occupant monitoring system is a so-called "Driver Monitoring System (DMS)." The occupant monitoring device monitors the occupant based on the captured image captured by the imaging device, and provides monitoring information based on the captured image to the outside based on the monitoring result. Details of the occupant monitoring device will be described later.

Here, as described above, the imaging device according to the first embodiment is a wide-angle camera that has an angle of view that can capture an image of a subject area where occupants in at least two seats may be present. A plurality of occupants may be captured in the captured image captured by the imaging device. Whether the captured image is, for example, a captured image of the passenger himself or herself, or a captured image of a crew member who has agreed that information based on the captured image (monitoring information) may be provided to an external party. For example, it can be said that there is no problem even if the occupant monitoring device provides monitoring information based on the captured image to the outside. However, if the captured image is based on the captured image of the crew member, for example, the crew member has not agreed that information based on the captured image of themselves may be provided to the outside, or If an image is taken of a passenger who is not aware that the information may be provided to the outside, the passenger monitoring device will not provide the personal information of the passenger to the outside even though the passenger's consent has not been obtained. There is a risk that it will happen.
To solve this problem, the imaging device according to Embodiment 1 has the advantage that even if the captured images are used to generate monitoring information based on the monitoring results of the occupants in the vehicle, the personal information of the vehicle occupants may not be used as intended. This makes it possible to prevent the information from being provided to a third party without doing so.

The imaging device 1 is installed, for example, in the center of the vehicle in the width direction. In Embodiment 1 below, as an example, it is assumed that the imaging device 1 is installed in a center console panel of a vehicle. In the first embodiment, the center in the width direction of the vehicle is not limited to the exact center, but includes approximately the center.
When the imaging device 1 is installed in a vehicle, the imaging device 1 is installed in a vehicle, and from the center console panel, an area corresponding to the driver's seat (hereinafter referred to as "driver seat compatible area"), an area corresponding to the passenger seat (hereinafter referred to as "passenger seat compatible area"), and a region corresponding to the passenger seat (hereinafter referred to as "passenger seat compatible area") A region including a region corresponding to a rear seat (hereinafter referred to as a "rear seat corresponding region") is set as a subject region, and an image of the subject region can be captured. In the first embodiment, the rear seat corresponding area is specifically an area corresponding to the rear seat between the driver's seat and the passenger seat, in other words, the area corresponding to the center of the rear seat.

1A, FIG. 1B, and FIG. 1C are diagrams for explaining a configuration example of an imaging device 1 according to the first embodiment.
FIG. 1A is a perspective view showing the appearance of an example of the imaging device 1 according to the first embodiment. FIG. 1B is a diagram for explaining the configuration of main parts of the imaging device 1 shown in FIG. 1A, and is a diagram of the imaging device 1 viewed from above in an installed state. For convenience, FIG. 1B is a diagram showing the imaging device 1 of FIG. 1A with the top surface portion 111 of the casing 11 removed so that the configuration of the main parts of the imaging device 1 inside the casing 11 can be seen. FIG. 1C is a view taken along arrow A in FIG. 1B. Note that the imaging device 1 shown in FIG. 1C is a view seen from the front side of the imaging device 1 in a state in which the imaging device 1 is installed, and for convenience, FIG. 1C shows a state in which the shielding part 13 shown in FIG. 1A is closed. It is said that
In FIG. 1A, FIG. 1B, and FIG. 1C, the direction of the Z-axis corresponds to the direction opposite to the traveling direction of the vehicle when the imaging device 1 is installed in the vehicle. Note that when the imaging device 1 is installed in a vehicle, the region on the opposite side to the direction of travel of the vehicle includes the subject region. In FIGS. 1A, 1B, and 1C, the direction of the X-axis corresponds to the width direction of the vehicle when the imaging device 1 is installed in the vehicle. In FIGS. 1A, 1B, and 1C, the direction of the Y-axis corresponds to the downward direction of the vehicle when the imaging device 1 is installed in the vehicle.

As shown in FIGS. 1A, 1B, and 1C, the imaging device 1 has a housing 11.
The imaging device 1 is installed in a vehicle by fixing the housing 11 to a center console panel.
The imaging device 1 includes a light projecting section 14 and an imaging section 15 inside a housing 11 .
The light projecting unit 14 includes an LED (Light Emitting Diode) 141, and emits infrared light toward the subject area. The LED 141 is an infrared LED. The LED 141 is provided on the substrate 142.
The imaging unit 15 is an imaging optical system and includes a lens 151 and an image sensor 152. The lens 151 guides light incident from the subject area, and forms an image of the guided light on the image sensor 152. The image sensor 152 is arranged behind the lens 151, in other words, on the opposite side of the lens 151 from the subject area, and receives the light guided by the lens 151 and converts it into an electrical signal.
The imaging unit 15 images the subject area using infrared light from the light projecting unit 14, specifically the LED 141, as illumination.

The casing 11 has a top surface part 111, a back surface part 114, and

side parts

112, 113, and one surface of the casing 11 on the subject area side, in other words, one surface of the casing 11 that is in the imaging direction of the imaging section 15. is formed by the front panel 12.
The front panel 12 transmits infrared light emitted by the LED 141 and blocks visible light. The front panel 12 serves to hide components such as the light projecting section 14 and the imaging section 15 inside the housing 11 of the imaging device 1 from the occupant when the imaging device 1 is installed in a vehicle. Note that, since the front panel 12 transmits infrared light, the front panel 12 does not affect the imaging by the imaging unit 15.
Light in the infrared wavelength range that is emitted from the LED 141 of the light projector 14 and reflected by the occupant's face, etc., or emitted from the occupant's face, etc., heads in the opposite direction to the Z-axis direction and passes through the front panel 12. The light passes through and is guided to the lens 151 of the imaging section 15, and is incident on the light receiving surface of the imaging element 152 of the imaging section 15.

The imaging device 1 includes a shielding section 13 on the front panel 12 that can partially shield light entering the imaging section 15 from the subject area.
In the first embodiment, a part of the surface of the front panel 12 on the subject area side, which is between the subject area and the imaging unit 15 and receives light entering the imaging unit 15 from the subject area. is referred to as a "light-receiving surface" (the surface in the range indicated by 12a in FIGS. 1A and 1C). The shielding part 13 is provided so that it can partially shield the light receiving surface of the front panel 12 by opening and closing. In the first embodiment, a surface of the light receiving surface of the front panel 12 that can be shielded by the shielding section 13 is referred to as a "shieldable surface." In the first embodiment, when the shielding section 13 is opened, it means that the shielding section 13 does not cover the light-receiving surface of the front panel 12 . The closing of the shielding part 13 means that the shielding part 13 covers the shieldable surface of the light receiving surface of the front panel 12.
The shielding section 13 partially shields the light entering the imaging section 15 from the subject area by closing and covering the shieldable surface of the light receiving surface of the front panel 12 .

In the first embodiment, the imaging device 1 includes two shielding parts 13, a first shielding part 131 and a second shielding part 132. That is, the shielding section 13 includes a first shielding section 131 and a second shielding section 132.
The first shielding section 131 and the second shielding section 132 can partially shield light entering the imaging section 15 from the subject area, independently of each other.

In the first embodiment, a region of the subject region where the shielding section 13 can shield light incident on the imaging section 15 is referred to as a "shielding target region." The shielding target area is an area associated with a seat.
It is assumed that the area corresponding to which seat the imaging device 1 can set as the shielding target area among the areas corresponding to each seat included in the subject area is determined in advance. In the first embodiment, as an example, among the driver seat corresponding area, passenger seat corresponding area, and rear seat corresponding area included in the subject area, the front passenger seat corresponding area or the rear seat corresponding area can be the shielding target area. It is assumed that the area is predetermined.
That is, the shielding unit 13 protects the light entering the imaging unit 15 from the subject area, the light entering the imaging unit 15 from the passenger seat corresponding area, the light entering the imaging unit 15 from the rear seat corresponding area, or the light entering the imaging unit 15 from the passenger seat corresponding area. It is possible to block light that enters the imaging unit 15 from the corresponding area and the rear seat corresponding area.
Specifically, the first shielding section 131 can block the light that is incident on the imaging section 15 from the passenger seat corresponding area out of the light that is incident on the imaging section 15 from the subject area, and the second shielding section 132 can block the light that is incident on the imaging section 15 from the area corresponding to the passenger seat. Of the light that enters the imaging section 15 from the rear seat area, it is possible to block the light that enters the imaging section 15 from the rear seat corresponding area.

The range of the shieldable surface of the light-receiving surface of the front panel 12 is determined by the position where the imaging device 1 is installed in the vehicle and which seat in the area included in the subject area in the vehicle where the imaging device 1 is installed. It depends on whether the area corresponding to is to be the area to be occluded. The shielding unit 13 can be configured to be shieldable in the imaging device 1 while the imaging device 1 is installed in the vehicle, depending on the shieldable surface determined from the installation position of the imaging device 1 in the vehicle and the shielding target area in the vehicle. It has a width that can cover the surface, and is provided at a position that can cover the shieldable surface when the shielding section 13 is closed.
Furthermore, when there are a plurality of shielding target regions, the imaging device 1 includes at least as many shielding parts 13 as there are shielding target regions, each of which corresponds to each shielding target region and can be opened and closed independently. Regarding the shielding part 13, the range within the shieldable surface that receives light from the shielding target area associated with itself becomes the shieldable surface corresponding to the shielding part 13.

In the first embodiment, the area corresponding to the passenger seat and the area corresponding to the rear seat are assumed to be the shielding target areas, so the imaging device 1 includes the first shielding part 131 and the second shielding part 131 as the shielding part 13, as described above. 132.
Among the shieldable surfaces of the front panel 12 , a range where light entering the imaging unit 15 from the passenger seat corresponding area can be shielded becomes a shieldable surface by the first shielding section 131 . Furthermore, among the shieldable surfaces of the front panel 12 , a range that can shield light entering the imaging section 15 from the rear seat corresponding area becomes a shieldable surface by the second shielding section 132 . Among the shieldable surfaces of the front panel 12, the surface that can be covered by the first shielding part 131 and the surface that can be covered by the second shielding part 132 do not overlap.

The shielding unit 13 partially shields the light that enters the imaging unit 15 from the subject area, in other words, the shielding unit 13 blocks the light that enters the imaging unit 15 from the shielding target area, so that the imaging unit 15, the shielding target area is no longer imaged. That is, the imaging range of the imaging device 1 no longer includes the shielding target area. In other words, the imaging device 1 excludes the shielding target area from the subject area and partially selects the area to be imaged.
The shielding of light from the shielding target area by the shielding unit 13 and the imaging range of the imaging device 1 that changes due to the shielding unit 13 shielding the light from the shielding target area will be explained in detail using FIGS. 3 to 6. An example will be described later.

In the first embodiment, it is assumed that the shielding part 13 is opened and closed manually by a vehicle occupant.
For example, a handle part 13a (first shield part handle part 131a, second shield part handle part 132a) protruding toward the subject area is formed at the tip of the shield part 13 in the direction in which the shield part 13 closes. There is. The occupant of the vehicle can open and close the shielding part 13 by pinching the handle part 13a with his/her hand.

The details of the shielding part 13 will be explained using FIG. 2A, FIG. 2B, and FIG. 2C.
2A, FIG. 2B, and FIG. 2C are diagrams for explaining an example of attachment of the shielding section 13 included in the imaging device 1 according to the first embodiment to the imaging device 1. FIG. 2A is a diagram showing a part of the top surface section 111 of the casing 11 of the imaging device 1 to which the shielding section 13 is attached. FIG. 2B is a diagram showing an example of a state in which the shielding part 13 is attached to the top surface part 111 as shown in FIG. 2A. Note that in FIG. 2B, the shielding part 13 is shown in an open state. 2C, as shown in FIG. 2B, the shielding part 13 attached to the top surface part 111 is lowered to the front panel 12 (not shown in FIGS. 2A, 2B, and 2C) and closed, and the front panel 12 is closed. It is a figure which shows an example of the state where the shieldable surface of the light-receiving surface of is covered.
Note that FIG. 2A, FIG. 2B, and FIG. 2C are diagrams showing an example of the top surface portion 111 when the back surface of the top surface portion 111 is partially viewed from inside the housing 11.

For example, as shown in FIG. 2A, on the back surface of the top surface portion 111 of the housing 11 of the imaging device 1, a guide rail 1111 and a guide groove (first A guide groove) 1112 is provided.

The guide rails 1111 are provided to be located at both ends of the shielding part 13 in the left and right width directions, and guide the shielding part 13 movably in the opening and closing directions while holding both ends of the shielding part 13 in the left and right width directions. It is a member for
The guide rail 1111 is attached to the back surface of the top surface portion 111 of the housing 11 using fixing means such as screws 16, for example. The guide rail 1111 may be attached to the back surface of the top surface portion 111 of the housing 11 via an attachment member, for example.

A stopper (first stopper) 1113 that prevents the shielding part 13 from falling off when the shielding part 13 is closed is formed at the end of the guide rail 1111 in the direction in which the shielding part 13 is closed.
Further, at the end of the guide rail 1111 in the direction in which the shielding part 13 opens, a stopper (second stopper) is provided to prevent the shielding part 13 from moving further toward the back surface 114 (see FIG. 1A) of the housing 11. ) 1114 are formed.

As shown in FIG. 2B, the shielding part 13 has a protrusion (first protrusion) 13b that fits into the first guide groove 1112 provided on the back surface of the top surface part 111 of the housing 11, and the shielding part 13 opens. A protrusion (second protrusion) 13c that protrudes in the width direction of the shielding part 13 at the end of the direction and prevents the shielding part 13 from falling off by being caught by the first stopper 1113 is formed. Note that 131b indicates the first protrusion 131b for the first shielding part, which is the first protrusion 13b formed on the

first shielding part

131, and 132b indicates the first protrusion 131b formed on the second shielding part 132. The first protrusion 132b for the second shielding part, which is the protrusion 13b, is shown. Further, 1112a indicates a first guide groove 1112 for the first shielding part, which is the first guide groove 1112 into which the first protrusion 131b for the first shielding part fits, and 1112b indicates the first protrusion 132b for the second shielding part. It shows the first guide groove 1112b for the second shielding part, which is the first guide groove 1112 into which the second shield part fits. In FIG. 1A, FIG. 1B, and FIG. 1C, illustration of the first protrusion 13b is omitted for ease of explanation.
Moreover, the end of the shielding part 13 in the direction in which it opens hits the second stopper 1114, so that it does not move any further in the closing direction.

The guide rail 1111 has a second guide groove (not shown) extending in the direction of movement of the shielding part 13, and the shielding part 13 has a second protrusion 13c in the opposite second guide rail 1111. It moves in the opening/closing direction while being inserted and held in the guide groove. Thereby, the shielding part 13 is guided in the opening/closing direction by the guide rail 1111 to open and close, and covers or does not cover the shieldable surface of the light receiving surface of the front panel 12.
In addition, when the shielding part 13 is pulled out from the top surface part 111 of the housing 11 and moved in the opening direction, it is movable along the front panel 12.

The shielding part 13 is fixed with the first protrusion 13b fitted into the first guide groove 1112 by the sliding fixing member 17 so as not to fall into the housing 11. The fixing member 17 is fixed to the top surface portion 111 of the housing 11 by welding or the like.
Note that, for convenience, illustration of the fixing member 17 is omitted in FIG. 2A.

Regarding the shielding of light from the shielding target area by the shielding unit 13 and the imaging range of the imaging device 1 that changes due to the shielding unit 13 shielding the light from the shielding target area in the imaging device 1 according to the first embodiment, An example will be explained using FIGS. 3 to 6.
3A, FIG. 4A, FIG. 5A, and FIG. 6A are views of the interior of the vehicle 10 viewed from above. Note that in FIGS. 3A, 4A, 5A, and 6A, for the sake of simplicity, illustrations of the interior of the vehicle other than the imaging device 1 and the occupant are omitted.
Further, in FIGS. 3A, 4A, 5A, and 6A, the vehicle 10 is a right-hand drive vehicle, as an example. 3A, FIG. 4A, FIG. 5A, and FIG. 6A, five passengers are riding in the vehicle 10, the driver is indicated by "D", and the passenger in the front passenger seat (hereinafter referred to as "passenger seat passenger") ) are indicated by "S", rear seat occupants (hereinafter referred to as "rear seat occupants") are indicated by "B1", "B2", or "B3", and the steering wheel is indicated by "H". has been done.
3B is a perspective view of the imaging device 1 shown in FIG. 3A, FIG. 4B is a perspective view of the imaging device 1 shown in FIG. 4A, FIG. 5B is a perspective view of the imaging device 1 shown in FIG. 5A, and FIG. 6B is a perspective view of the imaging device 1 shown in FIG. 6A is a perspective view of the imaging device 1 shown in FIG. 6A.
3B, FIG. 4B, FIG. 5B, and FIG. 6B, illustration of the top surface portion 111, back surface portion 114, and

side portions

112 and 113 of the housing 11 is omitted for simplicity of explanation.

3A and 3B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding part 13 is opened in the imaging device 1.
In FIGS. 3A and 3B, the imaging range of the imaging device 1 is indicated by "E1".
Note that in FIG. 3A, for convenience, the imaging range E1 extends to the area behind the driver's seat and the passenger seat, but in reality, it is blocked by the driver's seat and the passenger seat, and the imaging device 1 is Only the area between the seat and the passenger seat can be imaged.

As shown in FIGS. 3A and 3B, since the shielding section 13 is open in the imaging device 1, the shielding section 13 does not cover the light-receiving surface of the front panel 12. In other words, the imaging device 1 does not block light that enters the imaging unit 15 from the subject area.
In this case, the imaging device 1 images a region corresponding to the driver's seat, a region corresponding to the passenger seat, and a region corresponding to the rear seat. The occupants B1, B2, and B3 are imaged. This means that the imaging device 1 has selected the entire area of the subject area as the area to be imaged.
Then, images captured by the imaging device 1 of the driver D, the front passenger seat passenger S, and the rear seat passengers B1, B2, and B3 are output to the occupant monitoring device 2. The occupant monitoring device 2 generates monitoring information based on captured images of the driver D, the front passenger S, and the rear seat occupants B1, B2, and B3, and provides the monitoring information to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on captured images of the inside of the vehicle may be provided to the outside may be aware that the monitoring information may be provided to the outside. (Passenger S and rear seat occupants B1, B2, B3), there is no risk that the personal information of the occupants will be provided to outside parties even though the consent of the occupants has not been obtained. Even if there is no such thing, the shielding part 13 may be left open.

4A and 4B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding section 13 is closed in the imaging device 1.
In FIGS. 4A and 4B, the imaging range of the imaging device 1 is indicated by "E2".
Note that in FIG. 4A, for convenience, the imaging range E2 extends to the area behind the driver's seat and the passenger seat, but in reality, it is blocked by the driver's seat, and the imaging device 1 covers the area behind the driver's seat and the passenger seat. Only the area between the seats can be imaged.

As shown in FIGS. 4A and 4B, since the shielding section 13 is closed in the imaging device 1, the shielding section 13 covers the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 blocks light that enters the imaging unit 15 from the passenger seat corresponding area and the rear seat corresponding area in the subject area.
In this case, the imaging device 1 images the area corresponding to the driver's seat, and only the driver D is captured in the captured image captured by the imaging device 1. The imaging device 1 excludes the passenger seat corresponding area and the rear seat corresponding area from the subject area, and partially selects the area to be imaged.
Then, a captured image of only the driver D captured by the imaging device 1 is output to the occupant monitoring device 2. The occupant monitoring device 2 generates monitoring information based on a captured image of only the driver D, and provides it to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on captured images of the inside of the vehicle may be provided to the outside may be aware that the monitoring information may be provided to the outside. (front seat passenger S and rear seat passengers B1, B2, B3), the personal information of the fellow passenger may not be provided to an outside party even though the consent of the fellow passenger has not been obtained. Assuming that there is a possibility that the accident may occur, the shielding portion 13 may be closed.

5A and 5B show a case where the first shielding part 131 and the second shielding part 132 of the shielding part 13 in the imaging device 1 are closed and the second shielding part 132 is open. 2 is a diagram for explaining an example of an imaging range of the imaging device 1. FIG.
In FIGS. 5A and 5B, the imaging range of the imaging device 1 is indicated by "E3".
Note that in FIG. 5A, for convenience, the imaging range E3 extends to the area behind the driver's seat and the passenger seat, but in reality, it is blocked by the driver's seat and the front passenger seat, and the imaging device 1 is Only the area between the seat and the passenger seat can be imaged.

As shown in FIGS. 5A and 5B, since the first shielding section 131 is closed in the imaging device 1, the first shielding section 131 covers the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 blocks light that enters the imaging unit 15 from the area corresponding to the passenger seat in the subject area. On the other hand, since the second shielding section 132 is open in the imaging device 1, the second shielding section 132 does not cover the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 does not block the light that enters the imaging unit 15 from the rear seat corresponding area in the subject area.
In this case, the imaging device 1 images the driver's seat corresponding area and the rear seat corresponding area, and the captured image captured by the imaging device 1 includes the driver D and the rear seat passengers B1, B2, and B3. The passenger S in the passenger seat is not captured in the captured image. The imaging device 1 excludes the area corresponding to the passenger seat from the subject area, and partially selects the area to be imaged.
Then, captured images of the driver D and the rear seat passengers B1, B2, and B3 captured by the imaging device 1 are output to the occupant monitoring device 2. The occupant monitoring device 2 generates monitoring information based on captured images of the driver D and rear seat occupants B1, B2, and B3, and provides the monitoring information to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on a captured image of the inside of the vehicle may be provided to an outside party, may later notice that the monitoring information may be provided to an outside party. If the consent of seat occupants B1, B2, and B3 is obtained, but the consent of front passenger seat occupant S is not obtained, the personal information of front passenger seat occupant S may be transferred without the consent of passenger seat occupant S. However, since there is a possibility that the data may be provided to the outside, the first shielding part 131 may be closed.

FIGS. 6A and 6B illustrate a case where, of the first shielding part 131 and the second shielding part 132 included in the shielding part 13 in the imaging device 1, the first shielding part 131 is opened and the second shielding part 132 is closed. 2 is a diagram for explaining an example of an imaging range of the imaging device 1. FIG.
In FIGS. 6A and 6B, the imaging range of the imaging device 1 is indicated by "E4".
Note that in FIG. 6A, for convenience, the imaging range E4 extends to the area behind the driver's seat and the passenger seat, but in reality, it is blocked by the driver's seat and the passenger seat, and the imaging device 1 is Only the area between the seat and the passenger seat can be imaged.

As shown in FIGS. 6A and 6B, since the second shielding part 132 is closed in the imaging device 1, the second shielding part 132 covers the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 blocks light that enters the imaging unit 15 from the rear seat corresponding region of the subject region. On the other hand, since the first shielding section 131 is open in the imaging device 1, the first shielding section 131 does not cover the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 does not block the light that enters the imaging unit 15 from the area corresponding to the passenger seat in the subject area.
In this case, the imaging device 1 images the driver's seat corresponding area and the passenger seat corresponding area, and the driver D and the passenger S are captured in the image taken by the imaging device 1. The rear seat passengers B1, B2, and B3 are not captured in the captured image. The imaging device 1 excludes the rear seat corresponding area from the subject area, and partially selects the area to be imaged.
Then, captured images of the driver D and the passenger S in the passenger seat S are captured by the imaging device 1 and are output to the occupant monitoring device 2 . The occupant monitoring device 2 generates monitoring information based on captured images of the driver D and the passenger S in the passenger seat, and provides it to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on a captured image of the inside of the vehicle may be provided to the outside, may ask the driver to inform the driver that the monitoring information may be provided to the outside. If the consent of seat occupant S is obtained, but the consent of rear seat occupants B1, B2, and B3, especially rear seat occupant B2, is not obtained, the personal information of rear seat occupants B1, B2, and B3 will be Even though the consent of the occupants B1, B2, and B3 has not been obtained, there is a possibility that the information will be provided to the outside, so the second shielding portion 132 may be closed.

Note that in the examples shown in FIGS. 3 to 6, the vehicle 10 is a right-hand drive vehicle, but this is only an example. Vehicle 10 may be a left-hand drive vehicle.

In this way, the imaging device 1 according to the first embodiment is configured such that the imaging device 1 can be accessed from the subject area via the housing 11 and the front panel 12, which is provided in the housing 11 and forms one surface of the housing 11 on the subject area side. It includes an imaging section 15 that guides incident light, and a shielding section 13 that can partially block the light that enters the imaging section 15 from the subject area on the front panel 12.
Thereby, the imaging device 1 can partially select the area to be imaged from the imageable area (subject area). As a result, the imaging device 1 prevents the personal information of vehicle occupants from being unintentionally provided to a third party even if the captured image is used to output monitoring information based on monitoring inside the vehicle. be able to.

As mentioned above, conventionally, an imaging device for monitoring the interior of the vehicle is installed in a vehicle interior, and monitoring information based on images of occupants inside the vehicle interior is transmitted to devices inside or outside the vehicle interior, etc. Technologies for providing various services by outputting to the outside are known.
Examples of the above-mentioned services include providing data used for driving evaluation of vehicle drivers, and in the event that the vehicle, its occupants, other vehicles, occupants of other vehicles, or people around the vehicle are involved in the accident. In order to prove that there are no defects in the occupants of the vehicle in the event of an unforeseen situation, such as to store captured images of the vehicle interior or the condition of the occupants, or to improve the comfort of the occupants, etc. Examples include providing the results of determining the presence or absence of an occupant, personal authentication, and inattention determination based on images taken inside the vehicle interior.

However, in vehicles that provide various services by outputting monitoring information based on captured images of passengers inside the vehicle, the imaging device installed inside the vehicle to monitor the interior of the vehicle is a wide-angle camera. In this case, since the imaging device can image an area where multiple occupants may exist, it may image multiple occupants including an occupant who is not the owner of the vehicle. As a result, when attempting to provide the above-mentioned services in a vehicle, personal information of the occupant, such as images taken of an occupant who is not the owner of the vehicle, for which the consent of the occupant has not been obtained may be collected as monitoring information by a third party. There is a risk that it may be provided to someone else.
For example, if you are a car owner and driver, you understand and agree to a system that provides monitoring information based on images captured by a wide-angle camera inside the car before using the car. is assumed. However, passengers other than the driver may not be aware of the above system. In this case, the personal information of a passenger other than the driver is provided to a third party even though the consent of the passenger other than the driver has not been obtained.

On the other hand, as described above, the imaging device 1 according to the first embodiment has a housing 11 and a front panel 12 that is provided in the housing 11 and forms one surface of the housing 11 on the subject area side. , an imaging unit 15 that guides light incident from the subject area, and a shielding unit 13 that can partially block the light incident on the imaging unit 15 from the subject area on the front panel 12.
Thereby, the imaging device 1 can partially select the area to be imaged from the imageable area (subject area). That is, in the imaging device 1, the shielding section 13 can partially shield the light that enters the imaging section 15 from the subject area on the front panel 12. By making it possible to block the light that enters the imaging unit 15 from the shielding target area of the area, the imaging device 1 can prevent the shielding target area of the subject area from being included in the imaging range. As a result, the imaging device 1 prevents the personal information of vehicle occupants from being unintentionally provided to a third party even if the captured image is used to output monitoring information based on monitoring inside the vehicle. be able to.

In the first embodiment described above, the subject area includes a driver seat corresponding area, a passenger seat corresponding area, and a rear seat corresponding area, and among these, the passenger seat corresponding area or the rear seat corresponding area is set as the shielding target area. Although an example has been shown, this is just one example. It is possible to appropriately set which region of the subject region corresponds to a seat as the shielding target region.

Further, in the first embodiment described above, in the imaging device 1, the shielding section 13 has two shielding sections 13, the first shielding section 131 and the second shielding section 132, but this is only an example. do not have. The imaging device 1 may be configured to have three or more shielding parts 13.
The imaging device 1 may include at least as many shielding sections 13 as there are shielding target regions, each of which corresponds to each shielding target region and can be opened and closed independently.

For example, the imaging device 1 may be configured to include one shielding section 13.
Here, for example, it is assumed that the imaging device 1 has an angle of view that is capable of capturing an image of a driver who may be present in the driver's seat and a front passenger seat occupant who may be present in the front passenger seat, but cannot image a rear seat occupant. That is, it is assumed that the subject area includes a driver seat corresponding area and a passenger seat corresponding area, and that the rear seat corresponding area is not included in the subject area. It is assumed that the shielding target area is predetermined to be the area corresponding to the passenger seat. In this case, the imaging device 1 only needs to be capable of blocking light that enters the imaging unit 15 from the area corresponding to the passenger seat. That is, there is one region to be shielded. Therefore, the imaging device 1 only needs to include one shielding section 13.

In the first embodiment, when the imaging device 1 includes one shielding section 13, the shielding section 13 blocks light from the shielding target region, and the shielding section 13 blocks light from the shielding target region. An example of the imaging range of the imaging device 1 that changes by blocking light from the outside will be described with reference to FIGS. 7 and 8. In the following description, for example, it is assumed that the imaging device 1 has an angle of view that can image a driver who may be present in the driver's seat and a front passenger who may be present in the front passenger seat, but cannot image a rear seat passenger. do. Further, for example, it is assumed that the region to be shielded is determined in advance to be the region corresponding to the passenger seat.

7A and 8A are views of the interior of the vehicle 10 viewed from above. Note that in FIGS. 7A and 8A, for ease of explanation, only the front seat portion of the vehicle interior is shown, and illustrations of other parts than the imaging device 1 and the occupants in the vehicle interior are omitted.
Furthermore, in FIGS. 7A and 8A, the vehicle 10 is a right-hand drive vehicle, as an example. In FIGS. 7A and 8A, a driver and a passenger are riding in the vehicle 10, the driver is indicated by "D", the front passenger is indicated by "S", and the steering wheel is indicated by "H". It is shown.
7B is a perspective view of the imaging device 1 shown in FIG. 7A, and FIG. 8B is a perspective view of the imaging device 1 shown in FIG. 8A.
In FIGS. 7B and 8B, illustration of the top surface 111, back surface 114, and

side surfaces

112 and 113 of the housing 11 is omitted for ease of explanation.

7A and 7B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding part 13 is opened in the imaging device 1.
In FIGS. 7A and 7B, the imaging range of the imaging device 1 is indicated by "E5".

As shown in FIGS. 7A and 7B, since the shielding section 13 is open in the imaging device 1, the shielding section 13 does not cover the light-receiving surface of the front panel 12. In other words, the imaging device 1 does not block light that enters the imaging unit 15 from the subject area.
In this case, the imaging device 1 images the driver's seat corresponding area and the passenger seat corresponding area, and the driver D and the passenger S are captured in the image taken by the imaging device 1. This means that the imaging device 1 has selected the entire area of the subject area as the area to be imaged.
Then, captured images of the driver D and the passenger S in the passenger seat S are captured by the imaging device 1 and are output to the occupant monitoring device 2 . The occupant monitoring device 2 generates monitoring information based on captured images of the driver D and the passenger S in the passenger seat, and provides it to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on a captured image of the inside of the vehicle may be provided to the outside, may ask the driver to inform the driver that the monitoring information may be provided to the outside. If the consent of the seat occupant S is obtained, the shielding portion 13 may be left open, assuming that there is no risk that the occupant's personal information will be provided to the outside even though the consent of the occupant has not been obtained.

8A and 8B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding section 13 is closed in the imaging device 1.
In FIGS. 8A and 8B, the imaging range of the imaging device 1 is indicated by "E6".

As shown in FIGS. 8A and 8B, since the shielding section 13 is closed in the imaging device 1, the shielding section 13 covers the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 blocks light that enters the imaging unit 15 from the area corresponding to the passenger seat in the subject area.
In this case, the imaging device 1 images the area corresponding to the driver's seat, and only the driver D is captured in the captured image captured by the imaging device 1. That is, the front passenger seat occupant S is not captured in the captured image captured by the imaging device 1 . The imaging device 1 excludes the area corresponding to the passenger seat from the subject area, and partially selects the area to be imaged.
Then, a captured image of only the driver D captured by the imaging device 1 is output to the occupant monitoring device 2. The occupant monitoring device 2 generates monitoring information based on a captured image of only the driver D, and provides it to the outside.

For example, a driver who is the owner of the vehicle 10 and who is aware that monitoring information based on a captured image of the inside of the vehicle may be provided to the outside, may ask the driver to inform the driver that the monitoring information may be provided to the outside. If the consent of the passenger S is not obtained, the personal information of the passenger S may be provided to outside parties even though the consent of the passenger S has not been obtained, so the shielding unit 13 All you have to do is close it.

Note that in the examples shown in FIGS. 7 and 8, the vehicle 10 is a right-hand drive vehicle, but this is only an example. Vehicle 10 may be a left-hand drive vehicle.

Further, in the first embodiment described above, the imaging device 1 is installed in the center console panel, but this is only an example. The imaging device 1 may be a wide-angle camera, and may be installed so as to be able to image an area where occupants in at least two seats may be present in the vehicle interior. The imaging device 1 is installed in a vehicle interior by fixing the casing 11 to a structure in the vehicle interior.
The imaging device 1 may be installed on a pillar, for example.

In Embodiment 1, when the imaging device 1 is installed on a pillar, the shielding unit 13 shields light from the shielding target region, and the shielding unit 13 shields light from the shielding target region. An example of the imaging range of the imaging device 1 that changes depending on the situation will be described with reference to FIGS. 9 to 10. In the following description, it is assumed that the imaging device 1 is installed on the A pillar. Furthermore, in the following description, it is assumed that, for example, the vehicle 10 is a left-hand drive vehicle, and the imaging device 1 is installed on a pillar on the passenger seat side, and images a subject area from the pillar on the passenger seat side. Further, for example, it is assumed that the subject area includes a driver seat corresponding area and a passenger seat corresponding area, and the rear seat corresponding area is not included in the subject area. That is, it is assumed that the imaging device 1 has an angle of view that can image a driver who may be present in the driver's seat and a passenger who may be present in the front passenger seat, but cannot image a rear seat occupant. Further, for example, it is assumed that the region to be shielded is determined in advance to be the region corresponding to the passenger seat.

9A and 10A are views of the interior of the vehicle 10 viewed from above. Note that in FIGS. 9A and 10A, for ease of explanation, only the front seat portion of the vehicle interior is shown, and illustrations of other parts than the imaging device 1 and the occupants in the vehicle interior are omitted.
In FIGS. 9A and 10A, a driver and a front passenger are riding in the vehicle 10, the driver is indicated by "D", the front passenger is indicated by "S", and the steering wheel is indicated by "H". It is shown.
9B is a perspective view of the imaging device 1 shown in FIG. 9A, and FIG. 10B is a perspective view of the imaging device 1 shown in FIG. 10A.
In FIGS. 9B and 10B, illustration of the top surface portion 111, back surface portion 114, and

side portions

112 and 113 of the housing 11 is omitted for ease of explanation.

9A and 9B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding part 13 is opened in the imaging device 1.
In FIGS. 9A and 9B, the imaging range of the imaging device 1 is indicated by "E7".

As shown in FIGS. 9A and 9B, since the shielding section 13 is open in the imaging device 1, the shielding section 13 does not cover the light-receiving surface of the front panel 12. In other words, the imaging device 1 does not block light that enters the imaging unit 15 from the subject area.
In this case, the imaging device 1 images the driver's seat corresponding area and the passenger seat corresponding area, and the driver D and the passenger S are captured in the image taken by the imaging device 1. This means that the imaging device 1 has selected the entire area of the subject area as the area to be imaged.
Then, captured images of the driver D and the passenger S in the passenger seat S are captured by the imaging device 1 and are output to the occupant monitoring device 2 . The occupant monitoring device 2 generates monitoring information based on captured images of the driver D and the passenger S in the passenger seat, and provides it to the outside.

10A and 10B are diagrams for explaining an example of the imaging range of the imaging device 1 when the shielding section 13 in the imaging device 1 is closed.
In FIGS. 10A and 10B, the imaging range of the imaging device 1 is indicated by "E7".

As shown in FIGS. 10A and 10B, since the shielding section 13 is closed in the imaging device 1, the shielding section 13 covers the shielding target surface of the light receiving surface of the front panel 12. In other words, the imaging device 1 blocks light that enters the imaging unit 15 from the area corresponding to the passenger seat in the subject area.
In this case, the imaging device 1 images the area corresponding to the driver's seat, and only the driver D is captured in the captured image captured by the imaging device 1. That is, the passenger S is not captured in the captured image captured by the imaging device 1 . The imaging device 1 excludes the area corresponding to the passenger seat from the subject area, and partially selects the area to be imaged.
Then, a captured image of only the driver D captured by the imaging device 1 is output to the occupant monitoring device 2. The occupant monitoring device 2 generates monitoring information based on a captured image of only the driver D, and provides it to the outside.

In addition, in the example shown in FIG. 9 and FIG. 10, the imaging device 1 is installed in the pillar on the passenger seat side, but this is only an example. The imaging device 1 may be installed on a pillar on the driver's seat side.
Further, in the examples shown in FIGS. 9 and 10, the vehicle 10 is a left-hand drive vehicle, but this is only an example. Vehicle 10 may be a right-hand drive vehicle.

Furthermore, in the first embodiment described above, information indicating a region of the subject area that can be covered by the shielding section 13, that is, a region to be shielded, may be drawn on the shielding section 13.
For example, in the example explained using FIGS. 3 to 6 above, the first shielding handle 131a has "passenger seat" and the second shielding handle 132a has "rear seat". Characters may be drawn. By drawing the information indicating the shielding target area on the shielding part 13, the occupant can, for example, know which shielding part 13 to close and which one to close, even if the imaging device 1 is equipped with a plurality of shielding parts 13. It is possible to intuitively understand whether the area corresponding to the seat will no longer be imaged.

Furthermore, in the first embodiment described above, the occupant manually opens and closes the shielding part 13 of the imaging device 1, but this is only an example. For example, in the imaging device 1, the shielding section 13 may be able to be opened and closed automatically.
In this case, for example, the imaging device 1 includes a motor that transmits rotational force as a driving force in the opening/closing direction to the shielding section 13 and moving the shielding section 13 in the opening/closing direction, and a control section that controls the rotation of the motor. For example, the occupant operates an input device such as a touch panel display mounted on the vehicle to instruct which shielding portion 13 he or she wishes to close or open. The control unit receives instructions input by the occupant, and rotates the motor to open or close the shielding unit 13 according to the received instructions.

An occupant monitoring device that monitors an occupant based on a captured image captured by the imaging device 1 according to the first embodiment will be described.
FIG. 11 is a diagram showing a configuration example of the occupant monitoring device 2 according to the first embodiment.
The imaging device 1 and the occupant monitoring device 2 constitute an occupant monitoring system 100.
The occupant monitoring device 2 is mounted on a vehicle and connected to an output device 3. Note that the output device 3 may be included in the occupant monitoring system 100.

The output device 3 includes, for example, a storage device, a server, or a display device.
The storage device stores monitoring information output from the vehicle, specifically, the occupant monitoring device 2.
The storage device is mounted on a vehicle, for example. Note that this is just an example, and the storage device may be connected to the occupant monitoring device 2 outside the vehicle, for example. Further, the storage device may be included in the occupant monitoring device 2, for example.
The storage device is assumed to be, for example, a nonvolatile memory. Note that this is just an example, and the storage device may be any device that can store the monitoring information output by the occupant monitoring device 2.

The server is an external server provided outside the occupant monitoring device 2, and acquires monitoring information output from the vehicle, specifically, the occupant monitoring device 2 outside the vehicle, and performs calculations based on the acquired monitoring information. , or a computing device that stores acquired monitoring information. For example, the server is a personal computer located outside the vehicle.
The calculations performed by the server based on the acquired monitoring information include, for example, image processing of captured images included in the monitoring information, processing of the monitoring information, and calculation of the driver's driving from the past to the present based on the monitoring information. This is an analysis of the number of times of carelessness, etc. Note that the occupant monitoring device 2 can also output the monitoring information in association with information that allows identification of the occupant.
Further, the server can transmit the stored monitoring information or calculation results based on the monitoring information to a vehicle or an in-vehicle device, a personal computer or a mobile device owned by an individual, or the like, for example, by communication. A vehicle or in-vehicle device, or a personal computer or mobile device owned by an individual can receive monitoring information or calculation results based on the monitoring information from a server, and can perform various processes.
Note that the storage device and the server may be a common device.

The display device displays monitoring information output from the vehicle, specifically, the occupant monitoring device 2.
In the first embodiment, the display device is assumed to be, for example, a touch panel display included in a navigation device (not shown) installed in a vehicle. Note that this is just an example, and the display device may be, for example, a variety of display devices installed in a location other than the navigation device in a vehicle, or a display device installed in a personal computer or mobile device owned by an individual. It may also be a display device.

The occupant monitoring device 2 includes an image acquisition section 21, a monitoring section 22, and an output section 23.

The image acquisition unit 21 acquires a captured image captured by the imaging device 1.
The image acquisition unit 21 outputs the acquired captured image to the monitoring unit 22.

The monitoring unit 22 monitors the occupant based on the captured image acquired by the image acquisition unit 21.
For example, the monitoring unit 22 performs various image processing on the captured image acquired by the image acquisition unit 21 to generate monitoring information, and outputs the generated monitoring information to the output device 3.
Specifically, the monitoring unit 22 collects occupant characteristic information indicating body characteristics of the occupant, biological information of the occupant, personal identification information identifying the occupant, and attribute information of the occupant based on the captured image acquired by the image acquisition unit 21. Alternatively, at least one piece of occupant status information indicating the status of the occupant is generated as monitoring information, and the generated monitoring information is output to the output device 3. Specifically, the monitoring unit 22 stores the generated monitoring information in a storage device, outputs it to a server, or displays it on a display device.
When a plurality of occupants are captured in the captured image, the monitoring unit 22 can generate monitoring information for each occupant.

In detail, the occupant characteristic information indicating the body characteristics of the occupant includes, for example, information regarding the occupant's face or facial parts, information indicating the position of the occupant's head or body, information regarding the occupant's physique, or information about the occupant's movement. Contains information about. The occupant's face or facial parts are, for example, the occupant's face or facial parts on the captured image. The position of the occupant's head or body is, for example, the position of the occupant's head or body in real space. The movement of the occupant is, for example, the movement of the position of the occupant's head on the captured image.
For example, the monitoring unit 22 performs known image recognition processing on the captured image to detect the face or facial parts of the occupant. The monitoring unit 22 may detect the occupant's face or facial parts using, for example, a method using machine learning. Specifically, the monitoring unit 22 inputs the captured image to a trained machine learning model that receives the captured image and outputs information regarding the face or facial parts of the occupant in the captured image, and Get information about your face or facial parts. The monitoring unit 22 may detect the face or facial parts of the occupant on the captured image, for example, by pattern matching.

In addition, the monitoring unit 22 may perform a known image recognition process on the captured image to create a trained machine learning model that receives the captured image as input and outputs information indicating the position of the occupant's head on the captured image. The position of the occupant's head on the captured image, in other words, the two-dimensional position of the occupant's head is detected by inputting the captured image or by pattern matching. In addition, the monitoring unit 22 detects the positions of the occupant's eyes on the captured image by the method of detecting facial parts on the captured image as described above, and based on the distance between the occupant's eyes on the captured image, The depth distance from the device 1 to the position of the occupant's head is estimated. Then, the monitoring unit 22 detects the position of the passenger's head in real space based on the detected two-dimensional position of the passenger's head and the estimated depth distance from the imaging device 1 to the position of the passenger's head.

In addition, the monitoring unit 22 calculates the positional relationship between the structure in the vehicle interior and the body of the occupant by performing a known image recognition process on the captured image, and based on the calculated positional relationship, the monitoring unit 22 performs a known image recognition process on the captured image. Detect body position. The structure inside the vehicle interior is, for example, a window frame or a pillar. Since the positional relationship in real space between the structure inside the vehicle interior and the imaging device 1 is known in advance, the monitoring unit 22 can detect the positional relationship between the structure inside the vehicle interior and the occupant's body in the captured image. The position of the occupant's body in real space can be estimated.

Furthermore, the monitoring unit 22 determines the occupant's physique based on the height of the occupant's face in the captured image, for example. Note that this is just an example, and the monitoring unit 22 may determine the occupant's physique from the target image using a known image-based physique determination technique.

Further, the monitoring unit 22 stores, for example, the captured images acquired by the image acquisition unit 21 in chronological order, and performs known image recognition processing on the stored time-series captured images to It is sufficient to detect the movement of

Specifically, the occupant's biological information includes, for example, information regarding the occupant's pulse rate.
The monitoring unit 22 may calculate the pulse rate of the passenger using, for example, a known technique that estimates the pulse rate by utilizing minute changes in pixel values of the face image due to blood flow.

In detail, the personal identification information for identifying the occupant includes, for example, information regarding the occupant's name or a facial image.
The monitoring unit 22 uses, for example, a known technique for identifying individuals to perform a known image recognition process on personal authentication information including pre-registered human facial features and the like, and a target image. The occupant may be identified by comparing the facial features of the occupant detected.

In detail, the occupant attribute information includes, for example, information regarding whether the occupant is an adult or a child, or information regarding gender.
For example, the monitoring unit 22 performs known image recognition processing on the captured image to detect the attributes of the occupant. The monitoring unit 22 may detect the attributes of the occupant using a method using machine learning, for example. Specifically, the monitoring unit 22 inputs the captured image to a trained machine learning model that receives the captured image and outputs information regarding the attribute of the occupant on the captured image, and outputs information regarding the attribute of the occupant on the captured image. get. The monitoring unit 22 may detect the attributes of the occupant on the captured image, for example, by pattern matching.

Specifically, the information indicating the state of the occupant includes, for example, information indicating whether the occupant is inattentive or in a state where the alertness level is decreased. The information indicating the condition of the occupant may be, for example, information indicating the occupant's happiness, anger, sorrow, or happiness, information indicating whether the occupant is smoking, information indicating whether the occupant is eating or drinking, or , information indicating whether or not a call is being made using a mobile phone or the like.
The monitoring unit 22 detects the condition of the occupant by, for example, performing a known image recognition process on the captured image. The monitoring unit 22 may detect the condition of the occupant using a method using machine learning, for example. Specifically, the monitoring unit 22 inputs the captured image to a trained machine learning model that receives the captured image and outputs information indicating the condition of the occupant in the captured image, and then inputs the captured image to a trained machine learning model that outputs information indicating the condition of the occupant in the captured image. Get the information shown. The monitoring unit 22 may detect the state of the occupant on the captured image, for example, by pattern matching.

Note that, for example, the monitoring unit 22 first determines whether or not the occupant's face is captured in the captured image, and when it is determined that the occupant's face is captured, the monitoring unit 22 determines whether or not the occupant's body is captured based on the captured image. The monitoring information may include at least one of occupant characteristic information indicating the characteristics of the occupant, biometric information of the occupant, personal identification information identifying the occupant, attribute information of the occupant, and occupant status information indicating the condition of the occupant. good.
Since the example of the method for detecting the face of the passenger based on the captured image has already been explained, a redundant explanation will be omitted. For example, there may be cases where the captured image does not include the occupant. If no occupant is captured in the captured image, it is assumed that the monitoring unit 22 does not need to generate monitoring information. Therefore, the monitoring unit 22 may first determine whether or not the passenger's face is captured in the captured image. Thereby, the monitoring unit 22 can determine the presence or absence of an occupant and not perform unnecessary processing for generating monitoring information when an occupant is not present, thereby reducing the processing load.

After generating the monitoring information, the monitoring unit 22 stores the generated monitoring information in a storage device, outputs it to a server, or displays it on a display device. For example, the monitoring unit 22 includes the captured image in the monitoring information.

For example, the monitoring unit 22 may directly output the captured image acquired from the imaging device 1 to the output device 3 as monitoring information. The monitoring unit 22 stores the captured image as monitoring information in a storage device, outputs it to a server, or displays it on a display device.

The output unit 23 outputs the monitoring information output from the monitoring unit 22 to the output device 3.

The operation of the occupant monitoring device 2 according to the first embodiment will be explained.
FIG. 12 is a flowchart for explaining the operation of the occupant monitoring device 2 according to the first embodiment.

The image acquisition unit 21 acquires a captured image captured by the imaging device 1 (step ST1).
The image acquisition unit 21 outputs the acquired captured image to the monitoring unit 22.

The monitoring unit 22 monitors the occupant based on the captured image acquired by the image acquisition unit 21 in step ST1 (step ST2).
The monitoring unit 22 generates monitoring information and outputs the generated monitoring information to the output unit 23.

The output unit 23 outputs the monitoring information output from the monitoring unit 22 in step ST2 to the output device 3 (step ST3).

In this way, the occupant monitoring device 2 according to the first embodiment acquires a captured image in which the imaging device 1 captures a range in which occupants in at least two seats can exist in the cabin of a vehicle, and uses the captured image in the acquired captured image. Perform crew monitoring based on The occupant monitoring device 2 then generates monitoring information based on the captured image, and outputs the generated monitoring information to the output device 3.

As described above, the imaging device 1 is configured to be able to partially select the region to be imaged from among the imageable regions (subject regions). As a result, the imaging device 1 can, for example, be used by passengers who have not agreed that information (monitoring information) based on captured images of themselves may be provided to outside parties, or who have not been imaged in the first place. An occupant who is not aware that information based on the captured image that is taken can be provided to the outside can output the captured image that is not captured.
Therefore, even if the occupant monitoring device 2 generates monitoring information based on the captured image captured by the imaging device 1, the monitoring information is not provided to the outside, for example, based on the captured image of the occupant monitoring device 2. This does not constitute monitoring information for passengers who have not agreed to be allowed to be monitored, or who are not aware that monitoring information based on images of themselves may be provided to an external party. Therefore, the occupant monitoring device 2 can prevent the personal information of the vehicle occupant from being unintentionally provided to a third party.

13A and 13B are diagrams showing an example of the hardware configuration of the occupant monitoring device 2 according to the first embodiment.
In the first embodiment, the functions of the image acquisition section 21, the monitoring section 22, and the output section 23 are realized by the processing circuit 1001. That is, the occupant monitoring device 2 includes a processing circuit 1001 that generates monitoring information regarding the occupant based on a captured image captured by the imaging device 1, which is a wide-angle camera, and performs control to output the generated monitoring information.
The processing circuit 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 13B.

When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Circuit). Gate Array), or a combination of these.

When the processing circuit is the processor 1004, the functions of the image acquisition section 21, the monitoring section 22, and the output section 23 are realized by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 1005. The processor 1004 executes the functions of the image acquisition unit 21, the monitoring unit 22, and the output unit 23 by reading and executing a program stored in the memory 1005. That is, the occupant monitoring device 2 includes a memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST1 to ST3 in FIG. 12 described above. It can also be said that the program stored in the memory 1005 causes the computer to execute the processing procedures or methods of the image acquisition unit 21, the monitoring unit 22, and the output unit 23. Here, the memory 1005 is, for example, RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically non-volatile or volatile This includes semiconductor memory, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versatile Discs), and the like.

Note that some of the functions of the image acquisition section 21, the monitoring section 22, and the output section 23 may be realized by dedicated hardware, and some may be realized by software or firmware. For example, the function of the image acquisition unit 21 is realized by a processing circuit 1001 as dedicated hardware, and the processor 1004 reads and executes a program stored in a memory 1005 for the monitoring unit 22 and output unit 23. It is possible to realize this function by
The occupant monitoring device 2 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with a device such as the imaging device 1 or the output device 3.

Further, in the first embodiment described above, the occupant monitoring device 2 is an in-vehicle device mounted on a vehicle, and the image acquisition unit 21, the monitoring unit 22, and the output unit 23 are provided in the in-vehicle device.
However, the present invention is not limited to this, and some of the image acquisition unit 21, monitoring unit 22, and output unit 23 may be installed in the on-vehicle device of the vehicle, and the others may be provided in a server connected to the in-vehicle device via a network. Thus, the occupant monitoring system 100 may be configured by the in-vehicle device, the server, and the imaging device 1. Further, the image acquisition section 21, the monitoring section 22, and the output section 23 may all be included in the server. Note that the server here may be a server common to the output device 3, or may be a server different from the output device 3.

As described above, according to Embodiment 1, the imaging device 1 has a housing 11 and a front panel 12 that is provided inside the housing 11 and forms one surface of the housing 11 on the subject area side. It is configured to include an imaging section 15 that guides light incident from the subject area, and a shielding section 13 that can partially block the light that enters the imaging section 15 from the subject area on the front panel 12.
Therefore, the imaging device 1 can partially select the area to be imaged from among the imageable areas.

According to Embodiment 1, when the imaging device 1 is installed in a vehicle and the subject area includes the passenger seat corresponding area corresponding to the passenger seat, the imaging device 1 is configured such that the shielding unit 13 protects the subject area from the subject area. Of the light that enters the imaging section 15, the light that enters the imaging section 15 from the area corresponding to the passenger seat can be shielded.
Therefore, for example, a passenger in the front seat may not have agreed that information (monitoring information) based on an image of himself or herself may be provided to an outside party, or a passenger who has not been imaged in the first place. If the passenger is not aware that information based on the captured image in which the passenger is present can be provided to the outside, the imaging device 1 can output a captured image that does not include the front passenger seat occupant. As a result, the imaging device 1 can prevent the passenger's personal information from being unintentionally provided to a third party.

According to Embodiment 1, when the imaging device 1 is installed in a vehicle and the subject area includes a rear seat corresponding area corresponding to the rear seat, the imaging device 1 is configured so that the shielding unit 13 separates the subject area from the subject area. Of the light that enters the imaging section 15, the light that enters the imaging section 15 from the rear seat corresponding area can be shielded.
Therefore, for example, a rear seat passenger may not have agreed that information (monitoring information) based on the captured image of himself or herself may be provided to an outside party, or he or she may not have been imaged in the first place. If the occupant is not aware that information based on the captured image of the rear seat occupant may be provided to the outside, the imaging device 1 can output a captured image that does not include the rear seat occupant. As a result, the imaging device 1 can prevent the personal information of the rear seat occupant from being unintentionally provided to a third party.

According to the first embodiment, when the imaging device 1 is installed in a vehicle and the subject area includes a passenger seat corresponding area corresponding to the passenger seat and a rear seat corresponding area corresponding to the rear seat, the imaging device 1 In the above, the shielding section 13 includes a first shielding section 131 and a second shielding section 132 that can partially shield the light incident on the imaging section 15 from the subject area, independently of each other, and the first shielding section 131 includes: Of the light that enters the imaging unit 15 from the subject area, the second shielding unit 132 can block the light that enters the imaging unit 15 from the passenger seat corresponding area. , can be configured to be able to block light entering the imaging unit 15 from the rear seat corresponding area.
Therefore, for example, if at least one of the rear seat occupant or the front passenger seat occupant does not agree that information (monitoring information) based on the captured image of himself/herself may be provided to the outside, Alternatively, if the passenger is not aware that the information based on the captured image of the passenger himself/herself may be provided to an outside party, the imaging device 1 may output the captured image without the above-mentioned consent. Alternatively, the image may be a captured image in which the front passenger seat occupant or the rear seat occupant who has not been recognized as described above is not captured. As a result, the imaging device 1 is able to prevent the personal information of the front passenger or rear seat passenger for whom the above-mentioned consent has not been obtained or the above-mentioned recognition has not been obtained from being unintentionally provided to a third party. can.

According to Embodiment 1, the imaging device 1 can be configured such that information indicating a region of the subject area that can be shielded by the shielding section 13 is drawn on the shielding section 13.
Therefore, for example, even if a plurality of shielding parts 13 are provided, the imaging device 1 can intuitively tell the occupant which shielding part 13 should be closed to prevent the area corresponding to which seat from being imaged. can be made to understand.

Further, according to the first embodiment, the occupant monitoring system 100 includes the imaging device 1 as described above, the image acquisition unit 21 that acquires the captured image captured by the imaging device 1, and the image captured by the image acquisition unit 21. The vehicle is configured to include a monitoring section 22 that monitors the occupant based on images, and an output section 23 that outputs monitoring information based on the monitoring performed by the monitoring section 22.
For this reason, the occupant monitoring system 100 is designed to prevent occupants from obtaining consent that information (monitoring information) based on captured images of themselves may be provided to outside parties, or those who are themselves being imaged in the first place. A passenger who is not aware that information based on a captured image can be provided to the outside can output monitoring information based on a captured image that has not been captured. As a result, the occupant monitoring system 100 can prevent the personal information of the vehicle occupant from being unintentionally provided to a third party.

Further, according to the first embodiment, the vehicle is configured to include the imaging device 1 as described above. Therefore, the vehicle can prevent the personal information of the occupant of the vehicle from being unintentionally provided to a third party.

Note that in the present disclosure, any component of the embodiments can be modified or any component of the embodiments can be omitted.

The imaging device according to the present disclosure can partially select the region to be imaged from among the regions that can be imaged, and as a result, personal information of vehicle occupants may be unintentionally provided to a third party. This can be prevented.

1 Imaging device, 11 Housing, 111 Top surface section, 112, 113 Side surface section, 114 Back section, 1112 First guide groove, 1112a First guide groove for first shielding section, 1112b First guide groove for second shielding section, 1113 first stopper, 1114 second stopper, 12 front panel, 13 shielding part, 131 first shielding part, 132 second shielding part, 131a first shielding part handle part, 132a second shielding part handle part, 13b second shielding part 1 protrusion, 131b first protrusion for first shielding part, 132b first protrusion for second shielding part, 13c second protrusion, 14 light projecting part, 141 LED, 142 substrate, 15 imaging part, 151 lens, 152 Image sensor, 16 Screw, 17 Fixing member, 2 Occupant monitoring device, 21 Image acquisition unit, 22 Monitoring unit, 23 Output unit, 100 Occupant monitoring system, 10 Vehicle, 1001 Processing circuit, 1002 Input interface device, 1003 Output interface device , 1004 processor, 1005 memory.

Claims

An imaging device capable of imaging a subject area in which occupants in at least two seats may be present in a vehicle interior, the imaging device comprising:
A casing and
an imaging unit that is provided in the housing and guides light incident from the subject area through a front panel that forms one surface of the housing on the subject area side;
An imaging device comprising: a shielding section that is capable of partially shielding light that enters the imaging section from the subject area in the front panel.
When installed in the vehicle, the subject area includes a passenger seat corresponding area corresponding to a passenger seat,
The imaging device according to claim 1, wherein the shielding section is capable of blocking light that enters the imaging section from the passenger seat corresponding area, out of the light that enters the imaging section from the subject area.
When installed in the vehicle, the subject area includes a rear seat corresponding area corresponding to a rear seat,
The imaging device according to claim 1, wherein the shielding section is capable of blocking light that enters the imaging section from the rear seat corresponding area, out of the light that enters the imaging section from the subject area.
When installed in the vehicle, the subject area includes a passenger seat corresponding area corresponding to a passenger seat and a rear seat corresponding area corresponding to a rear seat,
The shielding section includes a first shielding section and a second shielding section that can partially shield light entering the imaging section from the subject area, independently of each other;
The first shielding unit is capable of blocking light that enters the imaging unit from the passenger seat corresponding area, out of the light that enters the imaging unit from the subject area,
The imaging device according to claim 1, wherein the second shielding unit is capable of blocking light that enters the imaging unit from the rear seat corresponding area, out of the light that enters the imaging unit from the subject area. Device.
The imaging device according to claim 3 or 4, wherein the rear seat corresponding area is a central area of a rear seat of the vehicle.
The imaging device according to any one of claims 1 to 5, wherein information indicating a region of the subject area that can be shielded by the shielding section is drawn on the shielding section. .
The imaging device according to any one of claims 1 to 6, wherein the casing is fixed to a central portion in the width direction of the vehicle in a center console panel provided in the vehicle. Device.
The imaging device according to any one of claims 1 to 6, wherein the housing is fixed to a pillar provided on the vehicle.
An imaging device according to any one of claims 1 to 8,
an image acquisition unit that acquires a captured image captured by the imaging device;
a monitoring unit that monitors the occupant based on the captured image acquired by the image acquisition unit;
An occupant monitoring system comprising: an output section that outputs monitoring information based on the monitoring performed by the monitoring section.
A vehicle comprising the imaging device according to any one of claims 1 to 8.
The vehicle according to claim 10, wherein the imaging device is provided on a center console panel or a pillar.