WO2021172038A1 - Information processing device, information processing method, and program - Google Patents

Abstract

The present technology relates to an information processing device, an information processing method, and a program with which it is possible to reduce a processing burden due to unnecessary information. On the basis of the state of equipment provided inside a vehicle, a sensing range of an observation sensor that observes a user using the equipment is set. The present technology is applicable to automobiles, trains, restaurants, theaters, etc.

Description

Information processing equipment, information processing methods, and programs

This technology relates to information processing devices, information processing methods, and programs, and in particular, to information processing devices, information processing methods, and programs designed to reduce the processing burden of unnecessary information.

Patent Document 1 discloses a technique of detecting the number of passengers based on an image of the interior of a vehicle taken by a camera and optimizing a method of editing an image of the interior of the vehicle according to the number of passengers. Patent Document 2 discloses a technique for presenting information according to the number of people in a vehicle interior and the like.

Japanese Unexamined Patent Publication No. 2019-87825 Japanese Unexamined Patent Publication No. 2018-167737

When analyzing the information acquired by the sensor, if the amount of information is large, the processing load will increase. It is desirable to reduce unnecessary information as much as possible.

This technology was made in view of such a situation, and makes it possible to reduce the processing burden of unnecessary information.

The information processing device or program of the present technology is an information processing device having a processing unit that sets a sensing range of an observation sensor that observes a user who uses the equipment based on the state of equipment provided in the vehicle. Or, it is a program for operating a computer as such an information processing device.

In the information processing method of the present technology, the processing unit of the information processing device having the processing unit sets the sensing range of the observation sensor for observing the user who uses the equipment based on the state of the equipment provided in the vehicle. This is the information processing method to be set.

In this technology, the sensing range of the observation sensor that observes the user who uses the equipment is set based on the state of the equipment installed in the vehicle.

It is a block diagram which showed the structural example of one Embodiment of the information processing system to which this technology is applied. It is a block diagram which showed the functional structure example of an information processing apparatus. It is a flowchart which showed the processing example which a processing unit 41 carries out. It is a flowchart which showed the processing example of the sensing setting. It is a figure which illustrated the arrangement of a seat and an occupant observation sensor. It is a figure explaining the sensing range of the occupant observation sensor 62 in Example 1. FIG. It is a figure explaining the sensing range of the occupant observation sensor 62 in Example 1. FIG. It is a figure which illustrated the depth image (observation information). It is a figure which illustrated the depth image (observation information). It is a flowchart which showed the processing example performed by the sensing setting unit 44 in Example 1. FIG. It is a flowchart which showed the processing example when the detection frequency is changed according to a sensing range. It is a figure explaining the sensing range in Example 2. FIG. It is a figure which illustrated the depth image (observation information). It is a flowchart which showed the processing example performed by the sensing setting unit 44 in Example 2. FIG. It is a figure explaining the sensing range in Example 3. FIG. It is a figure explaining the sensing range in the state which the seat belt is worn with respect to the state of FIG. It is a figure explaining the sensing range in Example 4. FIG. It is a figure which illustrated the observation information (depth image). It is a flowchart which showed the processing example performed by the sensing setting unit 44 in Example 4. FIG. It is a figure explaining the arrangement of the occupant observation sensor in Example 5. It is a figure which showed the sensing range of the occupant observation sensor 161 to 163. It is a figure which showed the sensing range when the seat is displaced greatly. It is a figure explaining the arrangement of the occupant observation sensor in Example 6. It is a figure which showed the sensing range of the occupant observation sensor of FIG. It is a figure which showed the sensing range when the seat is largely displaced from a standard position. It is a figure explaining the arrangement of the occupant observation sensor in Example 7. It is a figure explaining the arrangement of the occupant observation sensor in Example 7. It is a figure explaining the case where the arrangement of seats is changed. It is a figure explaining the arrangement of the occupant observation sensor in Example 8. It is a figure explaining the case where the arrangement of seats is changed. It is a figure explaining the arrangement of the occupant observation sensor in Example 9.

Hereinafter, embodiments of the present technology will be described with reference to the drawings.

<< An embodiment of an information processing system to which this technology is applied >>

In FIG. 1, the information processing system 11 detects, for example, the state of an occupant in a vehicle, and performs processing or control related to various devices or equipment of the vehicle in response to the state of the occupant.

In FIG. 1, the information processing system 11 includes an information processing device 21, an in-vehicle system 22, and a sensor 23.

The information processing device 21 mainly detects the state of the occupant and executes processing (response processing) corresponding to the state of the occupant. The response process includes a process of instructing a part of the control unit of the vehicle-mounted system 22 mounted on the automobile to execute a predetermined process or control.

Note that the information processing device 21 of FIG. 1 is exemplified as a configuration including a function of executing a predetermined application such as moving image playback designated by an occupant and displaying the presentation information of the application on a display or the like in the vehicle. Therefore, the response processing performed by the information processing apparatus 21 includes the response processing of the application executed by the information processing apparatus 21.

The in-vehicle system 22 includes a control system, a body system, and an information system control unit (ECU: Electronic Control Unit) included in the automobile, and is a system in which each control unit is communicably connected via an in-vehicle network. The control system performs engine control, steering control, brake control, and the like. The body system performs seat control, door control, mirror control, air conditioner control, and the like. The information system is composed of an audio system, a navigation system, a back monitor, and the like. A part of the control unit of the in-vehicle system 22 has a function of executing a predetermined control according to an instruction from the information processing device 21.

The sensor 23 is a sensor installed in an automobile for the purpose of being used by the in-vehicle system 22. The sensor 23 represents an arbitrary sensor used in the in-vehicle system 22, and is not limited to a specific type and a specific number.

The information processing device 21 includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a ROM (Read Only Memory) 33, an input unit 34, an output unit 35, a storage unit 36, a sensor 37, and a communication unit 38. do. The CPU 31, RAM 32, and ROM 33 are connected to each other by a bus 39.

An input / output interface 40 is further connected to the bus 39. An input unit 34, an output unit 35, a storage unit 36, a sensor 37, and a communication unit 38 are connected to the input / output interface 40.

The CPU 31 loads and executes, for example, the program stored in the ROM 33 or the storage unit 36 into the RAM 32 via the bus 39 or via the input / output interface 40 and the bus 39. By executing the program, the CPU 31 executes the sensing setting process, the occupant state detection process, the operation recognition process, the operation response process, and the like, which will be described later.

RAM 32 is a volatile memory and temporarily stores data such as programs processed by CPU 31.

The ROM 33 is a non-volatile memory, and stores data such as a program loaded in the RAM 32 and executed by the CPU 31.

The input unit 34 includes a switch, a microphone, and the like. The data input from the input unit 34 is supplied to the CPU 32 via the input / output interface 40 and the bus 39.

The output unit 35 includes a projector, a display, a speaker, and the like. The projector is installed in the vehicle and projects image information, text information, and the like on the ceiling, headrest, door glass, windshield, and the like. The output unit 35 may include one or more projectors that project information onto one or more parts.

The display may be an image display (liquid crystal display, organic EL (Electro-Luminescence), etc.) installed on a dashboard, ceiling, headrest, door glass, windshield, or the like.

Note that the information processing device 21 of FIG. 1 does not have to have a function of displaying information, and may not have an output unit 35. Further, the function of displaying information may be included in the in-vehicle system 22.

The storage unit 36 includes a hard disk, a non-volatile memory, and the like. The storage unit 36 stores an application program executed by the CPU 31, data referenced when an arbitrary program is executed by the CPU 31, and the like.

The sensor 37 is an occupant observation sensor (described later) that observes an occupant (equipment user) and a seat-related sensor (described later) that detects the state of each seat (equipment) of the automobile. The sensor 37 is representative of the sensor included in the information processing device 21, and is not limited to a specific type and a specific number. The data acquired by the sensor 37 is supplied to the CPU 31 via the input / output interface 40 and the bus 39.

The communication unit 38 includes a network interface and the like. The communication unit 38 is connected to the in-vehicle system 22 so as to be able to communicate by wire or wirelessly. The CPU 31 supplies an instruction signal or the like instructing the execution of a predetermined process or control to the control unit included in the in-vehicle system 22 via the communication unit 38 connected to the communication.

Further, the CPU 31 can acquire the data obtained by the sensor 23 used in the in-vehicle system 22 via the in-vehicle system 22 and the communication unit 38 which are connected by communication. Therefore, the information processing device 21 can use the sensor 23 used in the in-vehicle system 22.

The communication unit 38 has a communication interface such as a wireless LAN (Local Area Network), Bluetooth (registered trademark), and mobile communication, and can be connected to a terminal such as a smart phone or the Internet. May be good.

FIG. 2 is a block diagram showing a functional configuration example of the information processing device 21 that functionally represents the information processing device 21 of FIG.

The information processing device 21 of FIG. 2 has an output unit 35, a processing unit 41, a seat-related sensor 42, and an occupant observation sensor 43.

The processing unit 41 sets the range observed by the occupant observation sensor 43 (hereinafter referred to as the sensing range) and the like based on the detection information (hereinafter referred to as the seat information) detected by the seat-related sensor 42. Further, the processing unit 41 detects the state of the occupant based on the detection information (hereinafter referred to as observation information) detected by the occupant observation sensor 43, and executes the response processing corresponding to the state of the occupant. The processing in the processing unit 41 will be described later.

The seat-related sensor 42 is a sensor provided as the sensor 23 or the sensor 37 in FIG. 1, and is a sensor that detects the state of the seat. The condition of the seat also includes the condition of ancillary equipment (reclining mechanism, seat belt, steering wheel, etc.) related to the seat. Therefore, the seat-related sensor 42 includes a seating sensor, a reclining sensor, a seatbelt sensor, a seat position sensor, a handle sensor, and the like provided for each seat. Further, the seat-related sensors 42 are provided for each seat except for the sensors for only some seats, and the processing unit 41 acquires seat information from the seat-related sensors 42 for all seats.

The seat-related sensor 42 is basically a sensor provided in the automobile regardless of the present technology, and the presence / absence, position, orientation, posture, etc. of the occupant are estimated using these sensors. However, the seat-related sensor 42 may be a sensor installed separately from the sensor provided in the automobile. For example, the seat-related sensor 42 may be a motion sensor.

The occupant observation sensor 43 is a sensor provided as the sensor 23 or the sensor 37 in FIG. 1, and is a sensor for observing the occupant. The occupant observation sensor 43 is, for example, an image sensor (camera) or a depth sensor that acquires spatial information. The image sensor may be a visible light camera or an infrared camera. The depth sensor may be a sensor that acquires three-dimensional information (depth information) of the shooting space by a stereo camera, or may be a ToF (time of flight) sensor or the like.

The occupant observation sensor 43 is provided for each row with respect to the arrangement of seats (front row, middle row, rear row, etc.) in the vehicle, and one occupant observation sensor 43 observes occupants of a plurality of seats. A plurality of occupant observation sensors 43 are provided when the seats are arranged in two or more rows, and the processing unit 41 acquires observation information supplied from all the occupant observation sensors 43.

In addition to observing two people with one occupant observation sensor 43, one occupant observation sensor 43 may observe one or three or more people. Further, the occupant observation sensor 43 is not limited to a specific arrangement, and may be arranged for each seat. Further, the occupant observation sensor 43 may observe a part of the occupants in the vehicle.

The occupant observation sensor 43 may be a sensor that acquires physical information (including biological information) such as the occupant's body temperature, heart rate, sleep state, wakefulness state, and emotions.

Since the in-vehicle system 22 and the output unit 35 are as described in FIG. 1, the description thereof will be omitted here.

(Explanation of processing unit 41)
The processing unit 41 includes a sensing setting unit 44, an occupant state detection unit 45, an operation recognition unit 46, and an operation response unit 47.

The sensing setting unit 44 sets the sensing range, detection density, and detection frequency of the occupant observation sensor 43 based on the seat information acquired from the seat-related sensor 42.

The sensing range of the occupant observation sensor 43 indicates the observation range effectively observed by the occupant observation sensor 43 or the range of observation information effectively detected. The sensing range of the occupant observation sensor 43 is set to a range that excludes as much as possible the range in which no occupant exists within the maximum observable observation range of the occupant observation sensor 43. For example, the sensing range becomes smaller as the number of occupants decreases.

The detection density of the occupant observation sensor 43 represents the density of the observation points (detection points) of the observation information effectively detected by the occupant observation sensor 43. For example, when the occupant observation sensor 43 is used as an image sensor, the detection density represents the pixel density of the captured image as observation information.

The detection frequency of the occupant observation sensor 43 represents the frequency of detection effectively performed by the occupant observation sensor 43. When the observation information of the entire sensing range is obtained by the occupant observation sensor 43 as one detection, the detection frequency represents the number of detections per unit time.

For example, the smaller the sensing range of the occupant observation sensor 43, that is, the smaller the amount of information acquired from the occupant observation sensor 43 in one detection, the smaller the detection density and detection frequency of the occupant observation sensor 43. Set one or both of them to a large value. As a result, the accuracy of detecting the occupant state by the occupant state detection unit 45, which will be described later, can be improved without increasing the processing load on the processing unit 41 or the like.

Further, the sensing setting unit 44 sets the operation of each occupant observation sensor 43 based on the set sensing range, detection density, and detection frequency, and changes the processing operation of each occupant observation sensor 43.

The details of the processing of the sensing setting unit 44 will be described later.

The occupant state detection unit 45 acquires the observation information supplied from the occupant observation sensor 43 and also acquires the information of the sensing range set by the sensing setting unit 44.

The occupant state detection unit 45 detects the state of each occupant based on the observation information from each occupant observation sensor 43 and supplies the state to the operation recognition unit 46.

Specifically, the occupant state detection unit 45 extracts the physical feature points of each occupant from the continuously obtained observation information, and detects the positions, orientations, and movements of the hands, arms, eyes, face, and the like. , Detects the behavior of the occupants (gestures, etc.). The occupant state detection unit 45 may detect the facial expression of the occupant and other physical information as the occupant state.

Further, in the occupant state detection unit 45, even when the body of one occupant is divided and observed by a plurality of occupant observation sensors 43, which occupant observation sensor 43 has the observation information for the occupant. It can be grasped from the information of the sensing range from the sensing setting unit 44 whether or not it is divided into the sensing ranges and acquired (described later).

The occupant state detection unit 45 integrates (links) the occupant observation information acquired by being divided into the sensing ranges of the plurality of occupant observation sensors 43 based on the information of the sensing range from the sensing setting unit 44. As a result, the occupant state detection unit 45 can appropriately detect the occupant state even for the occupants who are divided and observed by the plurality of occupant observation sensors 43.

The operation recognition unit 46 recognizes an operation associated with the occupant state detected by the occupant state detection unit 45 in advance.

For example, the occupant state detection unit 45 detects the movement (state) of the occupant who holds his hand between the driver's seat and the passenger seat and draws a circle with his fingers, and the information indicating the occupant's state is operated from the occupant state detection unit 45. It is assumed that it is supplied to the recognition unit 46. The operation recognition unit 46 recognizes, for example, an operation of raising the volume of the audio system included in the in-vehicle system 22 as an operation associated with the state of the occupant in advance.

Further, it is assumed that information indicating the state of the driver with a drowsy expression is supplied from the occupant state detection unit 45 to the operation recognition unit 46. The operation recognition unit 46 recognizes, for example, an operation of generating a warning sound, an operation of decelerating, or an operation of evacuating to the shoulder as an operation associated with the state of the occupant in advance.

The operation recognition unit 46 supplies the operation recognized for the occupant's state from the occupant state detection unit 45 to the operation response unit 47.

The operation response unit 47 executes response processing corresponding to the operation recognized by the operation recognition unit 46. For example, it is assumed that the operation recognition unit 46 recognizes an operation of raising the volume of the audio system as described above, and the operation recognition unit 46 supplies information indicating the operation to the operation response unit 47. The operation response unit 47 gives an instruction to raise the volume to the control unit of the audio system included in the in-vehicle system 22 as a response process corresponding to the operation.

Further, it is assumed that the operation recognition unit 46 recognizes the operation of decelerating as described above, and the information indicating the operation is supplied to the operation response unit 47. The operation response unit 47 gives a deceleration instruction to the vehicle control control unit included in the in-vehicle system 22 as a response process corresponding to the operation.

Further, when the operation recognition unit 46 recognizes the operation for designating the start of the application, the operation response unit 47 starts the designated application, executes the processing of the application, and displays the information presented by the application in FIG. It is supplied to the output unit 35. When an operation for the processing of the application started by the operation recognition unit 46 is detected, the operation response unit 47 executes the processing of the application corresponding to the operation.

FIG. 3 is a flowchart showing a processing example executed by the processing unit 41 of FIG.

In step S11, the sensing setting unit 44 of the processing unit 41 performs the sensing setting process based on the seat information from the seat-related sensor 42. The sensing setting unit 44 sets the sensing range, the detection density, and the detection frequency of the occupant observation sensor 43 by the sensing setting process. The process proceeds from step S11 to step S12.

In step S12, the occupant state detection unit 45 of the processing unit 41 detects the occupant state based on the observation information from the occupant observation sensor 43. The occupant state detection unit 45 supplies the detected occupant state to the operation recognition unit 46. The process proceeds from step S12 to step S13.

In step S13, the operation recognition unit 46 of the processing unit 41 recognizes the operation associated with the occupant state supplied from the occupant state detection unit 45 in step S12. The operation recognition unit 46 supplies the recognized operation to the operation response unit 47. The process proceeds from step S13 to step S14.

In step S14, the operation response unit 47 of the processing unit 41 executes the response processing corresponding to the operation supplied from the operation recognition unit 46 in step S13. The process returns from step S14 to step S11, and repeats step S11 and subsequent steps.

According to the information processing device 21 shown in FIGS. 1 to 3 above, a predetermined operation is automatically recognized with respect to the state of the occupant of the automobile, and response processing corresponding to the recognized operation is performed. .. As a result, the occupant can operate the switches and screens of the navigation system and the audio system, for example, without directly touching them by the movements of his / her hands and arms.

<Details of sensing setting processing of occupant observation sensor 43>
(Setting of sensing range of occupant observation sensor 43)

The details of the sensing setting process of the occupant observation sensor 43 performed by the sensing setting unit 44 of the processing unit 41 of FIG. 2 will be described.

The sensing setting unit 44 sets the sensing range of the occupant observation sensor 43 based on the seat information from the seat-related sensor 42.

Specifically, the sensing setting unit 44 detects the state of each seat based on the seat information from the seat-related sensor 42 installed for each seat in the vehicle.

For example, the seat-related sensor 42 includes a seating sensor, a reclining sensor, a seatbelt sensor, a seat position sensor, and the like.

Then, as the state of the seat, the seating sensor detects whether or not the occupant is seated (seat state or non-seat state). That is, the presence or absence of seating of each seat is detected as a state of whether or not each seat, which is a facility provided in the vehicle, is used. Further, the state of the seat includes the form of the seat, and the reclining state (tilted state) or the non-reclining state (standing state) of the seat is detected by the reclining sensor. In addition, the state of the seat includes the state of ancillary equipment related to the seat, and the seatbelt sensor detects the wearing state or the non-wearing state of the seatbelt. In addition, as the state of the seat, the seat position and the orientation of the seat that can be moved back and forth are detected by the seat position sensor.

Further, the seat-related sensor 42 may include, for example, a handle sensor for detecting whether or not the handle is gripped for the driver's seat or the like.

The position of the occupant existing in the vehicle (spatial range of the occupant's body) is estimated from the state of each seat detected by the seat information from the seat-related sensor 42. For example, when the sensing setting unit 44 detects the state of each seat based on the seat information of only the seating sensor, which is the seat-related sensor 42, when the seating state is detected, the occupant is placed in the peripheral range of the seated portion of the seat. It is presumed to exist. On the contrary, when it is detected that the vehicle is not seated, it is presumed that there is no occupant in the area around the seated portion of the seat. The sensing setting unit 44 may estimate the position of the occupant based on the state of equipment other than the seats provided in the vehicle.

The sensing setting unit 44 uses the partial sensing range to set the sensing range of each occupant observation sensor 43 so that the existence range of each occupant estimated from the state of each seat becomes the sensing range of each occupant observation sensor 43. To set.

Here, the partial sensing range is the observation range when the occupants seated in each seat are individually observed by the occupant observation sensor 43, or the observation information of the occupants seated in each seat is observed by the occupant observation sensor 43. This is the range of observation information when it is determined to be detected individually.

The partial sensing range is predetermined for each seat and for each seat condition and is stored in the storage unit 36 or ROM 33 of FIG. For example, each seat is associated and stored with a partial sensing range for observing an occupant seated in that seat. Further, for example, in the case of the reclining state and the case of the non-reclining state, the partial sensing range of a different range is associated with and stored in each seat state. However, the position of the partial sensing range may be changed according to the position of the seat or the like, and the size of the sensing range may be changed as appropriate.

For example, when the sensing setting unit 44 detects the state of each seat based on the seat information of only the seating sensor, which is the seat-related sensor 42, the sensing setting unit 44 is associated with the seat (seat in the seated state) that has detected that the seat is in the seated state. The partial sensing range is enabled, and the partial sensing range associated with that seat is disabled for non-seat seats.

Then, the sensing setting unit 44 sets the effective partial sensing range of the maximum sensing range of each occupant observation sensor 43 to the sensing range of the occupant observation sensor 43. The maximum sensing range of each occupant observation sensor 43 is a sensing range when all the partial sensing ranges included in the observation range of each occupant observation sensor 43 are enabled. The maximum observable observation range of each occupant observation sensor 43 may be set as the maximum sensing range.

Further, one partial sensing range may be divided into observation ranges of a plurality of occupant observation sensors 43. In that case, when the partial sensing range is set as the sensing range, the corresponding portion of the partial sensing range is the sensing range for each observation range of the plurality of occupant observation sensors 43.

(Setting of detection density and detection frequency of occupant observation sensor 43)
The sensing setting unit 44 sets the detection density and the detection frequency of each occupant observation sensor 43 based on the sensing range set for each occupant observation sensor 43.

Here, the detection density of the occupant observation sensor 43 represents the density of observation points where observation information (observation data) is detected by the occupant observation sensor 43 within the sensing range of the occupant observation sensor 43.

The detection density of the occupant observation sensor 43 is, for example, the pixel density of the captured image acquired by the image sensor when the occupant observation sensor 43 is an image sensor, and when the occupant observation sensor 43 is a depth sensor. It is the pixel density of the depth image acquired by the depth sensor.

The detection frequency of the occupant observation sensor 43 is the number of times the observation information of the entire sensing range of the occupant observation sensor 43 is acquired by the occupant observation sensor 43 per unit time.

The detection frequency of the occupant observation sensor 43 is, for example, the number of frames (frame rate) of the captured image acquired by the image sensor per unit time when the occupant observation sensor 43 is an image sensor, and the occupant observation sensor 43 determines. In the case of a depth sensor, it is the number of frames (frame rate) of the depth image acquired by the depth sensor per unit time.

The higher the detection density or the detection frequency of these occupant observation sensors 43, the higher the detection accuracy of the occupant state detected based on the observation information. On the other hand, the higher the detection density or the detection frequency of the occupant observation sensor 43, the larger the amount of observation information acquired by the occupant observation sensor 43 per unit time, and the information processing device 21 (processing unit 41 (CPU 31). Etc.)), the processing load increases.

On the other hand, there is a certain limit to the processing capacity of the device, not limited to the information processing device 21 mounted on the automobile.

Therefore, the sensing setting unit 44 increases the processing amount in a state in which the information processing device 21 is assumed to require the most processing capacity, that is, in a state in which the sensing ranges of all the occupant observation sensors 43 are set to the maximum sensing range. , The detection density and detection frequency of the occupant observation sensor 43 are set so as not to exceed the limit of the processing capacity of the information processing device 21. The detection density and detection frequency of each occupant observation sensor 43 at this time are referred to as a reference detection density and a reference detection frequency. The reference detection density and the reference detection frequency do not necessarily have to match between the occupant observation sensors 43.

When the number of occupants in the vehicle is less than the maximum number and the sensing range of any of the occupant observation sensors 43 is smaller than the maximum sensing range, the sensing setting unit 44 determines the detection density and detection frequency of the occupant observation sensor 43. At least one of them should be larger than the reference detection density or the reference detection frequency.

For example, when the amount of information in the sensing range of the occupant observation sensor 43 is 1 / C with respect to the amount of information in the maximum sensing range, the detection density is D times the reference detection density and the detection frequency is relative to the reference detection frequency. And assume that it is set to E times. At this time, the sensing setting unit 44 determines D and E so as to satisfy the conditions of D and E = C, and sets the detection density and the detection frequency. For example, when E is 1, the sensing setting unit 44 sets the detection frequency to 1 times the reference detection frequency and the detection density to C times the reference detection density.

According to the above sensing setting process of the sensing setting unit 44, the sensing range of the occupant observation sensor 43 is set to an appropriate range with less waste according to the position of the occupant existing in the vehicle. Further, according to the sensing range of the occupant observation sensor 43, the detection density or the detection frequency of the occupant observation sensor 43 is appropriately set within a range that does not exceed the processing capacity of the information processing device 21.

<Example of sensing setting processing by the sensing setting unit 44>
FIG. 4 is a flowchart showing a processing example of the sensing setting (process of step S11 in FIG. 3) performed by the sensing setting unit 44.

In step S31, the sensing setting unit 44 detects the state of each seat based on the seat information from the seat-related sensor 42. The process proceeds from step S31 to step S32.

In step S32, the sensing setting unit 44 sets the variable n representing the seat number to 1. The variable n represents the seat number when the seats in the vehicle are numbered 1 to N, and the seat whose seat number is n is represented by the seat n. The process proceeds from step S32 to step S33.

In step S33, the sensing setting unit 44 effectively sets the partial sensing range corresponding to the state of the seat n detected in step S31. Specifically, when the state of the seat n is the seated state, the sensing setting unit 44 effectively sets the partial sensing range corresponding to the seat n. When the state of the seat n is the non-seated state, the partial sensing range corresponding to the seat n is set to be invalid. The process proceeds from step S33 to step S34.

In step S34, the sensing setting unit 44 determines whether or not the variable n (seat n) is N.

If it is determined in step S34 that the variable n is not N, the process proceeds to step S35, and the sensing setting unit 44 increments the value of the variable n. The process returns to step S33, and steps S33 and S34 are repeated.

If it is determined in step S34 that the variable n is N, the process proceeds from step S34 to step S36.

In step S36, the sensing setting unit 44 sets the sensing range of each occupant observation sensor 43 based on the partial sensing range (effective partial sensing range) enabled in step S33. The process proceeds from step S36 to step S37.

In step S37, the sensing setting unit 44 sets the detection density and detection frequency of each occupant observation sensor 43 based on the sensing range of each occupant observation sensor 43 set in step S36.

When the process of step S37 is completed, the process of this flowchart is completed.

According to the above sensing setting process of the sensing setting unit 44, the sensing range of the occupant observation sensor 43 is set to an appropriate range with few unnecessary ranges according to the position of the occupant existing in the vehicle. Further, according to the sensing range of the occupant observation sensor 43, the detection density or the detection frequency of the occupant observation sensor 43 is appropriately set within a range that does not exceed the processing capacity of the information processing device 21.

<< Example of sensing setting processing by sensing setting unit 44 >>
<Example 1>
FIG. 5 is a diagram illustrating the arrangement of the seat and the occupant observation sensor in the first embodiment of the sensing setting process.

In FIG. 5, a front row seat portion 53 and a rear row seat portion 54 are arranged in the vehicle interior 52 of the automobile 51. A seat 53A, which is a driver's seat, and a seat 53B, which is a passenger seat, are arranged in the front row seat portion 53. Seats 54A and 54B are arranged in the back row seat portion 54.

On the ceiling of the vehicle interior 52, an occupant observation sensor 61 and an occupant observation sensor 62, which are a form of the occupant observation sensor 43 of FIG. 2, are arranged.

The occupant observation sensor 61 is arranged at a position near the center of the width of the entire front row seat portion 53 and above the position near the front edge of the seat surface of the front row seat portion 53. The occupant observation sensor 61 observes the occupant (driver) seated in the seat 53A and the occupant seated in the seat 53B from diagonally above the front side.

The occupant observation sensor 62 is arranged at a position near the center of the width of the entire rear row seat portion 54 and above the position near the front edge of the seat surface of the rear row seat portion 54. The occupant observation sensor 62 observes the occupants seated in the seats 54A and 54B from diagonally above the front side.

Although not shown in FIG. 5, a seating sensor, which is a form of the seat-related sensor 42, is installed in each of the seats 53A, 53B, 54A, and 54B.

Hereinafter, in the description of the first embodiment, among the occupant observation sensor 61 and the occupant observation sensor 62, only the occupant observation sensor 62 for observing the occupant of the rear row seat portion 54 will be described.

FIG. 6 is a diagram for explaining the sensing range of the occupant observation sensor 62 of FIG. 5 in the first embodiment.

In FIG. 6, occupants P1 and P2 are seated in the seats 54A and 54B of the back row seat portion 54 in FIG. 5, respectively.

Further, in the seats 54A and 54B, a seating sensor 91 and a seating sensor 92 are installed as one form of the seat-related sensor 42 in FIG. 2, respectively.

The observation range of the occupant observation sensor 62 includes the partial sensing range 101-1 (partial sensing range 101- corresponding to the seated seat 54A), which is the sensing range when only the occupant P1 seated in the seat 54A is observed. 1) and a partial sensing range 101-2 (partial sensing range 101-2 corresponding to the seated seat 54B), which is a sensing range when only the occupant P2 seated in the seat 54B is observed, are included. ..

Further, the maximum sensing range 101 that surrounds the entire partial sensing range 101-1 and 101-2 is a range that combines both the partial sensing range 101-1 and the partial sensing range 101-2.

These partial sensing ranges 101-1 and 101-2 are predetermined ranges, and are associated with each of the seats 54A and 54B in the seated state and stored in the storage unit 36 or the like in FIG. There is.

In this state, the sensing setting unit 44 acquires an on signal from the seating sensors 91 and 92. As a result, the sensing setting unit 44 detects that the seats 54A and 54B are in the seated state.

Subsequently, the sensing setting unit 44 enables the partial sensing range 101-1 corresponding to the seat 54A in the seated state. Further, the sensing setting unit 44 enables the partial sensing range 101-2 corresponding to the seat 54B in the seated state.

Then, the sensing setting unit 44 sets the maximum sensing range 101, which is the combination of the valid partial sensing ranges 101-1 and 101-2, as the sensing range of the occupant observation sensor 62.

As a result, the occupant state detection unit 45 acquires the observation information of the maximum sensing range 101 from the occupant observation sensor 43 and detects the states of the occupants P1 and P2. Further, the detection density and the detection frequency of the occupant observation sensor 62 at this time are set to a predetermined reference detection density and the reference detection frequency.

FIG. 7 is a diagram for explaining the sensing range of the occupant observation sensor 62 of FIG. 5 in the first embodiment.

Note that, in FIG. 7, the same reference numerals are given to the parts corresponding to those in FIG. 6, and the description thereof will be omitted.

FIG. 7 is different from the case of FIG. 6 in that no occupant is seated in the seat 54B of the rear row seat portion 54.

In the state of FIG. 7, the sensing setting unit 44 acquires an on signal from the seating sensor 91 and an off signal from the seating sensor 92. As a result, the sensing setting unit 44 detects that the seat 54A is in the seated state and the seat 54B is in the non-seat state.

Subsequently, the sensing setting unit 44 enables the partial sensing range 101-1 corresponding to the seat 54A in the seated state. On the other hand, the sensing setting unit 44 invalidates the partial sensing range 101-2 corresponding to the case where the non-seated seat 54B is in the seated state.

Then, the sensing setting unit 44 sets the valid partial sensing range 101-1 as the sensing range of the occupant observation sensor 62. The sensing range of the occupant observation sensor 62 in this case is about half the amount of information (area) of the maximum sensing range 101, which is the sensing range of the occupant observation sensor 62 when two occupants are present in FIG. Become.

As a result, the sensing range of the occupant observation sensor 62 is limited to an appropriate range according to the occupant's existence range, and the burden of processing with unnecessary information is reduced.

Further, the sensing setting unit 44 can set the detection density of the occupant observation sensor 62 to twice the reference detection density, for example, after limiting the sensing range. The reference detection density is the detection density when the sensing range of the occupant observation sensor 62 is the maximum sensing range as described above. As a result, the density of the observation information acquired by the occupant state detection unit 45 from the occupant observation sensor 43 is doubled.

FIG. 8 is a diagram illustrating a depth image (observation information) in FIG. 6 in which the detection density of the occupant observation sensor 62 is set to the reference detection density when the occupant observation sensor 62 is used as a depth sensor.

In FIG. 8, the occupant images P1A and P2A are depth images of the occupants P1 and P2 in FIG. 6, respectively.

FIG. 9 is a diagram illustrating a depth image (observation information) in FIG. 7 in which the detection density of the occupant observation sensor 62 is set to twice the reference detection density when the occupant observation sensor 62 is used as a depth sensor. ..

In FIG. 9, the occupant image P1B is a depth image of the occupant P1 in FIG.

According to FIGS. 8 and 9, the resolution of the occupant image P1B of FIG. 9 is higher than that of the occupant image P1A of FIG. Therefore, by setting the detection density of the occupant observation sensor 62 to twice the reference detection density, the detection accuracy of the occupant state detection unit 45 that detects the occupant state based on the observation information from the occupant observation sensor 62 is improved. do.

FIG. 10 is a flowchart showing a processing example performed by the sensing setting unit 44 in the first embodiment of FIGS. 5 to 9.

In step S51, the sensing setting unit 44 detects the states of the seats 54A and 54B based on the seat information from the seating sensors 91 and 92, which are seat-related sensors. The process proceeds from step S51 to step S52.

In step S52, the sensing setting unit 44 sets the variable n representing the seat number to 1. The seat number of seat 54A is 1, and the seat number of seat 54B is 2. The process proceeds from step S52 to step S53.

In step S53, the sensing setting unit 44 detects whether or not the seat n is in the seated state. The seat n represents a seat 54A when n is 1, and represents a seat 54B when n is 2.

If it is determined in step S53 that the seat n is in the seated state, the process proceeds from step S53 to step S54, and the sensing setting unit 44 performs partial sensing corresponding to the seated state of the seat n (seat n in the seated state). Enable the range. The partial sensing range corresponding to the seat n in the seated state when the variable n is 1 is the partial sensing range 101-1 shown in FIGS. 6 and 7. The partial sensing range corresponding to the seat n in the seated state when the variable n is 2 is the partial sensing range 101-2 shown in FIGS. 6 and 7. The process proceeds from step S54 to step S56.

If it is determined in step S53 that the seat n is not in the seated state, the process proceeds from step S53 to step S55, and the sensing setting unit 44 has a partial sensing range corresponding to the seated state of the seat n (seat n in the seated state). Is disabled. The process proceeds from step S55 to step S56.

In step S56, the sensing setting unit 44 determines whether or not the variable n is 2.

If it is determined in step S56 that the variable n is not 2, the process proceeds to step S57, and the sensing setting unit 44 increments the value of the variable n. The process returns from step S57 to step S53.

If it is determined in step S56 that the variable n is 2, the process proceeds from step S56 to step S58.

In step S58, the sensing setting unit 44 sets the partial sensing range enabled in step S54 as the sensing range of the occupant observation sensor 62. The process proceeds from step S58 to step S59.

In step S59, the sensing setting unit 44 determines whether or not both the partial sensing ranges 101-1 and 101-2 corresponding to the two seats 54A and 54B are valid.

If it is determined in step S59 that both the partial sensing ranges 101-1 and 101-2 corresponding to the two seats 54A and 54B are valid, the process proceeds from step S59 to step S60.

In step S60, the sensing setting unit 44 sets the detection density of the occupant observation sensor 43 to 1 times the basic detection density. Since the processing of this flowchart does not assume a change in the detection frequency, the sensing setting unit 44 sets the detection frequency of the occupant observation sensor 43 to one times the basic detection frequency. When the process of step S60 is completed, the process of this flowchart is completed.

In step S59, when it is determined that both the partial sensing ranges 101-1 and 101-2 corresponding to the two seats 54A and 54B are not valid, that is, at least one of the partial sensing ranges 101-1 and 101-2. If is invalid, the process proceeds from step S59 to step S61.

In step S61, the sensing setting unit 44 determines whether or not the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is valid.

If it is determined in step S61 that the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is valid, the process proceeds from step S61 to step S62.

In step S62, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to twice the basic detection density, and sets the detection frequency of the occupant observation sensor 62 to one times the basic detection frequency. When the process of step S62 is completed, the process of this flowchart is completed.

In step S61, when the sensing setting unit 44 determines that the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is not valid, either the partial sensing range 101-1 or 101-2. Is invalid, and the observation information itself is not detected by the occupant observation sensor 62. Therefore, the sensing setting unit 44 does not set the detection density and the detection frequency of the occupant observation sensor 62. Then, the processing of this flowchart is completed.

According to the above-mentioned first embodiment, the sensing range of the occupant observation sensor 62 is limited to an appropriate range according to the existence range of the occupant, and the burden of processing by unnecessary information is reduced. Further, since the detection density of the occupant observation sensor 62 is set to twice the reference detection density when there is only one occupant, the occupant state detection unit 45 that detects the occupant state recognizes the detection accuracy and operation. The recognition accuracy of the operation recognition unit 46 is improved.

In the first embodiment, the sensing setting unit 44 may change the detection frequency of the occupant observation sensor 62 according to the sensing range of the occupant observation sensor 62.

FIG. 11 is a flowchart showing a processing example performed by the sensing setting unit 44 in the first embodiment of FIGS. 5 to 9 when the detection frequency is changed according to the sensing range.

In the flowchart of FIG. 11, since steps S81 to S89 are common to steps S51 to S59 of FIG. 10, the description of steps S81 to S89 will be omitted.

If it is determined in step S89 that both the partial sensing ranges 101-1 and 101-2 corresponding to the two seats 54A and 54B are valid, the process proceeds from step S89 to step S90.

In step S90, the sensing setting unit 44 sets the detection frequency of the occupant observation sensor 62 to one times the basic detection frequency. Since the processing of this flowchart does not assume a change in the detection density, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 1 times the basic detection density. When the process of step S90 is completed, the process of this flowchart is completed.

In step S89, when it is determined that both the partial sensing ranges 101-1 and 101-2 corresponding to the two seats 54A and 54B are not valid, that is, at least one of the partial sensing ranges 101-1 and 101-2. If is invalid, the process proceeds from step S89 to step S91.

In step S91, the sensing setting unit 44 determines whether or not the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is valid.

If it is determined in step S91 that the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is valid, the process proceeds from step S91 to step S92.

In step S92, the sensing setting unit 44 sets the detection frequency of the occupant observation sensor 62 to twice the basic detection frequency, and sets the detection density of the occupant observation sensor 62 to one times the basic detection density. When the process of step S92 is completed, the process of this flowchart is completed.

In step S91, when the sensing setting unit 44 determines that the partial sensing range 101-1 or 101-2 corresponding to one seat 54A or 54B is not valid, either the partial sensing range 101-1 or 101-2. Is invalid, and the observation information itself is not detected by the occupant observation sensor 62. Therefore, the sensing setting unit 44 does not set the detection density and the detection frequency of the occupant observation sensor 62. Then, the processing of this flowchart is completed.

According to the above-mentioned first embodiment, the sensing range of the occupant observation sensor 62 is limited to an appropriate range according to the existence range of the occupant, and the burden of processing by unnecessary information is reduced. Further, since the detection frequency of the occupant observation sensor 62 is set to twice the reference detection frequency when there is only one occupant, the occupant state detection unit 45 that detects the occupant state recognizes the detection accuracy and operation. The recognition accuracy of the operation recognition unit 46 is improved.

In the above-described first embodiment, the sensing setting unit 44 may change both the detection frequency and the detection frequency of the occupant observation sensor 62 according to the sensing range of the occupant observation sensor 62.

<Example 2>
The second embodiment of the sensing setting process by the sensing setting unit 44 will be described.

In the second embodiment, the arrangement of the seat and the occupant observation sensor in the automobile and the setting of the sensing range of the occupant observation sensor when two occupants (adults) are seated in the rear row seat portion 54 are described in the first embodiment. This is common to the cases of FIGS. 5 and 6. However, the seating sensors 91 and 92 of FIG. 6 are different from the case of the first embodiment in that the seating sensors 91 and 92 in the second embodiment detect not only the on / off of the seating but also the weight. Since the other parts are the same as those in the first embodiment, the description thereof will be omitted.

FIG. 12 is a diagram illustrating a sensing range of the occupant observation sensor 62 of FIG. 5 in the second embodiment.

Note that, in FIG. 12, the parts corresponding to those in FIG. 6 are designated by the same reference numerals, and the description thereof will be omitted.

FIG. 12 shows that no occupant is seated in the seat 54B of the rear row seat portion 54 and that the occupant seated in the seat 54A is a lightweight occupant P3 having a predetermined weight g0 or less, which is estimated to be a child. It is different from the case of. It is assumed that the occupants P1 and P2 in FIG. 6 are standard occupants heavier than the predetermined weight g0.

In the state of FIG. 12, the sensing setting unit 44 acquires a weight of the seating sensor 91 on and a weight g0 or less from the seating sensor 91. Further, the sensing setting unit 44 acquires from the seating sensor 92 that the seating sensor 92 is off.

As a result, the sensing setting unit 44 detects that the seat 54A is in the seated state and the seat 54B is in the non-seat state. Further, the sensing setting unit 44 detects that the occupant P3 seated in the seat 54A is a lightweight occupant because the weight is g0 or less. In this case, the sensing setting unit 44 detects that the state of the seat 54A is the seated state of the lightweight occupant.

Subsequently, the sensing setting unit 44 enables the partial sensing range 101-3 corresponding to the case where the seat 54A is in the seated state of the lightweight occupant. On the other hand, the sensing setting unit 44 invalidates the partial sensing ranges 101-2 and 101-4 corresponding to the case where the non-seated seat 54B is in the seated state.

Here, the partial sensing range 101-3 is the sensing range of the occupant observation sensor 62 when observing only the lightweight occupant seated in the seat 54A. Comparing the partial sensing range 101-1 and the partial sensing range 101-3, which are the sensing ranges of the occupant observation sensor 62 when observing the standard occupant seated in the seat 54A, the lightweight occupant is the standard occupant. Since it is estimated that the height is shorter than that of the above, the width of the partial sensing range 101-3 in the vertical direction is particularly small.

The partial sensing range 101-4 is the sensing range of the occupant observation sensor 62 when observing only the lightweight occupant seated in the seat 54B. Similar to the partial sensing range 101-3, the partial sensing range 101-4 is larger than the partial sensing range 101-2, which is the sensing range of the occupant observation sensor 62 when observing a standard occupant seated in the seat 54B. The vertical width of is particularly small.

These partial sensing ranges 101-3 and 101-4 are predetermined ranges, and are associated with each other when the seat 54A is in the seated state of the lightweight occupant and the seat 54B is in the seated state of the lightweight occupant. It is stored in the storage unit 36 and the like of 1.

When the partial sensing range 101-3 associated with the lightweight state of the lightweight occupant of the seat 54A is enabled, the sensing setting unit 44 sets the enabled partial sensing range 101-3 as the sensing range of the occupant observation sensor 62. do. As a result, the sensing range becomes smaller than when the partial sensing range 101-1 is set as the sensing range of the occupant observation sensor 62.

In this case, the sensing range of the occupant observation sensor 62 is, for example, about two-fifths of the maximum sensing range 101 (area).

As a result, the sensing range of the occupant observation sensor 62 is limited to an appropriate range in consideration of the height of the occupant and the like, and the burden of processing with unnecessary information is reduced.

Further, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 2.5 times the reference detection density, for example, after limiting the sensing range. As a result, the density of the observation information acquired by the occupant state detection unit 45 from the occupant observation sensor 43 becomes 2.5 times.

FIG. 13 is a diagram illustrating a depth image (observation information) in FIG. 11 when the detection density of the occupant observation sensor 62 is set to 2.5 times the reference detection density when the occupant observation sensor 62 is used as a depth sensor. Is.

In FIG. 12, the lightweight occupant image P3A is a depth image of the lightweight occupant P3 in FIG.

Comparing the lightweight occupant image P3A of FIG. 12 with the occupant image P1B of FIG. 9 when the detection density of the occupant observation sensor 62 is set to twice the reference detection density, the lightweight occupant image P3A of FIG. , High resolution. Therefore, by setting the detection density of the occupant observation sensor 62 to 2.5 times the reference detection density, the occupant state detection accuracy of the occupant state detection unit 45 and the operation recognition accuracy of the operation recognition unit 46 are improved. do.

FIG. 14 is a flowchart showing a processing example performed by the sensing setting unit 44 in the second embodiment of FIGS. 12 and 13.

In step S111, the sensing setting unit 44 detects the state of each of the seats 54A and 54B based on the seat information from the seating sensors 91 and 92, which are seat-related sensors. The process proceeds from step S111 to step S112.

In step S112, the sensing setting unit 44 sets the variable n representing the seat number to 1. The seat number of seat 54A is 1, and the seat number of seat 54B is 2. The process proceeds from step S112 to step S113.

In step S113, the sensing setting unit 44 detects whether or not the seat n is in the seated state. The seat n represents a seat 54A when n is 1, and represents a seat 54B when n is 2.

If it is determined in step S113 that the seat n is in the seated state, the process proceeds from step S113 to step S114, and the sensing setting unit 44 determines whether or not the seat n is in the seated state of the lightweight occupant.

If it is determined in step S114 that the seat n is not in the seated state of the lightweight occupant, the process proceeds from step S114 to step S115, and the sensing setting unit 44 puts the seat n in the seated state of the non-lightweight occupant (standard occupant). Enable the corresponding partial sensing range. The partial sensing range corresponding to the seated state of the non-lightweight occupant of the seat n when the variable n is 1 is the partial sensing range 101-1 shown in FIG. The partial sensing range corresponding to the seated state of the non-lightweight occupant of the seat n when the variable n is 2 is the partial sensing range 101-2 shown in FIG. The process proceeds from step S115 to step S118.

If it is determined in step S114 that the seat n is in the seated state of the lightweight occupant, the process proceeds from step S114 to step S116, and the sensing setting unit 44 determines the partial sensing range in which the seat n corresponds to the seated state of the lightweight occupant. Is enabled. The partial sensing range in which the seat n corresponds to the seated state of the lightweight occupant when the variable n is 1 is the partial sensing range 101-3 shown in FIG. The partial sensing range in which the seat n corresponds to the seated state of the lightweight occupant when the variable n is 2 is the partial sensing range 101-4 shown in FIG. The process proceeds from step S116 to step S118.

If it is determined in step S113 that the seat n is not in the seated state, the process proceeds from step S113 to step S117, and the sensing setting unit 44 invalidates the partial sensing range corresponding to all the states of the seat n. The process proceeds from step S117 to step S118.

In step S118, the sensing setting unit 44 determines whether or not the variable n is 2.

If it is determined in step S118 that the variable n is not 2, the process proceeds to step S119, and the sensing setting unit 44 increments the value of the variable n. The process returns from step S119 to step S113.

If it is determined in step S118 that the variable n is 2, the process proceeds from step S118 to step S120.

In step S120, the sensing setting unit 44 sets the partial sensing range enabled in steps S115 and S116 as the sensing range of the occupant observation sensor 62. The process proceeds from step S120 to step S121.

In step S121, the sensing setting unit 44 determines whether or not the partial sensing range corresponding to the two seats 54A and 54B is effective.

If it is determined in step S121 that the partial sensing range corresponding to the two seats 54A and 54B is valid, the process proceeds from step S121 to step S122, and the sensing setting unit 44 corresponds to the two seats 54A and 54B. It is determined whether or not both of the partial sensing ranges to be performed correspond to the seated state of the lightweight occupant.

If it is determined in step S122 that both of the partial sensing ranges corresponding to the two seats 54A and 54B are not the partial sensing ranges corresponding to the seated state of the lightweight occupant, the process proceeds from step S122 to step S123.

In step S123, the sensing setting unit 44 determines whether or not any of the partial sensing ranges corresponding to the two seats 54A and 54B is the partial sensing range corresponding to the lightweight occupant.

In step S123, when it is determined that any of the partial sensing ranges corresponding to the two seats 54A and 54B is not the partial sensing range corresponding to the lightweight occupant (when the seats 54A and 54B are in the seated state of the non-lightweight occupant). The process proceeds from step S123 to step S124.

In step S124, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 1 times. Then, the processing of this flowchart is completed.

In step S123, when it is determined that any of the partial sensing ranges corresponding to the two seats 54A and 54B is the partial sensing range corresponding to the lightweight occupant (one seat is in the seated state of the lightweight occupant and the other). If the seat is seated by a non-lightweight occupant), the process proceeds from step S123 to step S125.

In step S125, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 1 time (accurately, 10/9 times). Then, the processing of this flowchart is completed.

If it is determined in step S122 that both of the partial sensing ranges corresponding to the two seats 54A and 54B are the partial sensing ranges corresponding to the seated state of the lightweight occupant, the process proceeds from step S122 to step S126.

In step S126, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 1.3 times. Then, the processing of this flowchart is completed.

If it is determined in step S121 that the partial sensing ranges corresponding to the two seats 54A and 54B are not valid, the process proceeds from step S121 to step S127.

In step S127, the sensing setting unit 44 determines whether or not the partial sensing range corresponding to one seat 54A or 54B is valid.

If it is determined in step S127 that the partial sensing range corresponding to one seat 54A or 54B is valid, the process proceeds from step S127 to step S128.

In step S128, the sensing setting unit 44 determines whether or not the partial sensing range corresponding to one seat 54A or 54B is the partial sensing range corresponding to the lightweight occupant.

In step S128, when it is determined that the partial sensing range corresponding to one seat 54A or 54B is not the partial sensing range corresponding to the lightweight occupant (one seat is in the seated state of the non-lightweight occupant and the other seat is not. (In the case of a seated state), the process proceeds from step S128 to step S129.

In step S129, the sensing setting unit 44 doubles the detection density of the occupant observation sensor 62. Then, the processing of this flowchart is completed.

If it is determined in step S128 that the partial sensing range corresponding to one seat 54A or 54B is the partial sensing range corresponding to the lightweight occupant, the process proceeds from step S128 to step S130.

In step S130, the sensing setting unit 44 sets the detection density of the occupant observation sensor 62 to 2.5 times. Then, the processing of this flowchart is completed.

If it is determined in step S127 that the partial sensing range corresponding to one seat 54A or 54B is not valid, any of the partial sensing ranges is invalid and the observation information itself is not detected by the occupant observation sensor 62. .. Therefore, the sensing setting unit 44 does not set the detection density and the detection frequency of the occupant observation sensor 62. Then, the processing of this flowchart is completed.

According to the second embodiment of the above sensing setting, the sensing range of the occupant observation sensor 62 is appropriately set according to the position of the occupant existing in the vehicle and the height of the occupant. Further, the detection density of the occupant observation sensor 62 is appropriately set according to the sensing range of the occupant observation sensor 62 within a range not exceeding the processing capacity of the information processing device 21. In addition, as in Example 1, in Example 2, the sensing setting unit 44 may change the detection frequency instead of the detection density of the occupant observation sensor 62 according to the sensing range, and the detection density and the detection frequency may be changed. You may change both.

Further, in the second embodiment, whether or not the occupant is a lightweight occupant is determined by the weight, but a sensor for detecting the height of the seated occupant is provided as a seat-related sensor 42, and whether or not the occupant is a child is determined based on the detected height. Alternatively, the vertical width of the sensing range (partial sensing range 101-3, 101-4) may be determined.

<Example 3>
The third embodiment of the sensing setting process by the sensing setting unit 44 will be described.

In the third embodiment, the arrangement of the seat and the occupant observation sensor in the automobile is the same as in the case of FIG. 5 of the first embodiment. However, the third embodiment differs from the first embodiment in that the reclining sensor and the seatbelt sensor are used as one form of the seat-related sensor 42 of FIG. 2 in addition to the seating sensor.

FIG. 15 is a diagram for explaining the sensing range of the occupant observation sensor 62 of FIG. 5 in the third embodiment.

In FIG. 15, the seat 54B is the seat on the left side of the back row seat portion 54 in FIG. An occupant P4 is seated in the seat 54B, and the seat 54B is in a reclining state with the backrest tilted backward. Further, it is assumed that the seat belt is not worn (the seat belt is not worn).

A seating sensor 92 is arranged on the seat 54B as in the first embodiment. A reclining sensor 93 is arranged on the seat 54B. Further, a seatbelt sensor 94 is arranged at the seatbelt attachment / detachment portion of the seat 54B.

In the state of FIG. 15, the sensing setting unit 44 acquires sensor signals from each of the seating sensor 92, the reclining sensor 93, and the seatbelt sensor 94 as seat information from the seat-related sensor 42. As a result, the sensing setting unit 44 detects that the seat 54B is in the seated state, that the seat 54B is in the reclining state, and that the seat belt is not worn. It is assumed that the seat 54A of FIG. 5 (not shown) in FIG. 15 is not seated.

Subsequently, the sensing setting unit 44 enables the partial sensing range 101-5 associated with the detected state of the seat 54B. On the other hand, when the non-seated seat 54A is in the seated state, all the associated partial sensing ranges are invalidated.

Then, the sensing setting unit 44 sets the valid partial sensing range 101-5 as the sensing range of the occupant observation sensor 62. In this case, the sensing range of the occupant observation sensor 62 is about half of the maximum sensing range.

The sensing setting unit 44 sets, for example, the detection density of the occupant observation sensor 62 to twice the reference detection density.

FIG. 16 is a diagram illustrating a sensing range of the occupant observation sensor 62 in a state where the seatbelt is worn with respect to the state of FIG.

In the drawings, the parts corresponding to those in FIG. 15 are designated by the same reference numerals and the description thereof will be omitted.

In FIG. 16, it is assumed that the occupant P4 is wearing a seat belt. In this case, since it is considered that the occupant P4 cannot raise the upper body or the whole body, the sensing range can be limited to the range of the height near the seat surface as compared with the case of FIG.

In FIG. 16, the sensing setting unit 44 acquires sensor signals from each of the seating sensor 92, the reclining sensor 93, and the seatbelt sensor 94 as seat information from the seat-related sensor 42. As a result, the sensing setting unit 44 detects that the seat 54B is in the seated state, that the seat 54B is in the reclining state, and that the seat belt is in the wearing state.

The sensing setting unit 44 enables the partial sensing range 101-6 associated with the detected state of FIG. Here, the partial sensing range 101-6 is smaller than the partial sensing range 101-5 when the seatbelt is not fastened.

Therefore, when the seatbelt is worn, the detection density or detection frequency of the occupant observation sensor 62 can be higher than when the seatbelt is not fastened, and the detection accuracy of the occupant state by the occupant state detection unit 45 and the operation recognition unit 46 can be increased. It is possible to improve the recognition accuracy of the operation.

<Example 4>
FIG. 17 is a diagram illustrating a sensing range of the occupant observation sensors 61 and 62 in the fourth embodiment of the sensing setting process.

In FIG. 17, the seat 53B is the seat on the left side of the front row seat portion 53 in FIG. It is assumed that the occupant P5 is seated in the seat 53B, and the seat 54B is in the reclining state with the backrest tilted backward. Further, it is assumed that the seat belt is not worn and the seat belt is not worn.

A seating sensor 95 is arranged on the seat 53B as in the first embodiment. A reclining sensor 96 is arranged on the seat 53B. Further, a seatbelt sensor 97 is arranged at the seatbelt attachment / detachment portion of the seat 53B.

In the state of FIG. 17, the sensing setting unit 44 acquires sensor signals from each of the seating sensor 95, the reclining sensor 96, and the seatbelt sensor 97 as seat information from the seat-related sensor 42. As a result, the sensing setting unit 44 detects that the seat 53B is in the seated state, that the seat 53B is in the reclining state, and that the seat belt is not worn. It is assumed that the seat 53A of FIG. 5 (not shown) in FIG. 17 is not seated.

Subsequently, the sensing setting unit 44 enables the partial sensing range 103 corresponding to the detected state of the seat 53B. On the other hand, when the seat 53A in the non-seat state is in the seated state, all the associated partial sensing ranges are invalidated.

Here, when the seat 53B is in the reclining state, the occupant P5 covers both the observation range of the occupant observation sensor 61 and the observation range of the occupant observation sensor 62. In this case, the occupant P5 cannot be observed only by either one of the occupant observation sensor 61 and the occupant observation sensor 62.

Therefore, the partial sensing range 103 corresponding to such a state of the seat 53B is set to a range included in the observation range of the occupant observation sensor 61 and the observation range of the occupant observation sensor 62. Then, of the partial sensing range 103, the portion included in the observation range of the occupant observation sensor 61 is set as the partial sensing range 103A, and the portion included in the occupant observation sensor 62 is separated as the partial sensing range 103B.

The sensing setting unit 44 sets one of the separated partial sensing ranges 103A of the valid partial sensing ranges 103 as the sensing range of the occupant observation sensor 61. Further, the sensing setting unit 44 sets the other separated partial sensing range 103B of the valid partial sensing range 103 as the sensing range of the occupant observation sensor 62.

Then, when the occupant state detection unit 45 acquires the observation information of the sensing range of the occupant observation sensor 61 and the occupant observation sensor 62, the observation information of the partial sensing range 103A from the occupant observation sensor 61 and the portion from the occupant observation sensor 62 The observation information of the partial sensing range 103 is acquired by combining (integrating) with the observation information of the sensing range 103B.

FIG. 18 is a diagram illustrating observation information (depth image) acquired from each of the occupant observation sensor 61 and the occupant observation sensor 62 in the state of FIG. 17 when the occupant observation sensor 61 and the occupant observation sensor 62 are used as depth sensors. Is.

In FIG. 18, the occupant image P5A is a depth image extracted from the partial sensing range 103A of the sensing range of the occupant observation sensor 61.

The occupant image P5B is a depth image acquired from the partial sensing range 103B of the sensing range of the occupant observation sensor 62.

The occupant state detection unit 45 acquires the occupant image P5A by extracting the depth image in the range corresponding to the partial sensing range 103A from the depth image acquired from the occupant observation sensor 61. Further, the occupant state detection unit 45 acquires the occupant image P5B by extracting the depth image in the range corresponding to the partial sensing range 103B from the depth image acquired from the occupant observation sensor 62. Then, the occupant state detection unit 45 acquires a depth image of the occupant P5 in the partial sensing range 103 of FIG. 17 by integrating (connecting) the occupant image P5A and the occupant image P5B extracted from each.

According to the fourth embodiment, when the seat 53B is in a reclining state or the like, even when one occupant straddles the observation range of the plurality of occupant observation sensors 61 and 62, the occupant observation sensors 61 and 62 By integrating the observation information obtained from each of the above, a wide range of observation information of the occupant can be obtained. Therefore, the occupant state detection unit 45 appropriately detects the occupant state.

Note that, in FIG. 18, when the seatbelt is worn, the sensing range can be further reduced as in the third embodiment.

FIG. 19 is a flowchart showing a processing example performed by the sensing setting unit 44 in the fourth embodiment of FIGS. 17 and 18.

In step S151, the sensing setting unit 44 detects the state of each seat based on the seat information from the seat-related sensors 42 (seat sensor 95, reclining sensor 96, and seat belt sensor 97). The process proceeds from step S151 to step S152.

In step S152, the sensing setting unit 44 sets the variable n representing the seat number to 1. The seat number of seat 53A is 1, and the seat number of seat 53B is 2. The process proceeds from step S152 to step S153.

In step S153, the sensing setting unit 44 detects whether or not the seat n is in the seated state. The seat n represents a seat 53A when n is 1, and represents a seat 53B when n is 2.

If it is determined in step S153 that the seat n is in the seated state, the process proceeds from step S153 to step S154, and the sensing setting unit 44 determines whether or not the seat n is in the reclining state.

If it is determined in step S154 that the seat n is not in the reclining state, the process proceeds from step S154 to step S155, and the sensing setting unit 44 effectively activates the partial sensing range corresponding to the seated state and the non-reclining state of the seat n. Set. The process proceeds from step S155 to step S158.

When it is determined in step S154 that the seat n is in the reclining state, the process proceeds from step S154 to step S156, and the sensing setting unit 44 effectively activates the partial sensing range corresponding to the seated state and the reclining state of the seat n. Set. The process proceeds from step S156 to step S158.

If it is determined in step S153 that the seat n is not in the seated state, the process proceeds from step S153 to step S157, and the sensing setting unit 44 invalidates the partial sensing range corresponding to all the states of the seat n. The process proceeds from step S157 to step S158.

In step S158, the sensing setting unit 44 determines whether or not the variable n is 2.

If it is determined in step S158 that the variable n is not 2, the process proceeds to step S159, and the sensing setting unit 44 increments the value of the variable n. The process returns from step S159 to step S153.

If it is determined in step S158 that the variable n is 2, the process proceeds from step S158 to step S160.

In step S160, the sensing setting unit 44 sets the variable n to 1. The process proceeds from step S160 to step S161.

In step S161, it is determined whether or not the valid partial sensing range corresponding to the seat n is the partial sensing range corresponding to the reclining state.

If it is determined in step S161 that the valid partial sensing range corresponding to the seat n is not the partial sensing range corresponding to the reclining state, the process proceeds from step S161 to step S162.

In step S162, the sensing setting unit 44 sets the valid partial sensing range as the sensing range of the occupant observation sensor 61 (occupant observation sensor 61 of the seat n) for observing the occupant of the seat n. The process proceeds from step S162 to step S165.

If it is determined in step S161 that the valid partial sensing range corresponding to the seat n is the partial sensing range corresponding to the reclining state, the process proceeds from step S161 to step S163.

In step S163, of the valid partial sensing ranges, the range for the occupant observation sensor 61 for seat n (occupant observation sensor 61 for observing the occupants for seat n) is set as the sensing range for the occupant observation sensor 61 for seat n. The process proceeds from step S163 to step S164.

In step S164, of the valid partial sensing range, the range of the seat behind the seat n (seat 54A when n = 1 and seat 54B when n = 2) with respect to the occupant observation sensor 62 is set to the range of the seat n. It is set as the sensing range of the occupant observation sensor 62 on the rear seat. The process proceeds from step S164 to step S165.

In step S165, it is determined whether or not the variable n is 2.

If it is determined in step S165 that the variable n is not 2, the process proceeds from step S165 to step S166, and the sensing setting unit 44 increments the value of the variable n and returns from step S166 to step S161.

If it is determined in step S165 that the variable n is 2, the process of this flowchart ends.

According to the fourth embodiment, when the seat 53B is in a reclining state or the like, even when one occupant straddles the observation range of the plurality of occupant observation sensors 61 and 62, the occupant observation sensors 61 and 62 By integrating the observation information obtained from each of the above, a wide range of observation information of the occupant can be obtained. Therefore, the occupant state detection unit 45 appropriately detects the occupant state.

<Example 5>
FIG. 20 is a diagram illustrating the arrangement of the occupant observation sensor in the fifth embodiment of the sensing setting process.

In FIG. 20, the interior 152 of the automobile 151 is provided with three rows of front row seats 153, middle row seats 154, and rear row seats 155. A driver's seat 153A and a passenger seat 153B are arranged in the front row seat portion 153. Seats 154A and 154B are arranged in the middle row seat portion 154. Seats 155A and 155B are arranged in the back row seat portion 155.

Each seat 153A to 155A and 153B to 155B are provided with seating sensors 191A to 193A and 191B and 193B for detecting the presence or absence of seating, respectively. The seating sensors 191A to 193A, and 191B and 193B are one form of the seat-related sensor 42 of FIG.

Further, the seats 153A to 155A and 153B to 155B are provided with seat position sensors 194A to 196A, and 194B and 196B, respectively, which detect the position of the seat with respect to the sliding movement and the direction in the front-back or left-right direction. The seat position sensors 194A to 196A, and 194B and 196B are one form of the seat-related sensor 42 of FIG.

The occupant observation sensors 161 to 163, which are the occupant observation sensors 43 of FIG. 2, are arranged on the ceiling inside the vehicle. The occupant observation sensors 161 to 163 are arranged near the center of the width of the ceiling, and are arranged at positions closer to the front side than directly above the front row seat portion 153, the middle row seat portion 154, and the rear row seat portion 155, respectively. The occupant observation sensors 161 to 163 are a form of the occupant observation sensor 43 of FIG.

The occupant observation sensor 161 observes the occupants of the two seats 153A and 153B of the front row seat portion 153. The occupant observation sensor 162 observes the occupants of the two seats 154A and 154B of the middle row seat portion 154. The occupant observation sensor 163 observes the occupants of the seats 155A and 155B of the rear row seat portion 155.

FIG. 21 is a diagram showing a sensing range of the occupant observation sensors 161 to 163 of FIG.

In FIG. 21, the sensing ranges 181, 182, and 183 indicate the maximum sensing ranges of the occupant observation sensors 161, 162, and 163, respectively.

The partial sensing ranges 181-1, 182-1, and 183-1 within the maximum sensing ranges 181, 182, and 183 are seated by the seating sensors 191A to 193A of the right seats 153A to 155A. This is the partial sensing range that becomes effective when detected.

Further, the partial sensing ranges 181-2, 182-2, and 183-2 within the maximum sensing ranges 181, 182, and 183 are seated by the seating sensors 191B to 193B of the left seats 153B to 155B. This is the partial sensing range that becomes effective when a condition is detected.

The sensing setting unit 44 uses the seating sensors 191A to 193A and 191B to 193B of the seats 153A to 155A and 153B to 155B to determine whether or not the seats 153A to 155A and 153B to 155B are seated (seated state or Non-seated state) is detected.

Further, the sensing setting unit 44 detects the positions of the seats 153A to 155A and 153B to 155B by the seat position sensors 194A to 196A and 194B to 196B of the seats 153A to 155A and 153B to 155B.

The sensing setting unit 44 can observe the occupants present in the vehicle based on the detected states (presence or absence of seating and the position of the seat), and excludes the range in which the occupants do not exist from the sensing range as much as possible. As such, set the valid partial sensing range. Then, the sensing setting unit 44 sets the effective partial sensing range as the sensing range of the occupant observation sensors 161 and 162, and 163.

In the vehicle interior 152 of the automobile 151, the seats 153A to 155A and 153B to 155B may be configured to be movable. In this case, the partial sensing range corresponding to the seat position in the seated state is moved according to the seat position.

That is, as shown in FIG. 21, when the seats 153A to 155A and 153B to 155B are arranged at standard positions, each occupant is assigned to any of the occupant observation sensors 161, 162, and 163. It can be observed within the observation range of the occupant observation sensor.

On the other hand, if each seat 153A to 155A and 153B to 155B are largely displaced from the standard position, it may not be possible to observe within the observation range of one occupant observation sensor. Therefore, it may be necessary to acquire the observation information of one occupant by dividing it into a plurality of occupant observation sensors.

FIG. 22 is a diagram showing a sensing range when the seat is largely displaced in FIG. 21.

In FIG. 22, the seat 153B and the seat 154B are largely displaced rearward from the standard position. The seat 153B is out of the maximum sensing range 181 of the occupant observation sensor 161 and the seat 154B is partially out of the maximum sensing range 182 of the occupant observation sensor 162.

In this case, since the occupant observation sensor 161 cannot observe the occupant of the seat 153B, the occupant observation sensor 162 observes the occupant of the seat 153B.

Further, since the occupant observation sensor 162 can only partially observe the occupant of the seat 154B, the occupant observation sensor 162 and the occupant observation sensor 163 acquire and integrate the observation information of the occupant of the seat 154B.

The sensing setting unit 44 uses the seating sensors 191A to 193A and 191B to 193B of the seats 153A to 155A and 153B to 155B to determine whether or not the seats 153A to 155A and 153B to 155B are seated (seated state or Non-seated state) is detected.

Further, the sensing setting unit 44 detects the positions of the seats 153A to 155A and 153B to 155B by the seat position sensors 194A to 196A and 194B to 196B of the seats 153A to 155A and 153B to 155B.

The sensing setting unit 44 can observe the occupants present in the vehicle based on the detected states (presence or absence of seating and the position of the seat), and excludes the range in which the occupants do not exist from the sensing range as much as possible. As such, set the valid partial sensing range.

When setting the partial sensing range effective for the seat 153B, the sensing setting unit 44 may generate a partial sensing range corresponding to the position based on the position of the seat 153B, or the sensing setting unit 44 may generate the partial sensing range corresponding to the position of the seat 153B. The corresponding data may be read out from the data of the partial sensing range stored in the storage unit 36 in advance in association with each displaceable position. In this case, the partial sensing range is set for the seat 154B as well as for the seat 153B. In FIG. 22, the partial sensing range 186 is the partial sensing range for the seat 154B.

Then, the sensing setting unit 44 sets each of the effective partial sensing ranges as the sensing ranges of the occupant observation sensors 161 and 162, and 163. At this time, the partial sensing range 186 with respect to the seat 154B is included in the observation range of the occupant observation sensor 162 and the observation range of the occupant observation sensor 163. The sensing setting unit 44 sets the range 186A included in the observation range of the occupant observation sensor 162 in the partial sensing range 186 as the sensing range of the occupant observation sensor 162, and sets the observation range of the occupant observation sensor 163 in the partial sensing range 186. The included range 186B is set as the sensing range of the occupant observation sensor 163.

When the occupant state detection unit 45 acquires the observation information from the occupant observation sensors 161 and 162 and 163, the occupant state detection unit 45 obtains the observation information of the partial sensing range 186 from the observation information from the occupant observation sensor 162 and the occupant observation sensor 163. Obtain and integrate with the observation information of.

Further, the sensing setting unit 44 sets the detection densities and detection frequencies of the occupant observation sensors 161 and 162, and 163 based on the area of the sensing range of the occupant observation sensors 161 and 162 and 163. In FIG. 22, the sensing range of the occupant observation sensor 161 can be limited to the occupant range of the seat 153A. Therefore, the detection density can be made higher than that of the other occupant observation sensors 162 and 163, and the state of one occupant can be detected with high accuracy.

According to the fifth embodiment, even when one occupant straddles the observation range of the plurality of occupant observation sensors due to the movement of the seat, the observation information obtained from each of the plurality of occupant observation sensors is integrated. Allows a wide range of observation information for the occupant to be obtained. Therefore, the occupant state detection unit 45 appropriately detects the occupant state.

<Example 6>
FIG. 23 is a diagram illustrating the arrangement of the occupant observation sensor in the sixth embodiment of the sensing setting process.

In the drawings, the parts corresponding to those in FIG. 21 are designated by the same reference numerals and the description thereof will be omitted.

The automobile 151 of FIG. 23 has a front row seat portion 153, a middle row seat portion 154, a rear row seat portion 155, seats 153A to 155A, and 153B to 155B, occupant observation sensors 161 to 163, 211, and 212, and a seating sensor 191A. To 193A, and 191B and 193B, seat position sensors 194A to 196A, and 194B and 196B. Therefore, the automobile 151 of FIG. 23 has front row seats 153, middle row seats 154, rear row seats 155, seats 153A to 155A, and 153B to 155B, occupant observation sensors 161 to 163, seating sensors 191A to 193A, and , 191B and 193B, seat position sensors 194A to 196A, and 194B and 196B, which are common to the case of FIG. 21. However, the automobile 151 of FIG. 23 is different from the case of FIG. 21 in that the occupant observation sensor 211 and 212 are newly provided.

The occupant observation sensors 211 and 212 in FIG. 23 are the occupant observation sensors 43 in FIG. 2, and are arranged near the center of the width of the ceiling in the vehicle. The occupant observation sensor 211 is arranged between the occupant observation sensor 161 and the occupant observation sensor 162, and is arranged above the seats between the front row seat portion 153 and the middle row seat portion 154.

The occupant observation sensor 212 is arranged between the occupant observation sensor 162 and the occupant observation sensor 163, and is arranged above the seats between the middle row seat portion 154 and the rear row seat portion 155.

FIG. 24 is a diagram showing the sensing ranges of the occupant observation sensors 161 to 163, 211, and 212 of FIG. 23.

In FIG. 24, the sensing ranges 181 to 183 represent the sensing ranges (maximum sensing range) of the occupant observation sensors 161 to 163, respectively. The sensing ranges 231 and 232 represent the maximum sensing ranges of the occupant observation sensors 211 and 212, respectively.

When the seats 153A to 155A and 153B to 155B are arranged in standard positions as shown in FIG. 24, the occupants seated in the seats 153A to 155A and 153B to 155B are occupants. Observation is observed within the sensing range of any one of the observation sensors 161 to 163. Therefore, when the seats 153A to 155A and 153B to 155B are arranged in standard positions as shown in FIG. 24, the occupant observation sensors 211 and 212 may not be used.

On the other hand, the seats 153A to 155A and 153B to 155B are displaced from the standard positions and are located between the front row seat portion 153 and the middle row seat portion 154, or between the middle row seat portion 154 and the rear row seat portion 155. When placed in between, the occupants of that seat are observed by the occupant observation sensor 211 or 212.

FIG. 25 is a diagram showing a sensing range when the seat is largely displaced from the standard position with respect to FIG. 24.

In FIG. 25, the seat 153B and the seat 154B are largely displaced rearward from the standard position. In particular, seat 153B is located between the standard position of the front row seats 153 and the standard position of the middle row seats 154. Seat 154B is located between the standard position of the middle row seats 154 and the standard position of the rear row seats 155.

In this case, the sensing setting unit 44 changes the occupant observation sensor for observing the occupants seated in the seats 153A to 155A and 153B to 155B by the following processing.

The sensing setting unit 44 detects the positions of the seats 153A to 155A and 153B to 155B based on the seat information from the seat position sensors 194A to 196A and 194B and 196B of the seats 153A to 155A and 153B to 155B. ..

The sensing setting unit 44 is located between the standard position of the front row seat portion 153 and the standard position of the middle row seat portion 154 (seat) based on the detected positions of the seats 153A to 155A and 153B to 155B. It is detected whether or not a seat is arranged at the position (between the seats) or between the standard position of the middle row seat portion 154 and the standard position of the rear row seat portion 155 (between the seats).

As a result, the sensing setting unit 44 causes the seats arranged between the seats to be observed by the occupant observation sensor 211 or 212, and the other seats are observed by the occupant observation sensor 161 or 162 or 163. Let them observe. Then, the sensing setting unit 44 sets the sensing range of the occupant observation sensors 161 to 163, 211, and 212 based on the presence / absence and position of the seats 153A to 155A and 153B to 155B.

By the processing of the sensing setting unit 44 as described above, in FIG. 25, the seats 153A to 155A are observed by the sensing ranges 181-1, 182-1 and 183-1 of the occupant observation sensors 161 to 163, respectively. Seats 153B and 154B are observed by the occupant observation sensors 211 and the sensing ranges 231-1 and 232-1 of 212, respectively. For non-seated seats (for example, seat 155B) that are not seated, the sensing range is not set and observation by the occupant observation sensor is not performed.

According to the sixth embodiment, the number of occupant observation sensors is increased as compared with the fifth embodiment, but the occupant state detection unit 45 detects the occupant state and the operation recognition unit 46 operates without significantly increasing the processing amount. Can be recognized with high accuracy.

<Example 7>
26 and 27 are diagrams illustrating the arrangement of the occupant observation sensor in the seventh embodiment of the sensing setting process.

In the drawings of FIGS. 26 and 27, the parts corresponding to those of FIG. 21 are designated by the same reference numerals and the description thereof will be omitted.

The arrangement of seats 153A to 155A and 153B to 155B in the vehicle interior 152 of the automobile 151 in the seventh embodiment of FIG. 26, the arrangement of the occupant observation sensors 161 to 163, and the like are the same as in the case of FIG.

Further, in the seventh embodiment, the occupant observation sensors 251, 252, and 253 in FIG. 27 are added to the fifth embodiment.

The occupant observation sensors 251, 252, and 253 are arranged at positions in front of the front row seats 153, the middle row seats 154, and the rear row seats 155, respectively, and the front row seats 153 and the middle row seats 153. The occupants seated in 154 and the back row seat portion 155 are observed from the front side thereof in a direction close to the lateral direction (horizontal direction).

Further, the occupant observation sensors 251, 252, and 253 have higher performance than the occupant observation sensors 161, 162, and 163, and the detection density and the detection frequency can be set higher than those of the occupant observation sensors 161, 162, and 163. can.

The occupant observation sensor 251 is installed at a position in front of the front row seat portion 153 near the center of the width, for example, a dashboard or a windshield peripheral portion. The maximum sensing range 271 of the occupant observation sensor 251 includes the range of the occupants seated in the seats 153A and 153B of the front row seat portion 153, and observes the occupants seated in the seats 153A and 153B from the front side thereof.

Further, the seating sensors 191A and 191B arranged in the seats 153A and 153B respectively detect whether the seats 153A and 153B are in the seated state or the non-seat state, respectively, by the sensing setting unit 44. As a result, when only the seat 153A is seated, the sensing range of the occupant observation sensor 251 is limited to the sensing range 271-1. When only the seat 153B is seated, the sensing range of the occupant observation sensor 251 is limited to the sensing range 271-2.

The occupant observation sensor 252 is installed, for example, on the back portion of the front row seat portion 153 near the center of the width. The maximum sensing range 272 of the occupant observation sensor 252 includes the range of the occupants seated in the seats 154A and 154B of the middle row seat portion 154, and observes the occupants seated in the seats 154A and 154B from the front side thereof.

Further, the seating sensors 192A and 192B arranged in the seats 154A and 154B respectively detect whether the seats 154A and 154B are in the seated state or the non-seat state, respectively, by the sensing setting unit 44. As a result, when only the seat 154A is seated, the sensing range of the occupant observation sensor 252 is limited to the sensing range 272-1. When only the seat 154B is seated, the sensing range of the occupant observation sensor 252 is limited to the sensing range 272-2.

The occupant observation sensor 253 is installed, for example, on the back portion of the middle row seat portion 154 near the center of the width. The maximum sensing range 273 of the occupant observation sensor 253 includes the range of the occupants seated in the seats 155A and 155B of the rear row seat portion 155, and observes the occupants seated in the seats 155A and 155B from the front side thereof.

Further, the seating sensors 193A and 193B arranged in the seats 155A and 155B respectively detect whether the seats 155A and 155B are in the seated state or the non-seat state, respectively, by the sensing setting unit 44. As a result, when only the seat 155A is seated, the sensing range of the occupant observation sensor 253 is limited to the sensing range 273-1. When only the seat 155B is seated, the sensing range of the occupant observation sensor 253 is limited to the sensing range 273-2.

Further, in the arrangement of the seats 153A to 155A and 153B to 155B in FIGS. 26 and 27, the occupant observation sensors 161 to 163 and the occupant observation sensors 251 to 253 overlap with respect to the occupant of one seat. Therefore, the sensing setting unit 44 may allow the occupant observation sensors 251 to 253, which have higher performance than the occupant observation sensors 161 to 163, to observe the occupant. The occupant observation sensors 161 to 163 may have higher performance than the occupant observation sensors 251 to 253, and the performance of the occupant observation sensors 161 to 163 and the performance of the occupant observation sensors 251 to 253 are not the same. May be good. When there are a plurality of occupant observation sensors that can be used for observation for one seat, the sensing setting unit 44 may use the higher performance sensor for observing the occupants, or may be determined in advance. The one may be preferentially used for observing the occupants.

On the other hand, it may not be possible to observe the occupants due to the movement of seats.

FIG. 28 is a diagram illustrating a case where the seat arrangement is changed with respect to FIGS. 26 and 27.

In FIG. 28, the orientations of the seats 154A and 154B of the middle row seat portion 154 are changed by 180 degrees.

In this case, the occupant observation sensor 252 of FIG. 27 cannot be used for occupant observation because it is shielded by the backrests of the seats 154A and 154B.

Further, since the occupant observation sensor 253 of FIG. 27 is installed on the back surface of the middle row seat portion 154 (the back surface of the seats 154A and 154B, etc.), the back surface of the front row seat portion 153 can be changed by changing the orientation of the seats 154A and 154B. It is suitable as an observation direction. Therefore, the occupant observation sensor 253 cannot be used for observing the occupant.

In such a case, the occupant observation sensor 251 is used for observing the occupants of the front row seat portion 153, and the occupant observation sensors 162 and 163 are used for observing the occupants of the middle row seat portion 154 and the rear row seat portion 155. To use.

The sensing setting unit 44 detects the seat arrangement by the sensor signals from the seat position sensors 194A to 196A and 194B to 196B installed in each of the seats 153A to 155A and 153B to 155B, respectively. Then, the sensing setting unit 44 grasps the occupant observation sensor that can be effectively used based on the seat arrangement, and among the occupant observation sensors that can be effectively used, the occupant observation sensor used for observing the occupant of each seat is selected. decide. When a plurality of occupant observation sensors can be used for one seat, the sensing setting unit 44 may determine the occupant observation sensor to be used for occupant observation by giving priority to performance as described above. A predetermined occupant observation sensor may be used for observation.

According to the seventh embodiment, the occupant state detection unit 45 can detect the occupant state and the operation recognition unit 46 can recognize the operation with high accuracy without limiting the arrangement of the seats.

<Example 8>
FIG. 29 is a diagram illustrating the arrangement of the occupant observation sensor in the eighth embodiment of the sensing setting.

In FIG. 29, a front row seat portion 303, a middle row seat portion 304, and a rear row seat portion 305 are arranged in the vehicle interior 302 of the automobile 301.

Windows 311 to 313 are provided on the side surface of the automobile 301, and touch panels 331 to 333 are installed on the windows 311 to 313. A transparent image display may be provided on each of the windows 311 to 313, or image information or text information may be projected by a projector.

Further, the touch panels 331 to 333 are a form of the occupant observation sensor 43 of FIG. 2 for observing the occupant performing the touch operation.

FIG. 30 is a diagram illustrating a case where the seat arrangement is changed with respect to FIG. 29.

In FIG. 30, the arrangement of seats has been changed so that a meeting is held. In addition, the seat 304A has been changed to a position and orientation with the window 312 behind.

In this case, since the touch panel 332 of the window 312 is unlikely to be used, the sensing of the touch panel 332 is automatically stopped (disabled).

The sensing setting unit 44 detects the position of each seat by the sensor signal from the seat position sensor provided in each seat. Then, when the seat is arranged at a position and orientation such that the backrest (seat back) of the seat is close to any one of the touch panels 331 to 333, the sensing by the touch panel in which the backrest is close is stopped.

According to the eighth embodiment, by disabling the touch panel that is not used, the processing load is reduced and erroneous operation is prevented.

<Example 9>
FIG. 31 is a diagram illustrating the arrangement of the occupant observation sensor in the ninth embodiment of the sensing setting.

In FIG. 31, the driver's seat 352 of the automobile 351 is provided with a steering wheel (steering wheel) 353, and the steering wheel 353 is provided with a steering wheel sensor 361.

Further, a touch panel 371 is provided on the window 362 on the side surface of the driver's seat 352.

The driver, occupant P6, is holding the steering wheel (steering wheel) 353.

Note that the window 362 may be provided with a transparent image display, or the projector may project image information or text information.

Further, the touch panel 371 is a form of the occupant observation sensor 43 of FIG. 2 for observing the occupant performing the touch operation. The handle sensor 361 is a sensor that detects whether or not the occupant P6 seated in the driver's seat is holding the handle 353, and is a form of a sensor included in the seat-related sensor 42 of FIG. 2 that detects the state of the seat. ..

In this case, when the occupant P6 in the driver's seat is holding the handle 353, it is presumed that the vehicle is driving, so that the sensing of the touch panel 371 of the window 362 is stopped. When applied in combination with the first embodiment of FIG. 21, an operation by a gesture or the like other than the touch operation on the touch panel 371 may be continuously effective.

The sensing setting unit 44 detects whether or not the handle 353 is gripped by the sensor signal from the handle sensor 361. As a result, when the handle 353 is gripped, the sensing of the touch panel 371 is invalidated.

According to the ninth embodiment, it is possible to prevent the driver from being distracted by performing a touch operation on the touch panel while driving.

<Application example>
In addition to being applied to automobiles, this technology is a facility equipped with predetermined equipment such as seats, and the presence or absence and position of the user can be estimated by using the equipment. If it is such a facility, it can be applied to any facility.

For example, in railcars and buses with seats, conference rooms and classrooms, restaurants (restaurants), theaters, stadiums, facilities that provide VR (Virtual Reality) space and AR (Augmented Reality) space, etc. It can be applied to a predetermined indoor space. In this case, the equipment used by the user is not limited to the seat, but may be a floor surface or stairs on which the user can stand.

The present technology can also have the following configurations.
(1) An information processing device having a processing unit that sets a sensing range of an observation sensor that observes a user who uses the equipment based on the state of the equipment installed in the vehicle.
(2) The information processing apparatus according to (1), wherein the state of the equipment includes a state of whether or not the equipment is used.
(3) The information processing device according to (1) or (2), wherein the equipment includes a seat.
(4) The information processing apparatus according to any one of (1) to (3), wherein the state of the equipment includes a changeable form of the equipment.
(5) The information processing apparatus according to any one of (1) to (4), wherein the state of the equipment includes the state of ancillary equipment related to the equipment.
(6) The information processing apparatus according to any one of (1) to (5), wherein the state of the equipment includes the position of the equipment.
(7) The information processing device according to any one of (1) to (6), wherein the observation sensor includes a sensor that acquires spatial information of the user.
(8) The information processing device according to any one of (1) to (7), wherein the observation sensor includes a sensor that acquires physical information of the user.
(9) The information processing device according to any one of (1) to (8), wherein the observation sensor includes an image sensor or a depth sensor.
(10) The information processing apparatus according to any one of (1) to (9), wherein the processing unit sets a sensing range previously associated with the state of the equipment as the sensing range of the observation sensor.
(11) The information processing apparatus according to any one of (1) to (10), wherein the processing unit sets the detection density or detection frequency of the observation sensor according to the sensing range.
(12) The information processing apparatus according to (11), wherein the processing unit sets the detection density or the detection frequency based on the amount of information acquired from the sensing range of the observation sensor.
(13) The information processing device according to (1), wherein the processing unit detects the state of the user based on the observation information from the observation sensor.
(14) The information processing apparatus according to (13), wherein the processing unit recognizes an operation corresponding to the state of the user.
(15) The processing unit of the information processing device having the processing unit is
An information processing method that sets the sensing range of an observation sensor that observes a user who uses the equipment based on the condition of the equipment.
(16) Computer
A program for functioning as a processing unit that sets the sensing range of an observation sensor that observes the user who uses the equipment based on the condition of the equipment.

11 information processing system, 41 processing unit, 42 seat-related sensor, 43 occupant observation sensor, 44 sensing setting unit, 45 occupant status detection unit, 46 operation recognition unit, 47 operation response unit

Claims (16)

1. An information processing device having a processing unit that sets a sensing range of an observation sensor that observes a user who uses the equipment based on the state of the equipment installed in the vehicle.
2. The information processing apparatus according to claim 1, wherein the state of the equipment includes a state of whether or not the equipment is used.
3. The information processing device according to claim 1, wherein the equipment includes a seat.
4. The information processing apparatus according to claim 1, wherein the state of the equipment includes a form of the equipment that can be changed.
5. The information processing apparatus according to claim 1, wherein the state of the equipment includes a state of ancillary equipment related to the equipment.
6. The information processing apparatus according to claim 1, wherein the state of the equipment includes the position of the equipment.
7. The information processing device according to claim 1, wherein the observation sensor includes a sensor that acquires spatial information of the user.
8. The information processing device according to claim 1, wherein the observation sensor includes a sensor that acquires physical information of the user.
9. The information processing device according to claim 1, wherein the observation sensor includes an image sensor or a depth sensor.
10. The information processing apparatus according to claim 1, wherein the processing unit sets a sensing range previously associated with the state of the equipment as the sensing range of the observation sensor.
11. The information processing device according to claim 1, wherein the processing unit sets the detection density or detection frequency of the observation sensor according to the sensing range.
12. The information processing device according to claim 11, wherein the processing unit sets the detection density or the detection frequency based on the amount of information acquired from the sensing range of the observation sensor.
13. The information processing device according to claim 1, wherein the processing unit detects the state of the user based on the observation information from the observation sensor.
14. The information processing device according to claim 13, wherein the processing unit recognizes an operation corresponding to the state of the user.
15. The processing unit of the information processing device having the processing unit
    An information processing method for setting a sensing range of an observation sensor for observing a user who uses the equipment based on the state of equipment provided in the vehicle.
16. Computer,
    A program for functioning as a processing unit that sets the sensing range of an observation sensor that observes the user who uses the equipment based on the condition of the equipment installed in the vehicle.

Images