WO2019207625A1

WO2019207625A1 - Occupant detection device, occupant detection method, and occupant detection system

Info

Publication number: WO2019207625A1
Application number: PCT/JP2018/016455
Authority: WO
Inventors: 洸暉安部; 太郎熊谷
Original assignee: 三菱電機株式会社
Priority date: 2018-04-23
Filing date: 2018-04-23
Publication date: 2019-10-31
Also published as: JPWO2019207625A1

Abstract

An occupant detection device (100) provided with: a calibration processing unit (13) for using image data representing a photographic image from a vehicle interior imaging camera (2) to execute a calibration process for a first occupant detection process; an end determination unit (14) for determining whether the calibration process has ended; a first occupant detection processing unit (15) for using the image data and the result of the calibration process to execute a first occupant detection process after the calibration process has ended; and a second occupant detection processing unit (16) for using the image data to execute a second occupant detection process in which the calibration process is not required, before the calibration process has ended.

Description

Occupant detection device, occupant detection method, and occupant detection system

The present invention relates to an occupant detection device, an occupant detection method, and an occupant detection system.

Conventionally, a system that realizes so-called “occupant detection” by executing image recognition processing on an image captured by a camera for imaging in a vehicle interior has been developed (see, for example, Patent Document 1). The occupant detection includes, for example, detection of the presence or absence of an occupant in the vehicle, detection of the position of each occupant when there are one or more occupants in the vehicle, and detection of each occupant when there are one or more occupants in the vehicle. It includes estimation of physique and determination of the presence or absence of child seats in each seat in the vehicle. The result of occupant detection is used, for example, for controlling the operation of the airbag.

JP 2009-107527 A

From the viewpoint of appropriately executing the operation control of the airbag, for example, it is preferable that the occupant detection when the ignition switch is turned on is executed before the vehicle starts.

Here, for example, when the driver is in a hurry, the time T1 from when the ignition switch is turned on until the vehicle starts may be short. On the other hand, when occupant detection is based on a method that requires prior calibration, the time T2 required for calibration may be longer than the time T1. As a result, there is a problem in that the execution timing of the occupant detection is delayed with respect to the start timing of the vehicle, and the operation control of the airbag cannot be executed properly.

The present invention has been made to solve the above-described problems, and an occupant detection device and an occupant detection method capable of suppressing a delay in the execution timing of occupant detection with respect to an event generation timing such as vehicle start. And an occupant detection system.

The occupant detection device according to the present invention includes a calibration processing unit that executes a calibration process for a first occupant detection process using image data indicating an image captured by a camera for imaging in a vehicle interior, and the calibration process is completed. An end determination unit that determines whether or not, a first occupant detection processing unit that executes the first occupant detection process using the image data and the result of the calibration process after the end of the calibration process, and the end of the calibration process A second occupant detection processing unit that executes a second occupant detection process that does not require a calibration process using image data is provided.

According to the present invention, since it is configured as described above, it is possible to suppress a delay in the execution timing of occupant detection with respect to the occurrence timing of an event such as vehicle start.

It is a block diagram which shows the state by which the passenger | crew detection system which concerns on Embodiment 1 is provided in the vehicle. FIG. 2A is an explanatory diagram illustrating an example of an arrangement position of a camera in the vehicle and an example of an imageable range by the camera, and is an explanatory diagram illustrating a state viewed from above the vehicle. FIG. 2B is an explanatory diagram illustrating an example of a camera arrangement position in the vehicle and an example of an imageable range by the camera, and is an explanatory diagram illustrating a state viewed from the side of the vehicle. FIG. 3A is a block diagram illustrating a hardware configuration of the control device according to the first embodiment. FIG. 3B is a block diagram illustrating another hardware configuration of the control device according to the first embodiment. FIG. 4A is a flowchart showing an operation of the occupant detection device according to Embodiment 1. FIG. 4B is a flowchart showing another operation of the occupant detection device according to Embodiment 1. It is a block diagram which shows the state by which the passenger | crew detection system which concerns on Embodiment 2 is provided in the vehicle.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.

Embodiment 1 FIG.
FIG. 1 is a block diagram illustrating a state in which an occupant detection system according to Embodiment 1 is provided in a vehicle. FIG. 2 is an explanatory diagram illustrating an example of the arrangement position of the camera in the vehicle and an example of an imageable range by the camera. With reference to FIG.1 and FIG.2, the passenger | crew detection system 300 of Embodiment 1 is demonstrated.

The vehicle 1 has a camera 2 for imaging in the passenger compartment. The camera 2 is configured by, for example, an infrared camera, a visible light camera, or a distance image sensor. The camera 2 is provided, for example, on a dashboard (more specifically, a center cluster) of the vehicle 1.

FIG. 2 shows an example of an arrangement position of the camera 2 in the vehicle 1 and an example of an imageable range A by the camera 2. In the example shown in FIG. 2, all the seats in the vehicle 1 (more specifically, the driver's seat, the passenger seat, and the rear seat) are included in the imageable range A. For this reason, all seats in the vehicle 1 are to be imaged by the camera 2.

The vehicle 1 has sensors 3. The sensors 3 are, for example, a sensor that detects on / off of an ignition switch in the vehicle 1, a sensor that detects opening / closing of each door in the vehicle 1, a sensor that detects the traveling speed of the vehicle 1, and a seat surface portion of each seat in the vehicle 1. It is configured by at least one of a load sensor (a so-called “sitting sensor”) provided or a sensor for detecting a range position of the transmission in the vehicle 1.

The image data acquisition unit 11 acquires image data indicating an image captured by the camera 2 from the camera 2 at a predetermined time interval. This time interval is set to a different value according to, for example, the imaging frame rate by the camera 2. The image data acquisition unit 11 outputs the acquired image data to the calibration processing unit 13, the first occupant detection processing unit 15, and the second occupant detection processing unit 16.

The start determination unit 12 uses the output signals from the sensors 3 to determine whether or not a predetermined event (hereinafter referred to as “start event”) has occurred in the vehicle 1.

Specifically, for example, when an ignition switch in the vehicle 1 is turned on, the start determination unit 12 determines that a start event has occurred.

Alternatively, for example, when the opening / closing of any one or more of the plurality of doors in the vehicle 1 is detected (more specifically, the one or more doors are opened first, and then the 1 When more than one door is closed), the start determination unit 12 determines that a start event has occurred.

Alternatively, for example, when the traveling speed of the vehicle 1 exceeds a predetermined speed (hereinafter referred to as “reference speed”), the start determination unit 12 determines that a start event has occurred.

Or, for example, when the seating of an occupant on any one or more of a plurality of seats in the vehicle 1 is detected, the start determination unit 12 determines that a start event has occurred.

Alternatively, for example, when the transmission in the vehicle 1 is switched from another range to the drive range, the start determination unit 12 determines that a start event has occurred.

The calibration processing unit 13 uses the image data output from the image data acquisition unit 11 to perform calibration processing for occupant detection processing (hereinafter referred to as “first occupant detection processing”) by the first occupant detection processing unit 15. Is to execute. The calibration processing by the calibration processing unit 13 is started when the start determination unit 12 determines that a start event has occurred.

The end determination unit 14 determines whether the calibration processing by the calibration processing unit 13 has ended after the calibration processing unit 13 has started the calibration processing.

When the end determination unit 14 determines that the calibration process has ended, the first occupant detection processing unit 15 detects the first occupant detection using the image data output from the image data acquisition unit 11 and the result of the calibration process. The process is executed. That is, the first occupant detection process is based on a method that requires a prior calibration process, and is executed after the calibration process is completed.

When the end determination unit 14 determines that the calibration process has not ended (that is, when it is determined that the calibration process is being performed), the second occupant detection processing unit 16 uses the image data acquisition unit 11 to The occupant detection process (hereinafter referred to as “second occupant detection process”) is executed using the output image data. That is, the second occupant detection process is based on a method that does not require a prior calibration process, and is executed before the end of the calibration process.

Each of the first occupant detection process and the second occupant detection process includes, for example, detection of the presence or absence of an occupant in the vehicle 1 (hereinafter referred to as “occupant detection”), Detection of the occupant's boarding position (hereinafter referred to as “boarding position detection”), estimation of the physique of each occupant (hereinafter referred to as “physique detection”) when there are one or more occupants in the vehicle 1, This includes the determination of whether or not a child seat is installed in each seat (hereinafter referred to as “child seat detection”).

Here, specific examples of the calibration process, the first occupant detection process, and the second occupant detection process will be described.

<Specific examples of calibration processing>
The calibration processing unit 13 sets a plurality of reference values R used for occupant detection, boarding position detection, physique detection, and child seat detection by executing image recognition processing on a captured image by the camera 2. The reference value R corresponds to, for example, the value R ₁ corresponding to the position of each seat, the value R ₂ corresponding to the position of each detection target (such as an occupant or a child seat) in the depth direction, and the position of the seat surface portion of each seat it is intended to include the value R ₃ to.

<Specific example of first occupant detection process>
The first occupant detection processing unit 15 executes image recognition processing on an image captured by the camera 2, thereby a plurality of values corresponding to a plurality of elements used for occupant detection, boarding position detection, physique detection, and child seat detection. (Hereinafter referred to as “element value”) E is calculated. The element value E is, for example, a value E ₁ corresponding to the shoulder width of each occupant, a value E ₂ corresponding to the face width of each occupant, a value E ₃ corresponding to the seat height of each occupant, and a feature such as an uneven shape in each seat. it is intended to include the value E ₄ corresponding to.

Next, the first occupant detection processing unit 15 uses the reference value R set by the calibration processing unit 13 to convert each element value E from a pixel unit value (that is, a value in a captured image) E to a meter unit value. A coefficient α for conversion to E ′ (that is, a value in real space) is calculated. The first occupant detection processing unit 15 converts each element value E from the value E in the captured image to the value E ′ in the real space using the coefficient α.

Next, the first occupant detection processing unit 15 performs occupant detection, boarding position detection, physique detection, and child seat detection using the converted element value E ′. However, the boarding position detection and the physique detection are executed only when one or more passengers are detected by the passenger detection (that is, when one or more passengers are present in the vehicle 1).

For physique detection, for example, an element value E ₁ ′ corresponding to the shoulder width of each occupant, an element value E ₂ ′ corresponding to the face width of each occupant, and an element value E ₃ ′ corresponding to the seat height of each occupant are used. For example, the occupant's physique is classified into three types: the physique of an adult (for example, a person over 14 years old), the physique of a child (for example, a person over 6 years old and under 13 years old), and the physique of an infant (for example, a person under 12 years old). Shall be. The first passenger detection processing unit 15, an element value E ₁ corresponding to the adult body size _{', E 2', E 3} ' and each of the range, the element values E ₁ corresponding to the child's physique', E ₂ ' , E ₃ ′, and a database indicating each range of element values E ₁ ′, E ₂ ′, E ₃ ′ corresponding to the infant's physique are stored in advance. This database is generated by, for example, a statistical method. The first occupant detection processing unit 15 determines in which range each element value E ₁ ′ is in the database, and each element value E ₂ ′ is in which range in the database. And whether each element value E ₃ ′ is a value in the database, the physique of each occupant is an adult physique, a child physique or an infant physique Which of the two is estimated.

In the child seat detection, for example, an element value E ₂ ′ corresponding to the face width of each occupant and an element value E ₄ corresponding to features such as the uneven shape in each seat are used. That is, the first occupant detection processing unit 15 determines whether the face corresponding to each element value E ₂ ′ is an infant's face by comparing each element value E ₂ ′ with a predetermined threshold value. To do. The first passenger detecting processor 15 determines whether the edge shape encompassing uneven shape corresponding to the individual element values E ₄ (i.e. edge shape typical child seat has). Based on these determination results, the first occupant detection processing unit 15 determines whether a child seat is installed in each seat.

<Specific example of the second passenger detection process>
The dictionary data storage unit 17 stores a plurality of dictionary data in advance. The plurality of dictionary data is generated by machine learning, for example. Each of the plurality of dictionary data is to be compared with image data indicating an image captured by the camera 2. The plurality of dictionary data correspond to different states in the vehicle 1. More specifically, the plurality of dictionary data includes the number of occupants in the vehicle 1, the position of each occupant in the vehicle 1, the physique of each occupant in the vehicle 1, and the child seat installation in each seat in the vehicle 1. It corresponds to various states that are different from each other.

The second occupant detection processing unit 16 acquires a plurality of dictionary data stored in the dictionary data storage unit 17 from the dictionary data storage unit 17. The second occupant detection processing unit 16 compares the image data output by the image data acquisition unit 11 with each of the acquired plurality of dictionary data. The second occupant detection processing unit 16 selects one dictionary data having the highest similarity to the image data among the plurality of dictionary data. Thereby, occupant detection, boarding position detection, physique detection, and child seat detection are realized.

That is, the second occupant detection process is based on so-called “pattern recognition”. For this reason, the second occupant detection process is lower in robustness against the posture change of each occupant and the detection accuracy is lower than the first occupant detection process. In other words, the first occupant detection process is more robust with respect to changes in posture of each occupant and has higher detection accuracy than the second occupant detection process.

The result information storage unit 18 stores information indicating the result of the first occupant detection process when the first occupant detection processing unit 15 executes the first occupant detection process. The result information storage unit 18 stores information indicating the result of the second occupant detection process when the second occupant detection processing unit 16 executes the second occupant detection process. Hereinafter, information stored in the result information storage unit 18 is collectively referred to as “result information”.

The airbag control device 4 acquires the result information stored in the result information storage unit 18 from the result information storage unit 18. The airbag control device 4 controls the operation of the airbag in the vehicle 1 using the acquired result information.

Specifically, for example, the airbag control device 4 invalidates the operation of the airbag corresponding to the seat on which the infant is seated based on the result of the physique detection. For example, the airbag control device 4 invalidates the operation of the airbag corresponding to the seat on which the child seat is installed based on the child seat detection result.

The calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the second occupant detection processing unit 16 constitute a main part of the occupant detection device 100. The image data acquisition unit 11, the start determination unit 12, the dictionary data storage unit 17, the result information storage unit 18, and the occupant detection device 100 constitute a main part of the control device 200. The camera 2 and the control device 200 constitute the main part of the occupant detection system 300.

Next, the hardware configuration of the main part of the control device 200 will be described with reference to FIG.

As shown in FIG. 3A, the control device 200 is configured by a computer, and the computer includes a processor 31 and

memories

32 and 33. The memory 32 causes the computer to function as the image data acquisition unit 11, the start determination unit 12, the calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the second occupant detection processing unit 16. The program is stored. When the processor 31 reads and executes the program stored in the memory 32, the image data acquisition unit 11, the start determination unit 12, the calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the first The function of the two occupant detection processing unit 16 is realized. The functions of the dictionary data storage unit 17 and the result information storage unit 18 are realized by the memory 33.

Alternatively, as illustrated in FIG. 3B, the control device 200 may include a memory 33 and a processing circuit 34. In this case, the functions of the image data acquisition unit 11, the start determination unit 12, the calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the second occupant detection processing unit 16 are realized by the processing circuit 34. It may be a thing.

Alternatively, the control device 200 may include a processor 31,

memories

32 and 33, and a processing circuit 34 (not shown). In this case, some of the functions of the image data acquisition unit 11, the start determination unit 12, the calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the second occupant detection processing unit 16 are performed. It may be realized by the processor 31 and the memory 32, and the remaining functions may be realized by the processing circuit 34.

The processor 31 uses, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor).

The

memories

32 and 33 use, for example, a semiconductor memory or a magnetic disk. More specifically, the

memories

32 and 33 are RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Stable Memory). (Solid State Drive) or HDD (Hard Disk Drive) or the like is used.

The processing circuit 34 may be, for example, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-LargeSemi-ChemicalSigleSigleSigureSigureSigureSigureSigureSigureSigureSigureSigureSigure). Is used.

Next, the operation of the occupant detection device 100 will be described with reference to the flowchart of FIG.

When the start determination unit 12 determines that a start event has occurred, the calibration processing unit 13 starts the calibration process. Thereafter, the calibration processing unit 13 continues the calibration process until the calibration process is completed (step ST1). Since a specific example of the calibration process has already been described, the description thereof will be omitted.

After the calibration processing unit 13 starts the calibration processing, the end determination unit 14 determines whether the calibration processing by the calibration processing unit 13 has ended (step ST2).

When it is determined by the end determination unit 14 that the calibration process has ended (step ST2 “YES”), the first occupant detection processing unit 15 executes the first occupant detection process (step ST3). Since the specific example of the first occupant detection process has already been described, the description thereof will be omitted.

On the other hand, when it is determined by the end determination unit 14 that the calibration process has not ended (step ST2 “NO”), the second occupant detection processing unit 16 executes the second occupant detection process (step ST4). Since the specific example of the second occupant detection process has already been described, the description thereof will be omitted.

If the determination result in step ST2 is “NO”, the calibration process is continuously executed in the background of the processes in steps ST2 and ST4. Therefore, after the process of step ST4 is executed, the end determination unit 14 may repeatedly execute the process of step ST2 until it is determined as “YES” in step ST2. And when it determines with step ST2 "YES", the 1st passenger | crew detection process part 15 may perform the process of step ST3. In other words, the second occupant detection processing unit 16 executes the second occupant detection process before the end of the calibration process, and then the first occupant detection processing unit 15 executes the first occupant detection process after the end of the calibration process. It may be what you do.

Thus, in addition to the first occupant detection processing unit 15 that executes the first occupant detection process after the end of the calibration process, the occupant detection device 100 of Embodiment 1 includes the second occupant before the end of the calibration process. A second occupant detection processing unit 16 that executes detection processing is provided. This shortens the time from when the start event occurs until the first occupant detection process is executed, as compared with a conventional occupant detection device that does not have a functional unit corresponding to the second occupant detection processing unit 16. be able to. As a result, it is possible to prevent the execution timing of the occupant detection process from being delayed with respect to the occurrence timing of an event such as the start of the vehicle 1.

The arrangement position of the camera 2 in the vehicle 1 is not limited to the example shown in FIG. 2, and the imageable range A by the camera 2 is not limited to the example shown in FIG. For example, only a part of the plurality of seats in the vehicle 1 (for example, a driver seat and a passenger seat) may be included in the imageable range A. That is, only a part of the plurality of seats in the vehicle 1 may be an object to be imaged by the camera 2.

Also, the sensors 3 are not limited to the above specific example, and the start event is not limited to the above specific example. The sensors 3 may include any sensors as long as they are sensors provided in the vehicle 1. In addition, any event may be set as a start event as long as the event can be determined using an output signal from the sensors 3.

Also, the contents of the first occupant detection process are not limited to occupant detection, boarding position detection, physique detection, and child seat detection. The first occupant detection process may include one or more of these detections. The first occupant detection process may include other detections different from these detections.

Further, the content of the second occupant detection process is not limited to occupant detection, boarding position detection, physique detection, and child seat detection. The second occupant detection process may include one or more of these detections. The second occupant detection process may include other detections different from these detections.

The first occupant detection process may be performed by a method that requires a prior calibration process, and the first occupant detection process method is not limited to the above specific example. The second occupant detection process may be performed by a method that does not require a prior calibration process, and the method of the second occupant detection process is not limited to the above specific example.

Further, the contents and method of the calibration process may be any one according to the contents and method of the first occupant detection process, and are not limited to the above specific examples.

Also, the use of the results of the first occupant detection process and the second occupant detection process (that is, the use of the result information) is not limited to the operation control of the airbag. The result information may be used for any control in the vehicle 1.

As described above, the occupant detection device 100 according to Embodiment 1 uses the image data indicating the image captured by the camera 2 for imaging in the passenger compartment to perform the calibration process for the first occupant detection process. Unit 13, end determination unit 14 for determining whether or not the calibration process has ended, and a first occupant detection process that uses the image data and the result of the calibration process after the end of the calibration process An occupant detection processing unit 15 and a second occupant detection processing unit 16 that executes a second occupant detection process that does not require a calibration process using image data before the end of the calibration process are provided. Thereby, it can suppress that the execution timing of a passenger | crew detection process is delayed with respect to the generation timing of events, such as start of the vehicle 1. FIG.

In the first occupant detection process, a plurality of values (element values) E ′ corresponding to a plurality of elements are calculated by performing an image recognition process on the captured image. The calculation of the value (E ′) is based on a plurality of reference values R, and the calibration process is to set a plurality of reference values R by executing an image recognition process on the captured image. Thereby, the robustness with respect to the attitude | position change of each passenger | crew is high, and a highly accurate 1st passenger | crew detection process is realizable.

Further, the second occupant detection process is based on pattern recognition based on comparison between image data and each of a plurality of dictionary data, and the plurality of dictionary data are generated by machine learning. Accordingly, it is possible to realize a second occupant detection process that is less accurate than the first condition detection process but does not require a prior calibration process.

In the occupant detection method according to the first embodiment, the calibration processing unit 13 executes the calibration process for the first occupant detection process using image data indicating an image captured by the camera 2 for imaging in the vehicle interior. Step ST1, step ST2 in which the end determination unit 14 determines whether or not the calibration process has ended, and the first occupant detection processing unit 15 after the end of the calibration process, the result of the image data and the calibration process The first occupant detection process using step ST3 and the second occupant detection processing unit 16 execute the second occupant detection process that does not require the calibration process using the image data before the end of the calibration process. Step ST4. Thereby, the effect similar to the said effect by the passenger | crew detection apparatus 100 can be acquired.

Further, the occupant detection system 300 according to the first embodiment uses the camera 2 for imaging in the vehicle interior and the image data indicating the image captured by the camera 2 to execute calibration processing for the first occupant detection process. A processing unit 13; an end determination unit 14 for determining whether or not the calibration process is completed; and a first occupant detection process that executes the first occupant detection process using the image data and the result of the calibration process after the calibration process is completed. An occupant detection apparatus 100 having a first occupant detection processing unit 15 and a second occupant detection processing unit 16 that executes a second occupant detection process that does not require calibration processing using image data before the end of the calibration process; Is provided. Thereby, the effect similar to the said effect by the passenger | crew detection apparatus 100 can be acquired.

Embodiment 2. FIG.
FIG. 5 is a block diagram illustrating a state in which the vehicle occupant detection system according to Embodiment 2 is provided in the vehicle. With reference to FIG. 5, an occupant detection system 300a according to the second embodiment will be described. In FIG. 5, the same blocks as those shown in FIG.

For example, when the start determination unit 12 determines that a start event has occurred, the occupant detection device 100a performs personal authentication processing for each occupant in the vehicle 1 using the image data output by the image data acquisition unit 11. It has a function to do.

The result information storage unit 18a stores information indicating the result of the personal authentication process (that is, result information) when the occupant detection device 100a executes the personal authentication process. The result information storage unit 18a stores information indicating the result of the first occupant detection process (that is, result information) when the first occupant detection processing unit 15 executes the first occupant detection process.

Here, the first occupant detection process includes, for example, occupant detection, boarding position detection, physique detection, and child seat detection. Information indicating the result of physique detection in the first occupant detection process is stored in the result information storage unit 18a in association with information indicating the result of the personal authentication process. That is, information indicating the result of physique detection in the first occupant detection process is stored in the result information storage unit 18a for each individual.

The second occupant detection processing unit 16a executes a second occupant detection process. The second occupant detection process includes, for example, occupant detection, boarding position detection, physique detection, and child seat detection. The occupant detection, riding position detection, and child seat detection methods by the second occupant detection processing unit 16a are the same as the occupant detection, riding position detection, and child seat detection methods by the second occupant detection processing unit 16 shown in FIG. That is, because it is based on pattern recognition), the description thereof will be omitted.

Here, the physique detection method by the second occupant detection processing unit 16a is different from the physique detection method by the second occupant detection processing unit 16 shown in FIG. That is, the second occupant detection processing unit 16a displays information indicating the result of the physique detection in the first occupant detection process executed in the past among the result information stored in the result information storage unit 18a. Get from. The second occupant detection processing unit 16a is a result of physique detection in the first occupant detection process executed in the past, using the acquired information, and for the same person as each occupant in the current vehicle 1 The result of physique detection is used as the result of physique detection in the second occupant detection process. This is because there is a low probability that the physique of the same person will greatly change between the execution of the past first occupant detection process and the execution of the current second occupant detection process.

The result information storage unit 18a stores information (that is, result information) indicating a result of the second occupant detection process when the second occupant detection processing unit 16a executes the second occupant detection process. However, information indicating the result of physique detection in the second occupant detection process is excluded from the storage target.

The calibration processing unit 13, the end determination unit 14, the first occupant detection processing unit 15, and the second occupant detection processing unit 16a constitute a main part of the occupant detection device 100a. The image data acquisition unit 11, the start determination unit 12, the dictionary data storage unit 17, the result information storage unit 18a, and the occupant detection device 100a constitute a main part of the control device 200a. The camera 2 and the control device 200a constitute the main part of the occupant detection system 300a.

Since the hardware configuration of the main part of the control device 200a is the same as that described with reference to FIG. 3 in the first embodiment, illustration and description thereof are omitted. That is, the function of the second occupant detection processing unit 16a may be realized by the processor 31 and the memory 32, or may be realized by the processing circuit 34. The function of the result information storage unit 18a is realized by the memory 33.

Since the operation of the occupant detection device 100a is the same as that described with reference to the flowchart of FIG. 4 in the first embodiment, illustration and description thereof are omitted. However, in step ST4, the second occupant detection processing unit 16a uses the result of physique detection in the first occupant detection process executed in the past as the result of physique detection in the second occupant detection process.

In addition, when the information which shows the result of the physique detection with respect to the same person as each crew member in the present vehicle 1 is not memorize | stored in the result information storage part 18a, the 2nd crew member detection process part 16a is 2nd shown in FIG. The physique detection by pattern recognition similar to the occupant detection processing unit 16 may be executed. In this case, information indicating the result of physique detection in the second occupant detection process may be included in the storage target.

Also, the contents of the first occupant detection process are not limited to occupant detection, boarding position detection, physique detection, and child seat detection. The contents of the second occupant detection process are not limited to occupant detection, boarding position detection, physique detection, and child seat detection. The second occupant detection processing unit 16a may use the result of the first occupant detection process executed in the past as the result of the second occupant detection process for another detection different from the physique detection.

In addition, the occupant detection device 100a, the control device 200a, and the occupant detection system 300a can employ various modifications similar to those described in the first embodiment.

As described above, in the occupant detection device 100a according to the second embodiment, the second occupant detection process uses the result of the first occupant detection process executed in the past as the result of the second occupant detection process. As a result, for example, for the physique detection, it is possible to realize a second occupant detection process that does not require a prior calibration process and has the same accuracy as the first occupant detection process.

In the present invention, within the scope of the invention, any combination of the embodiments, any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

The occupant detection device, occupant detection method, and occupant detection system of the present invention can be applied to, for example, operation control of an airbag in a vehicle.

1 vehicle, 2 cameras, 3 sensors, 4 airbag control device, 11 image data acquisition unit, 12 start determination unit, 13 calibration processing unit, 14 end determination unit, 15 first occupant detection processing unit, 16 and 16a 2 occupant detection processing unit, 17 dictionary data storage unit, 18, 18a result information storage unit, 31 processor, 32 memory, 33 memory, 34 processing circuit, 100, 100a occupant detection device, 200, 200a control device, 300, 300a occupant Detection system.

Claims

A calibration processing unit that executes a calibration process for a first occupant detection process using image data indicating an image captured by a camera for imaging in a vehicle interior;
An end determination unit for determining whether or not the calibration process is completed;
A first occupant detection processing unit that executes the first occupant detection process using the image data and the result of the calibration process after the calibration process is completed;
A second occupant detection processing unit that executes a second occupant detection process that does not require the calibration process using the image data before the end of the calibration process;
An occupant detection device.
The occupant detection device according to claim 1, wherein the calibration processing unit starts the calibration processing when an ignition switch in a vehicle is turned on.
The occupant detection device according to claim 1, wherein the calibration processing unit starts the calibration processing when opening / closing of a door in the vehicle is detected.
The occupant detection device according to claim 1, wherein the calibration processing unit starts the calibration processing when the traveling speed of the vehicle exceeds a reference speed.
The occupant detection device according to claim 1, wherein the calibration processing unit starts the calibration process when the occupant is seated on a seat in the vehicle.
The occupant detection device according to claim 1, wherein the calibration processing unit starts the calibration processing when a transmission in the vehicle is switched from another range to a drive range.
The first occupant detection process calculates a plurality of values corresponding to a plurality of elements by executing an image recognition process on the captured image,
The calculation of the plurality of values is based on a plurality of reference values,
The occupant detection apparatus according to claim 1, wherein the calibration process sets the plurality of reference values by executing an image recognition process on the captured image.
The second occupant detection process is based on pattern recognition by comparison between the image data and each of a plurality of dictionary data.
The occupant detection device according to claim 1, wherein the plurality of dictionary data are generated by machine learning.
The occupant detection device according to claim 1, wherein the second occupant detection process uses a result of the first occupant detection process executed in the past as a result of the second occupant detection process.
The occupant detection device according to claim 1, wherein each of the first occupant detection process and the second occupant detection process includes detection of the presence or absence of an occupant in the vehicle.
2. The occupant according to claim 1, wherein each of the first occupant detection process and the second occupant detection process includes detection of a boarding position of each occupant when one or more occupants are present in the vehicle. Detection device.
2. The occupant detection according to claim 1, wherein each of the first occupant detection process and the second occupant detection process includes estimation of a physique of each occupant when one or more occupants are present in the vehicle. apparatus.
The occupant detection device according to claim 1, wherein each of the first occupant detection process and the second occupant detection process includes determination of whether or not a child seat is installed in each seat in the vehicle.
A calibration processing unit executing calibration processing for first occupant detection processing using image data indicating an image captured by a camera for imaging in a vehicle interior;
A step of determining whether or not the end determination unit has ended the calibration process;
A first occupant detection processing unit executing the first occupant detection process using the image data and a result of the calibration process after the calibration process is completed;
A second occupant detection processing unit executing a second occupant detection process that does not require the calibration process using the image data before the end of the calibration process;
An occupant detection method comprising:
A camera for imaging a vehicle interior;
A calibration processing unit that executes calibration processing for first occupant detection processing using image data indicating an image captured by the camera, an end determination unit that determines whether or not the calibration processing has ended, A first occupant detection processing unit that executes the first occupant detection process using the image data and a result of the calibration process after the calibration process ends, and the image data before the calibration process ends. An occupant detection device having a second occupant detection processing unit that executes a second occupant detection process that does not require the calibration process using
An occupant detection system.