JP2020135037A - Head determination device, computer program, and storage medium - Google Patents

Abstract

To increase the accuracy in determining a head of an occupant in a cabin.SOLUTION: A processor receives a position data set including a plurality of point elements each associated with a position in a three-dimensional space. The processor extracts a head data sub set corresponding to an area in the three-dimensional space where a head of an occupant is highly likely to be present from the position data set (STEP 511). The processor reads a previous position that is a previously identified position of the head of the occupant in a cabin from a storage (STEP 512). The processor limits the head data sub set by deleting a point element present in an area in the three-dimensional space away from the previous position by a predetermined distance (STEP 514). The processor estimates a center position of the head based on the limited position data sub set (STEP 515).SELECTED DRAWING: Figure 9

Description

The present invention relates to a device for discriminating the head of an occupant in a vehicle interior. The present invention also relates to a computer program executed by a processor included in the device and a storage medium in which the computer program is stored.

As disclosed in Patent Document 1, there is known a device for discriminating an imaged head of an occupant in a vehicle interior based on image data.

Japanese Unexamined Patent Publication No. 2008-285015

An object of the present invention is to improve the discrimination accuracy of the head of an occupant in a vehicle interior.

The first aspect for achieving the above object is a head discrimination device.
An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A first position data subset corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The first position data subset is restricted by the second position data subset corresponding to the region of the three-dimensional space that is more than a predetermined distance from the past position.
Based on the restricted first position data subset, the central position of the head is estimated.
The position of the head is specified based on the estimated center position.

A second aspect of achieving the above objectives is a position dataset containing multiple point elements, each associated with a position in three-dimensional space, and a previously identified position of the occupant's head in the vehicle interior. A computer program that causes a processor to determine the head based on a certain past position.
When the computer program is executed, the processor
A first position data subset corresponding to the region of the three-dimensional space in which the occupant's head is likely to be present is extracted from the position data set.
The first position data subset is restricted by the second position data subset corresponding to the region of the three-dimensional space that is more than a predetermined distance from the past position.
Based on the restricted first position data subset, the central position of the head is estimated.
The position of the head is specified based on the estimated center position.

The first data subset corresponds to an area of three-dimensional space where the occupant's head is likely to be present. However, depending on the posture and physique of the occupant, the acquisition conditions of the position data set, and the like, body parts other than the head may be included in the position data subset. According to the above configuration, it is possible to suppress the influence of the data corresponding to the body part other than the head unexpectedly included in the position data subset as noise on the estimation of the central position of the head. Therefore, the accuracy of estimating the center position of the head is improved. As a result, the accuracy of discriminating the head of the occupant in the vehicle interior can be improved.

The head discrimination device according to the first aspect may be configured as follows.
The predetermined distance is determined based on the size of the head and the mobility of the head.

The computer program according to the second aspect may be configured as follows.
The predetermined distance is determined based on the size of the head and the mobility of the head.

According to the above configuration, it is easy to avoid the situation where the data corresponding to the head, which is originally necessary for estimating the central position, is subject to the restriction. Therefore, it is possible to suppress a decrease in the estimation accuracy of the center position.

A third aspect for achieving the above object is a head discrimination device.
An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A subset of position data corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The center-of-gravity positions of the plurality of positions associated with the plurality of point elements included in the position data subset and the intermediate points between the past positions are specified.
Based on the midpoint, the center position of the head is estimated.
The position of the head is specified based on the estimated center position.

A fourth aspect of achieving the above objectives is a position dataset containing a plurality of point elements, each associated with a position in three-dimensional space, and a previously identified position of the occupant's head in the vehicle interior. A computer program that causes a processor to determine the head based on a certain past position.
When the computer program is executed, the processor
A subset of position data corresponding to the spatial region in which the occupant's head is likely to be present is extracted from the position data set.
The center of gravity positions of the plurality of positions associated with the plurality of point elements included in the position data subset and the intermediate points between the past positions are specified.
Based on the midpoint, the center position of the head is estimated.
The position of the head is specified based on the estimated center position.

According to the above configuration, the center position of the head is estimated based on the midpoint between the center of gravity position and the past position, not the center of gravity position specified from the plurality of point elements included in the position data subset acquired this time. Is performed. Since the center position is estimated assuming the position closer to the past position as the center of gravity position, the estimated position of the center position is the actual head, especially when multiple point elements included in the position data subset show a biased distribution. The amount of deviation from the center of the part can be suppressed. Therefore, the accuracy of estimating the center position of the head is improved. As a result, the accuracy of discriminating the head of the occupant in the vehicle interior can be improved.

A fifth aspect for achieving the above object is a head discrimination device.
An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A subset of position data corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The central position of the head is estimated based on the position data subset and
The radius of the sphere that approximates the head centered on the center position is specified as the first radius.
The first radius is compared to the second radius, which is the radius of the sphere that approximates the head at the past position.
When the difference between the first radius and the second radius is less than a predetermined value, the position of the head is specified based on the center position.

A sixth aspect for achieving the above objectives is a position dataset containing a plurality of point elements, each associated with a position in three-dimensional space, and a previously identified position of the occupant's head in the vehicle interior. A computer program that causes a processor to determine the head based on a certain past position.
A subset of position data corresponding to the region of the three-dimensional space in which the occupant's head is likely to be present is extracted from the position data set.
The central position of the head is estimated based on the position data subset,
The radius of the sphere that approximates the head centered on the center position is specified as the first radius.
The first radius is compared with the second radius, which is the radius of the sphere that approximates the head at the past position.
When the difference between the first radius and the second radius is less than a predetermined value, the position of the head is specified based on the center position.

The above configuration is based on the assumption that the radius of the sphere that approximates the head does not fluctuate significantly during repeated estimation of the center position. Therefore, if the difference between the radius of the sphere identified this time and the radius of the sphere identified in the past is large, the center of gravity position on which the radius identified this time is based is a biased distribution of multiple point elements that include a subset of position data. It is highly probable that it is caused by. When the difference between the radius of the sphere specified this time and the radius of the sphere specified in the past is greater than or equal to a predetermined value, by not adopting the estimated center position, multiple point elements included in the position data subset are particularly biased. It is possible to suppress the amount of deviation of the estimated position of the center position from the actual center of the head when the distribution is shown. Therefore, the accuracy of estimating the center position of the head is improved. As a result, the accuracy of discriminating the head of the occupant in the vehicle interior can be improved.

The head discrimination device according to the fifth aspect may be configured as follows.
When the difference between the first radius and the second radius is equal to or greater than a predetermined value, the processor
The center position of the head is re-estimated based on the second radius,
The position of the head is specified based on the re-estimated center position of the head.

The computer program according to the sixth aspect may be configured as follows.
When the difference between the first radius and the second radius is equal to or greater than a predetermined value, the processor
The center position of the head is re-estimated based on the second radius.
The position of the head is specified based on the re-estimated center position of the head.

According to the above configuration, the calculation load related to the re-estimation of the center position of the head can be suppressed.

The head discrimination device according to any one of the above-mentioned first aspect, third aspect, and fifth aspect can be configured as follows.
The processor
The orientation of the occupant's face is determined based on the position data set.
When the determined face orientation is a specific orientation, the center position of the head is estimated.

The computer program according to any one of the above-mentioned second aspect, fourth aspect, and sixth aspect can be configured as follows.
To the processor
The orientation of the occupant's face is determined based on the position data set.
When the determined orientation of the face is a specific orientation, the center position of the head is estimated.

For example, when the occupant's face does not face the position data set acquisition device, the distribution of a plurality of point elements included in the position data subset tends to be biased, making it difficult to identify the position of the head. By limiting the estimation of the center position of the head to such a case, it is possible to suppress the processing load of the head discrimination device and suppress the deterioration of the discrimination accuracy of the occupant's head.

The head discrimination device according to any one of the above-mentioned first aspect, third aspect, and fifth aspect can be configured as follows.
The input interface receives the position data set from a TOF (Time of Flight) camera.

The computer program according to any one of the above-mentioned second aspect, fourth aspect, and sixth aspect can be configured as follows.
The position data set is acquired from a TOF (Time of Flight) camera.

According to the above configuration, the position information related to the occupant's head can be efficiently acquired together with the image information.

One aspect for achieving the above object is a storage medium that stores a computer program according to any one of the second, fourth, and sixth aspects described above.

According to the present invention, it is possible to improve the discrimination accuracy of the head of the occupant in the vehicle interior.

The configuration of the head discrimination system according to one embodiment is illustrated. A part of the vehicle equipped with the head discrimination system of FIG. 1 is illustrated. The flow of processing executed by the head discriminating device of FIG. 1 is illustrated. It is a figure for demonstrating the process executed by the head discriminating device of FIG. It is a figure for demonstrating the process executed by the head discriminating device of FIG. It is a figure for demonstrating the process executed by the head discriminating device of FIG. It is a figure for demonstrating the process executed by the head discriminating device of FIG. It is a figure for demonstrating the process executed by the head discriminating device of FIG. A first specific example of the head center position estimation process in FIG. 3 is shown. It is a figure for demonstrating the process of FIG. A second specific example of the head center position estimation process in FIG. 3 is shown. It is a figure for demonstrating the process of FIG. A third specific example of the head center position estimation process in FIG. 3 is shown. It is a figure for demonstrating the process of FIG.

An embodiment will be described in detail below with reference to the accompanying drawings. In each drawing used in the following description, the scale is appropriately changed so that each member has a recognizable size.

FIG. 1 schematically shows the configuration of the head discrimination system 1 according to the embodiment. The head discrimination system 1 includes a TOF (Time of Flight) camera 2 and a head discrimination device 3. FIG. 2 shows a part of the vehicle 4 on which the head discrimination system 1 is mounted. The arrow L indicates the direction along the front-rear direction of the vehicle 4. The arrow H indicates the direction along the height direction of the vehicle 4.

The TOF camera 2 is arranged at an appropriate position in the passenger compartment 41 of the vehicle 4 shown in FIG. 2, and acquires an image including the captured passenger compartment 41.

The TOF camera 2 includes a light emitting element and a light receiving element. The light emitting element emits, for example, infrared light as detection light. The emitted detection light is reflected by the object and enters the light receiving element as return light. The distance to the object that generated the return light is calculated by measuring the time from when the detection light is emitted from the light emitting element to when the return light is incident on the light receiving element. By calculating the distance for each of the plurality of pixels constituting the image acquired by the TOF camera 2, each pixel can be attached to the pixel in addition to the two-dimensional position coordinates (U, V) in the image. Includes distance information d (U, V) indicating the distance (depth) to a part of the corresponding object.

The TOF camera 2 outputs an image data ID corresponding to the acquired image. The image data ID includes a plurality of pixel data sets. Each of the plurality of pixel data sets is associated with a corresponding one of the plurality of pixels constituting the acquired image. Each of the plurality of pixel data sets includes position coordinates (U, V) and distance information d (U, V). That is, each of the plurality of pixel data sets is associated with a position in the three-dimensional space. Each of the plurality of pixel data sets is an example of a point element. The image data ID is also an example of a position data set.

The head discrimination device 3 is mounted at an appropriate position on the vehicle 4. The head discrimination device 3 is a device for discriminating the head 51 of the driver 5 in the vehicle interior 41 that has been imaged based on the image data ID provided by the TOF camera 2. The driver 5 is an example of an occupant.

The head discrimination device 3 includes an input interface 31. The input interface 31 receives the image data ID output from the TOF camera 2.

The head discrimination device 3 includes a processor 32. The processor 32 executes a process of determining the head 51 of the driver 5 that has been imaged based on the image data ID input to the input interface 31.

The flow of processing performed by the processor 32 will be described with reference to FIG. The processor 32 first executes preliminary processing on the image data ID (STEP 1).

The position coordinates (U, V) and distance information d (U, V) included in each pixel data are in the camera coordinate system using the following equation when the image center coordinates are defined as (c _X , c _Y ). It can be converted into points (X, Y, Z) in three-dimensional space. f represents the focal length of the lens included in the TOF camera 2.

Based on the above equation, the processor 32 converts the position coordinates (U, V) and the distance information d (U, V) into points (X, Y, Z) in the three-dimensional space in the camera coordinate system. The conversion of the position coordinates (U, V) and the distance information d (U, V) to the points (X, Y, Z) in the three-dimensional space in the camera coordinate system is performed by the processor built in the TOF camera 2. It may be done. In this case, each of the plurality of pixel data sets included in the image data ID output from the TOF camera 2 includes the position coordinates (X, Y, Z). Even in this case, each of the plurality of pixel data sets is associated with a position in the three-dimensional space. Even in this case, each of the plurality of pixel data sets is an example of a point element. In this case as well, the image data ID is also an example of the position data set.

The position coordinates (X, Y, Z) in the camera coordinate system can be converted into the position coordinates (W, L, H) in the vehicle coordinate system having a specific position in the vehicle 4 as the origin. The W axis is a coordinate axis extending in the left-right direction of the vehicle 4. For example, when a specific position in the driver's seat is set as the origin, the coordinate value of the W axis takes a positive value to the right of the origin and a negative value to the left of the origin when viewed from the driver 5. .. The L-axis is a coordinate axis extending in the front-rear direction of the vehicle 4. For example, the coordinate value of the L-axis takes a positive value in front of the origin and a negative value in the rear of the origin. The H axis is a coordinate axis extending in the vertical direction of the vehicle 4. For example, the H-axis coordinate value takes a positive value above the origin and a negative value below the origin.

The processor 32 converts each pixel data from the position coordinates (X, Y, Z) in the camera coordinate system to the position coordinates (W, L, H) in the vehicle coordinate system. The origin is selected, for example, as a position corresponding to the hipbone of the driver 5. The coordinate transformation to the vehicle coordinate system can be performed using well-known coordinate rotation transformation, translation transformation, scale transformation, or the like.

Subsequently, the processor 32 performs a process of limiting the spatial area to be processed. Specifically, a plurality of pixel data set PDs corresponding to a limited spatial area are extracted from the image data ID acquired from the TOF camera 2.

FIG. 4A shows an example of the image data ID acquired from the TOF camera 2. As shown in FIG. 2, it is highly probable that the head 51 of the driver 5 seated in the driver's seat is located above the passenger compartment 41. Therefore, the space area to be processed is limited to the upper part of the vehicle interior 41. For example, a limited spatial region can be defined as a spatial region in which the W-axis coordinate value ranges from -510 mm to 360 mm, the L-axis coordinate value ranges from -280 mm to 700 mm, and the H-axis coordinate value ranges from 430 mm to 750 mm. .. FIG. 4B shows an example of a plurality of pixel data sets PD corresponding to a limited spatial area.

By performing the process of limiting the spatial area, the load of the discrimination process of the head 51, which will be described later, can be reduced. However, this process may be omitted.

Subsequently, the processor 32 divides the plurality of pixel data set PDs extracted as described above into a plurality of data subset DSs (STEP 2 in FIG. 3).

FIG. 5A shows a plurality of pixel data sets PD projected on the LH coordinate plane formed by the L-axis and the H-axis. The processor 32 divides a plurality of pixel data sets PD projected on the LH coordinate plane into a plurality of data subsets DS, thereby dividing the plurality of pixel data sets PD into one of a plurality of space areas (vehicle compartment space areas) constituting a part of the vehicle interior 41. Each data subset DS is associated with one.

The vehicle interior space area corresponding to each data subset DS has a predetermined width dL along the L axis. The width dL can be predetermined as a dimension corresponding to the human head. "Dimensions corresponding to the human head" are, for example, the size of the head published in the human body size database of the National Institute of Advanced Industrial Science and Technology and the human characteristics database of the National Institute of Technology and Evaluation Technology. It can be determined based on the information related to.

The processor 32 then identifies the high likelihood data subset LDS (STEP 3 in FIG. 3). The high-likelihood data subset LDS is a data subset DS corresponding to the vehicle interior space region in which the head 51 of the driver 5 is likely to be present.

Specifically, the processor 32 counts the number of significant pixel data sets contained in each data subset DS. The “significant pixel data set” means a pixel data set having distance information D (U, V) that is likely to have been acquired by reflecting the detected light by the driver 5. The numerical range of such distance information D can be appropriately determined based on the positional relationship between the TOF camera 2 and the driver 5.

In the case of a plurality of pixel data sets PD projected on the LH coordinate plane, the cabin space area corresponding to the data subset DS containing the most significant pixel data sets includes the head 51 and the fuselage 52 of the driver 5. There is a high possibility that it has been.

Therefore, as shown in FIG. 5B, the processor 32 identifies the data subset DS, which contains the most significant pixel data sets, as the high likelihood data subset LDS. In other words, the processor 32 is in at least one of the plurality of cabin space areas where the head 51 is likely to be present, based on the number of pixel data sets corresponding to the driver 5 included in each data subset DS. Identify the corresponding high likelihood data subset LDS.

The processor 32 then discriminates the head based on the identified high likelihood data subset LDS (STEP 4 in FIG. 3).

First, as shown in FIG. 5B, the processor 32 identifies the crown 51a of the driver 5. Specifically, among the significant pixel data sets included in the high-likelihood data subset LDS, the one at the highest position is identified as the crown 51a. In other words, among the significant pixel data sets included in the high likelihood data subset LDS, the one having the largest H-axis coordinate value is identified as the crown 51a.

Subsequently, as shown in FIGS. 6A and 6B, the processor 32 defines a probable head space region HSR that includes the imaged driver 5 head 51. The head space region HSR is determined based on the coordinates of the identified parietal region 51a. Specifically, the head is based on the information on the size of the head published in the human body size database of the National Institute of Advanced Industrial Science and Technology and the human characteristics database of the National Institute of Technology and Evaluation Technology. The dimension hdL in the direction along the L axis of the partial space region HSR, the dimension hdH in the direction along the H axis, and the dimension hdW in the direction along the W axis are determined.

For example, the dimension hdL is determined to take a value of 176 mm centered on the coordinates of the crown 51a. The dimension hdH is determined to take a value of 146 mm downward from the coordinates of the crown 51a. The dimension hdW is determined to take a value of 176 mm centered on the coordinates of the crown 51a.

Next, as shown in FIG. 7A, the processor 32 extracts the head data subset HSD corresponding to the head space region HSR defined as described above from the plurality of pixel data sets PD.

Further, as shown in FIG. 7B, the processor 32 is located at the center of gravity position G (Wg, Lg, Hg) of the head space region HSR defined as described above in the head data subset HSD. Identify the corresponding pixel dataset.

Subsequently, the processor 32 performs a process of estimating the center position of the head 51 (STEP 5 in FIG. 3). Since the head data subset HSD is acquired based on the return light from the surface of the head 51, the center of gravity position G identified as described above is more surface than the center of the head 51 (actual center of gravity). There is a tendency to approach. As shown in FIG. 8A, the head 51 of the driver 5 can be approximated by a sphere S. Assuming that a plurality of pixel data sets (point elements) included in the head data subset HSD are distributed on the surface of the sphere S, the distances from each point element to the center of the sphere S are equal.

Therefore, the processor 32 searches for a point where the variation in the distance from each point element is the minimum, and corrects the center of gravity position G to be the position of such a point. The processor 32 estimates the corrected center of gravity position G'as the center position C of the head 51.

The point where the variation in the distance from each point element is minimized can be searched for, for example, the point where the value of the evaluation function represented by the following equation is minimized.
n represents the number of plurality of point elements contained in the head data subset HSD.
p _k represents the position of each point element included in the head data subset HSD.
G represents the position of the center of gravity of the head 51 specified as described above.
d ( _pk , G) represents the Euclidean distance between the position of a certain point element and the position of the center of gravity.

Further, the processor 32 specifies the radius of the sphere S that approximates the head 51. The radius can be specified, for example, based on the Euclidean distance between the initial center of gravity position G and the corrected center of gravity position G'.

The processor 32 identifies the position of the driver 5's head 51 based on the center position C thus estimated. As shown in FIG. 1, the head discrimination device 3 includes an output interface 33. The output interface 33 can output data HD indicating the position of the determined head of the driver 5. The output data HD is used in the subsequent recognition process. In the recognition process, for example, the direction, inclination, movement, etc. of the head 51 of the driver 5 can be recognized by monitoring the change with time of the face orientation indicated by the data. As a result, inattentiveness, dozing, and abnormal behavior due to seizures of the driver 5 during driving can be detected.

As shown in FIG. 1, the head discrimination device 3 includes a storage 34. The storage 34 is a device capable of storing predetermined data in cooperation with the processor 32. Examples of the storage 34 include a semiconductor memory and a hard disk drive device.

The processor 32 stores the data indicating the estimated center position C of the head 51 and the data indicating the radius of the sphere S that approximates the head 51 in the storage 34 (STEP 6 in FIG. 3).

The head discrimination device 3 is configured to repeatedly discriminate the head 51 of the driver 5. The cycle of repetition is determined as appropriate. Once the data indicating the center position C of the head 51 is stored in the storage 34, the processor 32 heads the driver 5 based on the head data subset HSD and the data indicating the center position C stored in the storage. The position of the part 51 is specified. In other words, the processor 32 discriminates the head 51 based on the head data subset HSD acquired this time and the position of the head 51 of the driver 5 identified in the past. The center position C of the head 51 corresponding to the data stored in the storage 34 is an example of the past position.

FIG. 9 shows a first example of the process of estimating the center position C of the current head 51 based on the past position of the head 51.

As described with reference to FIGS. 3 to 7, the processor 32 extracts the head data subset HSD from the image data ID acquired from the TOF camera 2 (STEP 511). As mentioned above, the head data subset HSD corresponds to a region of three-dimensional space in which the head 51 of the driver 5 is likely to be present. In this example, the head data subset HSD is an example of a first position data subset.

FIG. 10A shows the head data subset HSD acquired this time. In this example, not only the head 51 of the driver 5 but also a pixel data set (point element) based on the return light from the left shoulder 53 is included. The center of gravity position G obtained based on the plurality of point elements included in such a head data subset HSD is closer to the left shoulder portion 53 than the actual center of gravity position of the head 51. When the center position C is estimated based on the center of gravity position G, the position closer to the left shoulder portion 53 than the actual center position of the head 51 is estimated.

Therefore, the processor 32 reads out the data corresponding to the center position C of the past head portion 51 stored in the storage 34 (STEP 512 in FIG. 9). In the following description, the center position C of the past head 51 will be referred to as the past center position C'.

The processor 32 then determines whether there is a point element included in the head data subset HSD in a region of three-dimensional space that is more than a predetermined distance from the past center position C'(STEP 513). In FIG. 10B, the outside of the sphere S corresponds to a region of three-dimensional space that is more than a predetermined distance from the past center position C'.

In this example, the point element of the head data subset HSD exists in the region (YES in STEP 513 of FIG. 9). In this case, the processor 32 deletes a part of the head data subset HSD corresponding to the region of the three-dimensional space that is more than a predetermined distance from the past center position C'(STEP 514). A portion of the head data subset HSD corresponding to a region of three-dimensional space that is more than a predetermined distance from the past center position C'is an example of a second position data subset.

Subsequently, as shown in (C) of FIG. 10, the processor 32 identifies the center-of-gravity positions G of a plurality of point elements included in a part of the remaining head data subset HSD, and is based on the center-of-gravity position G. The center position C of the head 51 is estimated (STEP 515 in FIG. 9). The method of estimating the center position C is as described with reference to FIG. 8A. Comparing with the example shown in FIG. 10A, it can be seen that the center position C is estimated closer to the center of the actual head 51.

If there is no point element included in the head data subset HSD in the region of the three-dimensional space more than a predetermined distance from the past center position C'(NO in STEP 513), the process shifts to STEP 515 and the head data subset HSD The center of gravity position G is specified and the center position C is estimated based on all the point elements included in.

Subsequently, the processor 32 stores the data corresponding to the center position C thus estimated in the storage 34 (STEP 516 in FIG. 9). That is, the data corresponding to the center position C estimated this time overwrites the data corresponding to the past center position C'stored in the storage 34. The data stored this time is treated as data corresponding to the past center position C'in the next estimation process of the center position C of the head 51.

That is, in this example, the head data subset HSD used to estimate the center position C of the head 51 may be restricted. The limitation is made by deleting a portion of the head data subset HSD in a region of three-dimensional space that is more than a predetermined distance from the past center position C'. A portion of the head data subset HSD may not be deleted if it is not used to estimate center position C.

As mentioned above, the head data subset HSD corresponds to a region of three-dimensional space in which the head 51 of the driver 5 is likely to be present. However, depending on the posture and physique of the driver 5, the image acquisition conditions by the TOF camera 2, and the like, body parts other than the head 51 may be included in the head data subset HSD. According to the above configuration, the data corresponding to the body parts other than the head 51 unexpectedly included in the head data subset has an effect as noise on the estimation of the center position C of the head 51. Can be suppressed. Therefore, the estimation accuracy of the center position C of the head 51 is improved. As a result, the discrimination accuracy of the head 51 of the driver 5 in the vehicle interior 41 can be improved.

The predetermined distance from the past center position C', which is the reference for the judgment in STEP 513, can be determined as a distance corresponding to the general size of the head 51. In this case, the "distance corresponding to the general size of the head 51" is disclosed in, for example, the human body size database of the National Institute of Advanced Industrial Science and Technology and the human characteristics database of the National Institute of Technology and Evaluation Technology. It can be determined based on the information regarding the size of the head.

More preferably, the distance that the head 51 can move during the repeated estimation of the center position C is added to the distance determined based on the size of the head 51. The value of the distance added can be, for example, 60 mm. That is, the predetermined distance from the past center position C'can be determined based on the size of the head 51 and the mobility of the head 51.

With such a configuration, it becomes easy to avoid a situation in which the data corresponding to the head 51, which is originally necessary for estimating the central position C, is subject to restriction. Therefore, it is possible to suppress a decrease in the estimation accuracy of the center position C.

FIG. 11 shows a second example of the process of estimating the center position C of the current head 51 based on the past position of the head 51.

As described with reference to FIGS. 3 to 7, the processor 32 extracts the head data subset HSD from the image data ID acquired from the TOF camera 2 (STEP 521). As mentioned above, the head data subset HSD corresponds to a region of three-dimensional space in which the head 51 of the driver 5 is likely to be present. The head data subset HSD is an example of a position data subset.

Subsequently, the processor 32 identifies the center of gravity positions G of the plurality of point elements included in the extracted head data subset HSD.

FIG. 8B illustrates a case where the driver 5's face is facing to the right in the passenger compartment 41. In this example in which the face of the driver 5 does not face the TOF camera 2, the distribution of a plurality of point elements included in the head data subset HSD is biased in a specific direction. The radius of the sphere S, which is assumed to have such a plurality of point elements distributed on the surface, tends not to have a valid value. The center position C estimated based on the center of gravity position G identified from the plurality of point elements of the biased distribution also tends to deviate from the center of the actual head 51.

Therefore, the processor 32 reads out the data corresponding to the past center position C'of the head 51 stored in the storage 34 (STEP 522 in FIG. 11). Further, the processor 32 identifies the midpoint I between the identified center of gravity position G and the past center position C'as shown in FIG. 12 (A) (STEP 523).

Subsequently, the processor 32 estimates the center position C of the head 51 based on the identified midpoint I, as shown in FIG. 12 (B) (STEP 524 in FIG. 11). The method of estimating the center position C is as described with reference to FIG. 8A. Comparing with the example shown in FIG. 8B, it can be seen that the center position C is estimated closer to the center of the actual head 51.

Subsequently, the processor 32 stores the data corresponding to the center position C thus estimated in the storage 34 (STEP 525 in FIG. 11). That is, the data corresponding to the center position C estimated this time overwrites the data corresponding to the past center position C'stored in the storage 34. The data stored this time is treated as data corresponding to the past center position C'in the next estimation process of the center position C of the head 51.

That is, in this example, the head is based on the midpoint I between the center of gravity position G and the past center position C', not the center of gravity position G specified from the plurality of point elements included in the head data subset HSD acquired this time. The center position C of the unit 51 is estimated. Since the center position C is estimated assuming that the position closer to the past center position C'is the center of gravity position, the center position C is particularly when a plurality of point elements included in the head data subset HSD show a biased distribution. It is possible to suppress the amount of deviation of the estimated position of the head 51 from the center of the actual head 51. Therefore, the estimation accuracy of the center position C of the head 51 is improved. As a result, the discrimination accuracy of the head 51 of the driver 5 in the vehicle interior 41 can be improved.

FIG. 13 shows a third example of the process of estimating the center position C of the current head 51 based on the past position of the head 51.

As described with reference to FIGS. 3 to 7, the processor 32 extracts the head data subset HSD from the image data ID acquired from the TOF camera 2 (STEP 531). As mentioned above, the head data subset HSD corresponds to a region of three-dimensional space in which the head 51 of the driver 5 is likely to be present. The head data subset HSD is an example of a position data subset.

Subsequently, the processor 32 identifies the center-of-gravity positions G of a plurality of point elements included in the extracted head data subset HSD, and estimates the center position C of the head 51 based on the center-of-gravity position G (STEP 532). The method of estimating the center position C is as described with reference to FIG. 8A.

Subsequently, the processor 32 specifies the radius of the sphere S that approximates the head 51 (STEP 533 in FIG. 13). The method of specifying the radius is as described with reference to FIG. 8A. The radius of the sphere S that approximates the head 51 centered on the center position C estimated this time is an example of the first radius.

FIG. 8C illustrates a case where the face of the driver 5 is facing downward in the passenger compartment 41. In this example in which the face of the driver 5 does not face the TOF camera 2, the distribution of a plurality of point elements included in the head data subset HSD is biased in a specific direction. The radius of the sphere S, which is assumed to have such a plurality of point elements distributed on the surface, tends not to have a valid value.

Therefore, the processor 32 reads out the data corresponding to the radius of the sphere S'that approximates the past head 51 stored in the storage 34 (STEP 534 in FIG. 13). In the following description, the radius of the sphere S'that approximates the past head 51 is referred to as the past radius. The past radius is an example of the second radius.

Subsequently, the processor 32 compares the radius of the sphere S specified in STEP 533 with the past radius, and determines whether the difference between the two is less than a predetermined value (STEP 535). The predetermined value can be, for example, ± 30 mm.

If it is determined that the difference between the two is less than a predetermined value (YES in STEP 535), the processor 32 has data corresponding to the center position C estimated in STEP 532 and data corresponding to the radius of the sphere S estimated in STEP 533. Is stored in the storage 34 (STEP 536). That is, the data corresponding to the center position C estimated this time and the data corresponding to the radius of the sphere S specified this time correspond to the data corresponding to the past center position C'stored in the storage 34 and the past radius. Each data is overwritten. The data stored this time will be treated as data corresponding to the past center position C'and data corresponding to the past radius at the time of the next estimation processing of the center position C of the head 51.

FIG. 14A shows a case where the radius of the sphere S specified in STEP 533 is large and the difference from the past radius is equal to or more than a predetermined value (NO in STEP 535 of FIG. 13). In this case, the process returns to STEP 532, and the processor 32 re-estimates the center position C.

The processing of this example is based on the premise that the radius of the sphere S that approximates the head 51 does not fluctuate significantly during the repeated estimation of the center position C. Therefore, when the difference between the radius of the sphere S specified this time and the past radius is large, the center of gravity position G based on the radius specified this time has a biased distribution of a plurality of point elements included in the head data subset HSD. It is highly probable that it is caused. When the difference between the radius of the sphere S specified this time and the past radius is greater than or equal to a predetermined value, by not adopting the estimated center position C, a distribution in which a plurality of point elements included in the head data subset HSD are particularly biased. In the case of, the amount that the estimated position of the center position C deviates from the center of the actual head 51 can be suppressed. Therefore, the estimation accuracy of the center position C of the head 51 is improved. As a result, the discrimination accuracy of the head 51 of the driver 5 in the vehicle interior 41 can be improved.

When the difference between the radius of the sphere S specified in STEP 533 and the past radius is equal to or greater than a predetermined value (NO in STEP 535), the processor 32 is the center position of the head 51 based on the radius of the sphere S'specified in the past. C can be re-estimated (STEP 537).

Specifically, a straight line CL connecting the center of gravity positions G of a plurality of point elements included in the head data subset HSD specified in STEP 532 and the center position C of the sphere S specified in STEP 533 is set. Subsequently, a point on the straight line CL whose distance from the center of gravity position G coincides with the radius of the sphere S'specified in the past is specified. The point identified in this way is the center position C of the re-estimated head 51.

FIG. 14B shows the center position C re-estimated in this way. Comparing with the example shown in FIG. 14 (A), it can be seen that the center position C is estimated closer to the center of the actual head 51.

In this case, the calculation load related to the re-estimation of the center position C of the head 51 can be suppressed.

The head discrimination device 3 may be configured to perform a process of estimating the center position C of the head 51 according to each processing example only when the face of the driver 5 does not face the TOF camera 2. In this case, the head discrimination device 3 performs a process of discriminating the orientation of the driver 5's face based on the image data ID acquired from the TOF camera 2. “When the face of the driver 5 does not face the TOF camera 2” is an example of a case where the determined face orientation of the driver 5 is a specific orientation.

When the face of the driver 5 does not face the TOF camera 2, the distribution of the plurality of point elements included in the head data subset HSD tends to be biased, and it tends to be difficult to specify the position of the head 51. By limiting the estimation of the center position C of the head 51 to such a case, it is possible to suppress a decrease in the discrimination accuracy of the head 51 of the driver 5 while suppressing the processing load of the head discrimination device 3.

In the present embodiment, the input interface 31 of the head discrimination device 3 receives information relating to the position of the head 51 of the driver 5 in the three-dimensional space from the TOF camera 2 as a part of the image data ID. According to such a configuration, the position information related to the head 51 of the driver 5 can be efficiently acquired together with the image information. However, the image pickup device that outputs the image information and the device that outputs the position information may be different. Examples of the latter device include a LiDAR (Light Detection and Ranging) sensor.

The functions of the processor 32 described above can be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. Examples of general-purpose microprocessors include CPUs, MPUs, and GPUs. ROM and RAM can be exemplified as the general-purpose memory. In this case, the ROM may store a computer program that executes the above-described processing. The ROM is an example of a storage medium for storing a computer program. The processor 32 specifies at least a part of the program stored in the ROM, expands it on the RAM, and executes the above-described processing in cooperation with the RAM. The storage 34 may be used as the general-purpose memory described above. The processor 32 may be realized by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA capable of executing a computer program that realizes the above-mentioned processing. The processor 32 may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

As shown in FIG. 1, the head discrimination device 3 can be configured to be communicable with the external server 7 via the network 6. In this case, the computer program that executes the above-described processing can be downloaded from the external server 7 via the network 6. The external server 7 is an example of a storage medium that stores a computer program.

The above embodiments are merely examples for facilitating the understanding of the present invention. The configuration according to the above embodiment may be appropriately changed or improved without departing from the gist of the present invention.

In the above embodiment, the past position of the head 51 of the driver 5 is based on the center position C of the head 51. However, other parts of the head 51, such as the crown, may be used as a reference for the past position.

In the above embodiment, the "face orientation not facing the TOF camera 2" is to the right and downward in the passenger compartment 41. However, the "direction of the face not facing the TOF camera 2" may change depending on the positional relationship between the TOF camera 2 and the driver's seat. For example, in the case of a vehicle in which the driver's seat is arranged on the left side of the passenger compartment 41, the TOF camera 2 acquires an image of the driver 5 from the upper right. In this case, the "direction of the face not facing the TOF camera 2" is to the left and downward in the passenger compartment 41.

In the above embodiment, the head 51 of the driver 5 imaged in the vehicle interior 41 is used for discrimination. However, by appropriately arranging the TOF camera 2, the heads of other occupants imaged may be used for discrimination.

2: TOF camera, 3: Head discrimination device, 31: Input interface, 32: Processor, 41: Vehicle room, 5: Driver, 51: Head, 7: External server, ID: Image data, PD: Pixel data Set, HSD: Head data subset, C: Head center position, G: Center of gravity of multiple point elements, S: Sphere

Claims (15)

An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A first position data subset corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The first position data subset is restricted by the second position data subset corresponding to the region of the three-dimensional space that is more than a predetermined distance from the past position.
Estimate the central position of the head based on the restricted first position data subset and
Identify the position of the head based on the estimated center position.
Head discrimination device. The predetermined distance is determined based on the size of the head and the mobility of the head.
The head discrimination device according to claim 1. An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A subset of position data corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The center-of-gravity positions of the plurality of positions associated with the plurality of point elements included in the position data subset and the intermediate points between the past positions are specified.
Based on the midpoint, the center position of the head is estimated.
Identify the position of the head based on the estimated center position.
Head discrimination device. An input interface that accepts position datasets, each containing multiple point elements associated with positions in three-dimensional space.
A processor that determines the head based on the position data set and the past position, which is the position of the head of the occupant in the vehicle interior identified in the past.
Is equipped with
The processor
A subset of position data corresponding to the region of the three-dimensional space where the occupant's head is likely to be present is extracted from the position data set.
The central position of the head is estimated based on the position data subset and
The radius of the sphere that approximates the head centered on the center position is specified as the first radius.
The first radius is compared to the second radius, which is the radius of the sphere that approximates the head at the past position.
When the difference between the first radius and the second radius is less than a predetermined value, the position of the head is specified based on the center position.
Head discrimination device. When the difference between the first radius and the second radius is equal to or greater than a predetermined value, the processor
The center position of the head is re-estimated based on the second radius,
Identify the position of the head based on the re-estimated center position of the head.
The head discrimination device according to claim 4. The processor
The orientation of the occupant's face is determined based on the position data set.
When the determined orientation of the face is a specific orientation, the center position of the head is estimated.
The head discrimination device according to any one of claims 1 to 5. The input interface receives the position data set from a TOF (Time of Flight) camera.
The head discrimination device according to any one of claims 1 to 6. Processor the head based on a position dataset containing multiple point elements, each associated with a position in three-dimensional space, and a past position, which is the position of the occupant's head in the passenger compartment identified in the past. It ’s a computer program that lets you know
When the computer program is executed, the processor
A first position data subset corresponding to the region of the three-dimensional space in which the occupant's head is likely to be present is extracted from the position data set.
The first position data subset is restricted by the second position data subset corresponding to the region of the three-dimensional space that is more than a predetermined distance from the past position.
Based on the restricted first position data subset, the central position of the head is estimated.
The position of the head is specified based on the estimated center position.
Computer program. The predetermined distance is determined based on the size of the head and the mobility of the head.
The computer program according to claim 8. Processor the head based on a position dataset containing multiple point elements, each associated with a position in three-dimensional space, and a past position, which is the position of the occupant's head in the passenger compartment identified in the past. It ’s a computer program that lets you know
When the computer program is executed, the processor
A subset of position data corresponding to the spatial region in which the occupant's head is likely to be present is extracted from the position data set.
The center of gravity positions of the plurality of positions associated with the plurality of point elements included in the position data subset and the intermediate points between the past positions are specified.
Based on the midpoint, the center position of the head is estimated.
The position of the head is specified based on the estimated center position.
Computer program. Processor the head based on a position dataset containing multiple point elements, each associated with a position in three-dimensional space, and a past position, which is the position of the occupant's head in the passenger compartment identified in the past. It ’s a computer program that lets you know
When the computer program is executed, the processor
A subset of position data corresponding to the region of the three-dimensional space in which the occupant's head is likely to be present is extracted from the position data set.
The central position of the head is estimated based on the position data subset,
The radius of the sphere that approximates the head centered on the center position is specified as the first radius.
The first radius is compared with the second radius, which is the radius of the sphere that approximates the head at the past position.
When the difference between the first radius and the second radius is less than a predetermined value, the position of the head is specified based on the center position.
Computer program. When the difference between the first radius and the second radius is equal to or greater than a predetermined value, the processor
The center position of the head is re-estimated based on the second radius.
The position of the head is specified based on the re-estimated center position of the head.
The computer program according to claim 11. To the processor
The orientation of the occupant's face is determined based on the position data set.
When the determined orientation of the face is a specific orientation, the center position of the head is estimated.
The computer program according to any one of claims 8 to 12. The position data set is acquired from a TOF (Time of Flight) camera.
The computer program according to any one of claims 8 to 13. A storage medium that stores the computer program according to any one of claims 8 to 14.