DE112019007569T5 - Operator judging device and operator judging method - Google Patents

Abstract

It is an object of the present invention to provide an operator judging apparatus and method by which an occupant operating an apparatus installed in a vehicle can be correctly judged. The operator judging device according to the present invention includes an image acquisition section for acquiring an image of the occupant in a vehicle picked up by a camera, a facial feature inference section that infers facial features of the occupant based on the image, a head position inference section that based inferring the relative position of the occupant's head to the camera based on the facial features, a body part inference section inferring, based on the image, the size of a plurality of body parts of the occupant in the image, a physique inference section, inferring based on the relative position of the head of the occupant and the size of the respective body parts infers the physique of the occupant, a predictive operation inferring section that infers a predictive operation indicating a prediction of the occupant's operation of a device installed in the vehicle based on a movement of the physique, and an operation ener judging section that judges the occupant operating the device based on the predicted operation.

Description

[Technical Field]

The present invention relates to an operator judging device and an operator judging method for judging an occupant who operates an apparatus installed in a vehicle.

[background technology]

There has hitherto been a technology in which the driver's manipulation of devices installed in the vehicle, such as a navigation device, etc., is disabled while driving a vehicle. With this technology, there is a problem that not only an operation by the driver but also by other occupants including the occupant on the front passenger seat is disabled, thereby degrading the convenience of the device.

As a countermeasure against this problem, there is disclosed a technology (e.g., see Patent Document 1) in which movement of an operator's hand is predicted using an image picked up by a camera, and based on the movement of the hand concerned, the operator is judged . Further, a technology is disclosed (see, e.g., Patent Document 2) in which an operator is judged based on the position of the operator's arm in an image captured by a camera. Such an assessment of the occupant can disable the driver's operation while driving a vehicle and enable the operation of other occupants.

[Prior Art Documents]

[patent documents]

• [Patent Document 1] Patent Publication No. JP 2014-58268 A
• [Patent Document 2] Patent Publication No. JP 2005-274409 A

[Outline of the Invention]

[Problem to be solved by the invention]

In the technology disclosed in Patent Documents 1 and 2, an operator's hand or arm is detected by image processing of an image captured by a camera. However, if the image captured by the camera includes an object other than a hand or an arm, the object in question may be misdetected as a hand or an arm. In this case, there is a possibility of operator misjudgment. Therefore, it cannot be said that the operator who operates an on-vehicle device is correctly judged according to the prior art.

The present invention has been made to solve this problem and has as an object to provide an operator judging apparatus and method by which an operator who operates an apparatus installed in a vehicle can be correctly judged.

[means for solving the task]

In order to achieve the above object, the operator judging device according to the present invention includes an image acquisition section for acquiring an image of the occupant in a vehicle picked up by a camera, a facial feature inference section that derives facial features based on the image acquired by the image acquisition section of the occupant, a head position inference section inferring the relative position of the occupant's head to the camera based on facial features inferred by the facial feature inference section, a body part inference section inferring the size of a plurality of body parts based on the image acquired by the image acquisition section of the occupant in the image, a physique inference section which, based on the relative position of the head of the occupant inferred by the head position inference section and the size of the respective body parts di. inferred by the body part inference section e physique of the occupant infers, a predicted operation inferring section that infers a predicted manipulation indicating a prediction of the manipulation of a device installed in the vehicle by an occupant based on a movement of the physique inferred by the physique inferred section, and an operator judging section, which judges the occupant operating the device based on the predicted operation inferred by the predicted operation inference section.

[Effects of the Invention]

According to the present invention, the operator judging device includes a physique inference section that infers the physique of the occupant based on the relative position of an occupant's head inferred by the head position inference section and the size of the respective body parts inferred by the body part inference section, a prediction operation - an inference section that infers, based on a movement of the physique inferred by the physique inference section, a prediction operation indicating a prediction of an occupant's operation of a device installed in the vehicle, and an operator judging section that judges the occupant operating the device based on the predicted operation inferred by the predicted operation inferring section, thereby enabling correct judgment of the operator operating the device installed in the vehicle.

The object, characteristics, aspects and advantages of the present invention will become clearer from the following detailed explanation and attached figures.

character list

• [ 1 ] 1 14 is a block diagram showing an example of the configuration of an operator judgment device according to an embodiment of the present invention.
• [ 2 ] 1 14 is a block diagram showing an example of the configuration of an operator judgment device according to an embodiment of the present invention.
• [ 3 ] 3 14 is a view showing an example of outputting a judgment result of the operator judgment device according to an embodiment of the present invention.
• [ 4 ] 4 14 is a view showing an example of outputting a judgment result of the operator judgment device according to an embodiment of the present invention.
• [ 5 ] 5 12 is a flow chart showing an example of the operation of an operator judgment device according to an embodiment of the present invention.
• [ 6 ] 6 12 is a flow chart showing an example of the operation of an operator judgment device according to an embodiment of the present invention.
• [ 7 ] 7 Fig. 12 is a block diagram showing an example of the hardware construction of an operator judgment device according to an embodiment of the present invention.
• [ 8th ] 8th Fig. 12 is a block diagram showing an example of the hardware construction of an operator judgment device according to an embodiment of the present invention.
• [ 9 ] 9 14 is a block diagram showing an example of the structure of an operator judgment system according to an embodiment of the present invention.

[Forms for carrying out the invention]

In the following, forms for carrying out the present invention are explained with reference to the figures.

<Embodiments>

<structure>

1 12 is a block diagram showing an example of the configuration of an operator judgment device 1 according to the present embodiment. 1 FIG. 12 shows the required minimum configuration of an operator judgment device according to the present embodiment. Furthermore, it is assumed here that the operator evaluation device 1 is installed in a vehicle.

As in 1 1, the operator judgment device 1 comprises an image acquisition section 2, a facial feature inference section 3, a head position inference section 4, a body part inference section 5, a physique inference section 6, a predictive operation inference section 7, and an operator judgment section 8. Further, FIG the image acquisition section 2 is connected to a camera 9 . The camera 9 is, as for example in 3 and 4 shown, which will be described below, installed in a vehicle.

The image acquisition component 2 acquires an image of the occupant in a vehicle captured by the camera 9 . The facial feature inference section 3 infers facial features of the occupant based on the image acquired by the image acquisition component 2 . The head position inference section 4 infers the relative position of the occupant's head to the camera 9 based on the facial features inferred by the facial feature inference section 3. The body part inference section 5 infers the size of a plurality of body parts based on the image acquired by the image acquisition component 2 of the occupant in the picture.

The physique inference section 6 infers the physique of the occupant based on the relative position of the occupant's head inferred by the head position inference section 4 and the size of the respective body parts inferred by the body part inference section 5 . The predicted-operation inferring section 7 infers, based on a movement of the physique inferred by the physique inferred by the physique inferring section 6, a predicted manipulation indicating a prediction of the occupant's manipulation of a device installed in the vehicle. The operator judgment section 8 judges based on the prediction inferred by the prediction operation inference section 7 tion operation, the occupant operating the device.

Next, another configuration of an operator judging device using the in 1 operator judging device 1 shown.

2 12 is a block diagram showing an example of the configuration of an operator judgment device 10 according to another configuration. Here, it is assumed that the operator judgment device 10 is installed in a vehicle.

As in 2 1, the operator judging device 10 includes an image acquisition section 2, a facial feature inference section 3, a head position inference section 4, a body part inference section 5, a physique inference section 6, a predicted operation inference section 7, an operator judgment section 8, and a judgment result - Output section 11. Further, the image acquisition section 2 is connected to a camera 9. The judgment result output section 11 is connected to a CID (Center Information Display) 12 . The CID 12 is installed in an instrument cluster (center cluster) in a vehicle. The instrument cluster is the center section of the dashboard and corresponds to the space between the driver's seat and the passenger's seat.

The image acquisition component 2 acquires an image of the occupant in a vehicle captured by the camera 9 . The camera 9 captures at least the occupants seated on the driver's seat and the front passenger's seat, respectively. The recording by the camera 9 can also be performed such that not only the occupants seated in the driver's seat and the front passenger's seat but also occupants seated in the rear seat are included.

The facial feature inference section 3 infers facial features of the occupant based on the image acquired by the image acquisition component 2 . More specifically, the facial feature inference section 3 compares the image acquired by the image acquisition section 2 with a prepared learning dictionary (not shown), and infers features of the faces showing the position of the occupant's face, eyes, nose, etc., which are included in the through the image acquired in the image acquiring section 2 are included. For example, the facial feature inference section 3 infers, as features of the faces, the position and size of the pupils, the distance between the two eyes, which is the distance between the centers of the respective pupils of the two eyes, the position of both ends of the eyebrows, the position of the both ends of the eyes, the position of the tip of the nose, the position of the both ends of the mouth, etc. of the occupant, not limited to these, and other features of the faces can also be inferred. The facial features of a large number of people are recorded in the learning lexicon. When the image includes a plurality of occupants, the facial feature inference section 3 infers the facial features of the respective occupants.

The head position inference section 4 infers the relative position of the occupant's head to the camera 9 based on the facial features inferred by the facial feature inference section 3. More specifically, the head position inference section 4 infers based on the distance between the two eyes or the size of the occupant's pupils , inferred by the facial feature inference section 3, the physical relative position of the occupant's head to the camera 9. That is, the head position inference section 4 infers the distance between the camera 9 and the occupant's head.

In the present embodiment, the relative position of the occupant's head to the camera 9 is inferred based on the distance between the two eyes or the size of the occupant's pupils, in which an influence of physical differences is small. As a result, regardless of the physique of the occupant, the relative position of the occupant's head to the camera 9 can be precisely inferred. Further, even if the seating position in which the occupants are seated changes, the relative position of the occupant's head to the camera 9 can be precisely inferred according to the change.

The body part inference section 5 infers, based on the image acquired by the image acquisition component 2, the size of a plurality of occupant's body parts in the image. More specifically, the body part inference section 5 compares the image acquired by the image acquisition section 2 with a prepared learning dictionary (not shown), and infers the size and position in the image of the occupant's respective body parts such as neck, shoulders, arms, hands, torso, etc. included in the image acquired by the image acquiring section 2. In the learning dictionary, the size and position in the image of the respective body parts of a plurality of people are recorded. When the image includes a plurality of occupants, the body part inference section 5 infers the size of a plurality of body parts of the respective occupants. The unit of the size of the respective body parts of the occupant in the image inferred by the body part inference section 5 is pix (pixel).

The physique inference section 6 infers the physique of the occupant based on the relative position of the occupant's head to the camera 9 inferred by the head position inference section 4 and the size of the respective body parts of the occupant in the image inferred by the body part inference section 5 . More specifically, the physique inference section 6 first converts the size of the occupant's respective body parts in the image inferred by the body part inference section 5 into the actual size of the occupant's respective body parts. When there are a plurality of occupants, the physique inference section 6 infers the actual size of the respective body parts of the respective occupants.

The method by which the physique inference section 6 converts the shoulder width of an occupant in the image inferred by the body part inference section 5 into the actual shoulder width will now be explained. The physique inference section 6 calculates an actual shoulder width d using the following expression (1).
[Equation 1] $$\text{i.e}=\frac{\text{<figure-callout id="2" label="Mn" filenames="DE112019007569T5\_0002.png,DE112019007569T5\_0003.png" state="{{state}}">Mn</figure-callout>}}{2\times \text{tan}\left(\theta /2\right)}$$

In expression (1), n indicates the size of the shoulder width in the image. M indicates the transverse width of the image and is determined by the angle of view θ of the camera 9. Indicated by x is the distance between the camera 9 and the occupant's head, and corresponds to the relative position of the occupant's head to the camera 9 inferred by the head position inference section 4. In this way, the physique inference section 6 converts the size of each Body parts in the image inferred by the body part inference section 5 by correction using the relative position of the occupant's head inferred by the head position inference section 4 into the actual size of the respective occupant's body parts.

As expression (1) shows, the physique inference section 6 calculates the actual size of the respective body parts of the occupant by using the relative position of the occupant's head to the camera 9 inferred by the head position inference section 4 . As a result, even if the seating position in which the occupants are seated changes, the physique inference section 6 can correctly calculate the actual size of the respective body parts of the occupant. The calculation of the shoulder width of an occupant has been explained above, but a similar calculation can be made with respect to other body parts of an occupant. For example, the physique inference section 6 can also calculate the seated height of the occupant, the size of the face, and the length of the arms.

Conversion of the size of each occupant's body part in the image to the actual size of each occupant's body part is possible because the relative position of the occupant's head to the camera 9 is considered to be all the body parts of the occupant. While it is conceivable that the shoulders, spine and face could be in substantially the same location as the relative position of the occupant's head to the camera 9, since the arms can move extensively around the shoulders, this is not limited. Also, regarding the arms, when the size of the arms has become maximum by a change in the chronological history of the images, the body part inference section 5 can infer the size of the arms that has become the maximum as the size of the occupant's arms. In this case, the physique inference section 6 converts the size of the occupant's arms in the image inferred by the body part inference section 5 into the actual size of the occupant's arms.

Next, the physique inference section 6 infers the physique of the occupant based on the actual size of the occupant's respective body parts. More specifically, the physique inference section 6 inputs the actual size of the respective body parts of the occupant to a learning device (not shown) that has completed a learning process in which mechanical learning has been performed to infer the physique of an occupant, and acquires from the learning device through executing arithmetic processing of the learning device an inference result of the physique of the occupant. When there are a plurality of occupants, the physique inference section 6 infers the physiques of the respective occupants.

The learning device has a learning dictionary (not shown) in which the actual size of the respective body parts of a large number of people is recorded in connection with the physique of the respective people. For example, when the seat height, shoulder width, face size, and arm length are input from the physique inference section 6 to the learning device, the learning device compares the actual size of the input body parts with the learning dictionary and infers the physique with the highest probability as the physique of the occupant of the inputted respective body parts. In this way, there is a correlation between the actual size of the respective body parts of the occupant and the physique of the occupant. The physique inference section 6 acquires the inference result of the occupant's physique from the learning device.

The predicted manipulation inferring section 7 infers, based on a movement of the physique of an occupant inferred by the physique inferred by the physique inferring section 6, a predicted manipulation indicating a prediction of the manipulation of the CID 12 by the occupant. For example, the prediction manipulation inferring section 7 infers a prediction manipulation of the CID 12 by the occupant when the movement of the occupant's physique from the shoulder to the hand is a movement directed to the CID 12 .

The prediction operation inferring section 7 is not limited to a movement of the occupant's physique from the shoulder to the hand, so a prediction operation can also be inferred based on a movement of the physique of another part of the occupant's body. Further, the predictive operation inferring section 7 is not limited to a movement directed toward the CID 12, so a predictive operation can be inferred based on another movement as well.

The predicted operation inferring section 7 can infer a predicted operation not only based on a movement of the physique of an occupant but also based on a movement of the occupant's joints. For example, flexion of the occupant's elbow joint to the inside is possible, while flexion to the outside is not possible. Thus, since the movement range of the joints is limited, by considering not only a movement of the physique but also a movement of the joints, the precision of the prediction operation can be improved.

In the prediction operation inference section 7, it is not limited to a movement of an occupant's physique, but inference of a prediction operation based on the line of sight and the direction of the face of an occupant is also possible. The line of sight and the direction of the face of an occupant can also be detected by image processing of an image acquired by the image acquisition section 2 . For example, the prediction manipulation inferring section 7 may conclude that the occupant is not a prediction manipulation when the physical movement of the occupant concerned is directed to the CID 12 but the line of sight and the face direction of the occupant are not directed to the CID 12 are. Thus, since not only a movement of the physique but also the line of sight and the direction of the face of the occupant are taken into account, the precision of the prediction operation can be improved. It is also possible for the prediction operation inference section 7 to consider only either the eye direction or the face direction of the occupant. Further, the predicted operation inferring section 7 can also infer a predicted operation considering a movement of the joints.

The prediction operation inferring section 7 can also infer a prediction operation using a learning dictionary (not shown). In this case, the predictive manipulation inferring section 7 infers a predicted manipulation of an apparatus by the occupant by comparing the movement of a physique inferred by the physique inferring section 6 with the learning dictionary. In the learning lexicon, a prediction operation of devices by a plurality of people is recorded.

The operator judging section 8 judges the occupant who operates the CID 12 based on the predicted manipulation inferred by the predicted manipulation inferring section 7 . For example, assume a vehicle whose steering wheel is on the right-hand side in the direction of travel. In this case, when the movement from the shoulder to the hand of an occupant's right side is a movement toward the CID 12, the operator judging section 8 judges that the operator of the CID 12 is the occupant of the passenger seat acts. Further, when the movement from the shoulder to the hand of an occupant's left side is a movement toward the CID 12, the operator judging section 8 judges that the operator of the CID 12 is the passenger seat occupant.

The above explained an example in which an occupant is judged by a movement from the occupant's shoulder to the hand, but is not limited to this. As explained above, a predictive operation inferred by the predictive operation inference section 7 is not only a movement of an occupant's physique, but also includes a movement of the occupant's joints, the occupant's gaze direction, and the occupant's face direction . The operator judging section 8 judges the operator of an apparatus based on a predicted manipulation inferred by the predicted manipulation inferring section 7, i. H. of the movement of an occupant inferred by the prediction operation inference section 7 . Thereby, the operator judging section 8 can judge whether an occupant seated in either the driver's seat, the front passenger's seat, or the rear seat has operated the CID 12 .

The judgment result output section 11 gives the CID 12 a judgment result indicating the occupant who the operator judging section 8 has judged operates the CID 12 .

The CID 12 performs operation enabling or disabling control according to the occupant who operates the relevant CID 12 . For example, in the case that the driver operates the CID 12 as in 3 shown, operation by the driver concerned is deactivated. In the example in 3 an “X” sign will be displayed to represent that operation of the CID 12 screen is disabled, which may be any piece of information as long as that information represents that operation is disabled. Further, in the case that the passenger seat occupant operates the CID 12 as in FIG 4 shown, the operation activated by the relevant occupant of the front passenger seat. In the example in 4 “OK” is displayed, which represents that the operation of the screen of the CID 12 is enabled, which can be any information as long as this information represents that the operation is enabled.

In 3 and 4 illustrates the case where information representing that an operation is disabled or enabled is displayed, but is not limited to this. For example, voice output is also possible, which indicates that an operation is deactivated or activated. Furthermore, it is also possible that information is displayed which represents that the operation is deactivated or activated, and a voice output also takes place which indicates that the operation is deactivated or activated.

The case where the judgment result output section 11 outputs the judgment result to the CID 12 has been explained above, but is not limited to this. An output example of the judgment result is explained below.

For example, the judgment result output section 11 can also output the judgment result to an air conditioner installed in the vehicle. In this case, the air conditioner can deactivate the relevant operation when the driver operates the air conditioner, and activate the relevant operation when the passenger seat occupant operates the air conditioner. Furthermore, the air conditioner can also change the wind direction toward the operator of the air conditioner in question.

The judgment result output section 11 can also output the judgment result to an audio device installed in the vehicle. In this case, the audio device can deactivate the relevant operation when the driver operates the audio device, and activate the relevant operation when the audio device is operated by the passenger seat occupant. Furthermore, the audio device can also play back a piece of music corresponding to the taste of the user of the audio device in question.

<How it works>

5 is a flowchart showing an example of the operation of the in 1 shown operator judging device 1 shows.

In step S<b>11 , the image acquisition component 2 acquires an image of the occupant of a vehicle captured by the camera 9 . In step S<b>12 , based on the image acquired by the image acquisition component 2 , the facial feature inference section 3 infers facial features of the occupant.

In step S13, the head position inference section 4 infers the relative position of the occupant's head to the camera 9 based on the facial features inferred by the facial feature inference section 3 Size of a variety of body parts of the occupant in the image.

In step S15, based on the relative position of the occupant's head inferred by the head position inference section 4 and the size of the respective body parts inferred by the body part inference section 5, the physique inference section 6 infers the physique of the occupant. In step S16, based on a movement of the physique of an occupant inferred by the physique inference section 6, the predicted manipulation inferring section 7 infers a predicted manipulation indicating a prediction of the manipulation of a device installed in the vehicle by an occupant. In step S17, based on the predicted operation inferred by the predicted operation inference section 7, the operator judging section 8 judges the occupant who operates the apparatus.

6 is a flowchart showing an example of the operation of the in 2 shown operator evaluation device 10 shows. Step S21 to Step S27 in 6 correspond to step S11 to step S17 in 5 , so an explanation is omitted here. Step S28 will be explained below.

In step S<b>28 , the judgment result output section 11 outputs a judgment result indicating the occupant that the operator judgment section 8 judged to operate the CID 12 to a device.

<Effects>

According to the present embodiments, the head position inference section 4 can precisely infer the relative position of the occupant's head to the camera 9 based on the distance between the two eyes or the size of the occupant's pupils, in which an influence of physique differences is small. The physique inference section 6 can precisely infer the physique of the occupant by using the precisely inferred relative position of the occupant. The predicted manipulation inferring section 7 can precisely infer a predicted manipulation based on a movement of an occupant's exact physique. In this way, the operator judging device 1, 10 can accurately judge the operator operating an apparatus installed in the vehicle even if the seating position at which the occupants are seated changes.

<Hardware structure>

The respective functions of the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation inference section 7, the operator judgment section 8 and the judgment result output section 11 in the The operator judging device 10 explained above are realized by means of a processing circuit. That is, the operator judgment apparatus 1, 10 comprises a processing circuit for acquiring an image of the occupant in a vehicle picked up by a camera, inferring the facial features of the occupant, inferring the relative position of the occupant's head to the camera, for inferring the size of a plurality of body parts of the occupant in the image, inferring the physique of the occupant, inferring a predictive operation of the occupant, and judging the operator. The processing circuit can be special hardware, but it can also be formed by means of a processor (also referred to as CPU (Central Processing Unit), processing device, computing device, microprocessor, microcomputer, DSP (Digital Signal Processor)) which programs stored in a memory executes

When the processing circuit is special hardware, the processing circuit corresponds to 13 as shown in FIG 7 shown, e.g. B. a simple circuit, a complex circuit, a programmed processor, parallel programmed processors, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or a combination thereof. The respective functions of the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation inference section 7, the operator judgment section 8 and the judgment result output section 11 can be respective processing circuits 13 can be realized, but the respective functions can also be realized by a single processing circuit 13 as a whole.

If the processing circuit 13 is a processor 14, as in 8th shown, the respective functions of the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation inference section 7, the operator judgment section 8 and the judgment result output section 11 implemented by software, firmware, or a combination of software and firmware. The software or firmware is presented as a program and is stored in a memory 15 . The processor 14 reads out programs recorded in the memory 15 and executes them, thereby realizing the respective functions. That is, the operator judgment device 1, 10 includes a memory 15 for storing programs through which finally a step of acquiring an image of the occupant in a vehicle picked up by a camera, a step of inferring the facial features of the occupant, a step of inferring the relative position of the occupant's head to the camera, a step of inferring the size of a plurality of body parts of the occupant in the image, a step of inferring the physique of the occupant, a step of inferring a prediction operation of the occupant, and a step be executed to evaluate the operator. It can also be said that these programs are the sequence or the method for executing the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation inference section 7, operator judgment section 8 and judgment result output section 11 on a computer. The memory can be non-volatile or volatile semiconductor memory such as e.g. a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), etc., a magnet disk, floppy disk, optical disk, compact disk, DVD (Digital Versatile Disc), etc., or various storage media used in the future.

Regarding the respective functions of the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation inference section 7, the operator judgment section 8 and the judgment result output section 11 it is also possible that part of the functions are realized by means of special hardware and other functions are realized by software or firmware.

Here, the processing circuit can realize the respective functions described above by hardware, software, firmware, or a combination thereof.

<system structure>

The operator evaluation device explained above can be applied not only to an onboard navigation device, i. H. a car navigation device, but also to a navigation device constructed as a system in which a vehicle-mountable PND (Portable Navigation Device) and a vehicle-provided server, etc. are appropriately combined, or a device other than one Navigation device are applied. In this case, the respective functions of the operator judgment device or the respective constituent parts can be distributed to the respective functions constituting the system.

Concretely, as an example, the function of the operator judgment device can be arranged on a server. As in 9 As shown, for example, a vehicle includes a camera 9 and a CID 12. Further, a server 16 includes the image acquisition section 2, the facial feature inference section 3, the head position inference section 4, the body part inference section 5, the physique inference section 6, the prediction operation- inference section 7, the operator judgment section 8, and the judgment result outputting section 11. By constructing like this, an operator judgment system can be constructed.

Thus, even with a structure in which the respective functions of the operator judgment device are distributed to the respective functions constituting the system, the same effects as the above embodiment can be obtained.

The software for executing the operation in the above embodiment can also be installed on a server, for example. In the operator judgment method realized by executing this software through the server, an image of the occupant in a vehicle picked up by a camera is acquired based on the acquired image, facial features of the occupant are inferred based on the inferred ones Facial features infer the relative position of the occupant's head to the camera, based on the acquired image, the size of a variety of occupant's body parts in the image is inferred, based on the relative position of the occupant's head and the size of each body part becomes the physique of the occupant, based on a movement of the inferred physique, a prediction operation indicating a prediction of operation of an apparatus installed in the vehicle by an occupant is inferred, and based on the inferred prediction operation, the occupant operating the apparatus is judged.

In this way, since the software for executing the operation in the above embodiment is installed and operated on a server, the same effects as in the above embodiment are obtained.

In the present invention, it is possible to appropriately modify or omit forms of embodiment within the scope of the invention.

The present invention has been explained in detail, and the above explanation is an example in all aspects, and the invention is not limited thereto. Also innumerable modified examples not illustrated by way of example are contemplated as conceivable without departing from the scope of this invention.

Reference List

1

operator evaluation device

2

image acquisition section

3

Facial Feature Inference Section

4

Head position inference section

5

Body part inference section

6

Physis Inference Section

7

prediction operation inference section

8th

Operator Judgment Section

9

camera

10

operator evaluation device

11

judgment result output section

12

CID

13

processing circuit

14

processor

15

Storage

16

server

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• JP 2014058268 A [0003]
• JP2005274409A [0003]

Claims (9)

Operator evaluation device comprising an image acquisition section for acquiring an image of the occupant in a vehicle picked up by a camera, a facial feature inference section that infers facial features of the occupant based on the image acquired by the image acquisition section, a head position inference section that infers the relative position of the occupant's head to the camera based on facial features inferred by the facial feature inference section, a body part inference section that infers the size of a plurality of body parts of the occupant in the image based on the image acquired by the image acquisition section, a physique inference section that infers the physique of the occupant based on the relative position of the occupant's head inferred by the head position inference section and the size of the respective body parts inferred by the body part inference section, a prediction operation inference section that infers a prediction operation based on a movement of the physique inferred by the physique inference section, indicating a prediction of the occupant's operation of a device installed in the vehicle, and an operator judging section that judges the occupant operating the apparatus based on the predicted manipulation inferred by the predicted manipulation inferring section. operator judging device claim 1 , characterized in that the head position inference section infers the relative position of the occupant's head to the camera based on the distance between the two eyes or the size of the occupant's pupils. operator judging device claim 1 , characterized in that the predicted operation inferring section infers the predicted operation based on a movement of joints of the occupant. operator judging device claim 1 , characterized in that the predictive operation inferring section infers the predictive operation based on the line of sight and the face direction of the occupant. operator judging device claim 1 , characterized in that the physique inference section corrects the size of the respective body parts inferred by the body part inference section using the relative position of the occupant's head inferred by the head position inference section and the actual size of the respective body parts of the occupant is charged. operator judging device claim 5 , characterized in that the actual size of the respective body parts of the occupant includes the seat height, the shoulder width, the size of the face and the length of the arms. operator judging device claim 5 , characterized in that the physique inference section inputs the actual size of the respective body parts of the occupant into a learning apparatus that has completed a learning process in which mechanical learning has been performed to infer the physique of the occupant, and in that arithmetic processing of the Learning device is executed, from the learning device acquires the inference result of the physique of the occupant. operator judging device claim 1 characterized by further comprising a judgment result outputting section for outputting a judgment result representing the occupant judged by the operator judging section to operate the apparatus. Operator Evaluation Process by which an image of an occupant in a vehicle captured by a camera is acquired, facial features of the occupant are inferred based on the acquired image, based on the inferred facial features, the relative position of the occupant's head to the camera is inferred, based on the acquired image, the size of a variety of body parts of the occupant in the image is inferred, based on the inferred relative position of the occupant's head and the inferred size of the respective body parts, the physique of the occupant is inferred, based on a movement of the inferred physique, inferring a prediction operation indicating a prediction of the occupant's operation of a device installed in the vehicle, and based on the inferred prediction operation, the occupant operating the device is judged.

Images