JP7053014B2 - Motion judgment device, motion judgment method, and program - Google Patents

Description

The present invention relates to a motion determination device, a motion determination method, and a program.

The driver who drives the vehicle is required to safely operate the audio or the car navigation when it becomes necessary to operate the audio or the car navigation while driving. In recent years, in order to reduce the influence of operations such as audio on driving operations, a technique for performing audio operations or the like in response to a specific motion of the driver has been studied.

Patent Document 1 defines a behavior detection space and describes that the behavior such as the movement of a human hand or finger is recognized in the behavior detection space.

JP-A-2017-97608

A plurality of motions are defined as motions performed by a driver or a passenger (hereinafter referred to as a driver or the like). Among the plurality of motions, there are motions with similar movements. Therefore, in the determination device for determining the motion of the driver or the like, the motion may be determined as a motion different from the intention of the driver or the like who executed the specific motion operation. In such a case, there is a problem that a device in the vehicle such as a car navigation system or an audio system performs an operation different from the intention of the driver or the like.

An object of the present disclosure is to provide a motion determination device, a motion determination method, and a program capable of discriminating similar motions with high accuracy in view of the above-mentioned problems.

The motion determination device according to the first aspect of the present invention performs at least two or more motion operations using an area detection unit that detects a specific area in the imaging area and a specific part of the human body in the specific area. When detected, a motion determination unit that determines that any of the two or more detected motion operations has been performed based on the determination criteria associated with the specific area. Be prepared.

The motion determination method according to the second aspect of the present invention detects a specific area in the imaging area, and detects at least two or more motion operations using a specific part of the human body in the specific area. , It is determined that any of the two or more detected motion operations has been performed based on the determination criteria associated with the specific area.

The program according to the third aspect of the present invention detects a specific area in the imaging area, and detects at least two or more motion operations using a specific part of the human body in the specific area. Based on the determination criteria associated with the specific area, the computer is made to determine that one of the two or more detected motion operations has been performed.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a motion determination device, a motion determination method, and a program capable of identifying similar motions with high accuracy.

It is a block diagram of the motion determination apparatus which concerns on Embodiment 1. FIG. It is a block diagram of the motion determination apparatus which concerns on Embodiment 2. FIG. It is a figure explaining the outline of the motion determination processing executed by the motion determination apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the flow of the motion determination processing executed by the motion determination apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the flow of the motion determination processing executed by the motion determination apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the flow of the motion determination processing executed by the motion determination apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the flow of the motion determination processing executed by the motion determination apparatus which concerns on Embodiment 2. FIG.

(Embodiment 1)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. A configuration example of the motion determination device 10 according to the first embodiment will be described with reference to FIG. 1. The motion determination device 10 may be a computer device operated by the processor executing a program stored in the memory. The motion determination device 10 may be a server device, a personal computer device, or the like.

The motion determination device 10 has an area detection unit 11 and a motion determination unit 12. The area detection unit 11 and the motion determination unit 12 may be software or modules whose processing is executed by the processor executing a program stored in the memory. Alternatively, the area detection unit 11 and the motion determination unit 12 may be hardware such as a circuit or a chip.

The area detection unit 11 detects a specific part of the human body in a specific area in the imaging area. The imaging area is an area that appears in an image captured by a camera. The specific part of the human body may be, for example, a finger, a hand, or the like. The specific area is a part of the area included in the imaging area.

When the motion determination unit 12 detects at least two or more motion operations using a specific part of the human body in the specific area detected by the area detection unit 11, it is determined that one of the motion operations has been performed. do. The motion determination unit 12 determines that any of the two or more detected motion operations has been performed based on the determination criteria associated with the specific area.

The motion operation may be, for example, for a driver or the like in a vehicle to move a specific part of the human body in a predetermined direction or the like in order to operate a device or the like in the vehicle. The device in the vehicle may be, for example, a car navigation device, an audio device, or the like, or may be another device.

The criterion associated with the specific area may indicate the motion operation that is preferentially adopted when a plurality of motion operations are detected in the specific area. When a plurality of specific areas exist, the motion operation preferentially adopted may be different for each specific area. The determination criterion associated with a specific area may indicate, for example, a motion operation to be rejected as a motion operation that is unlikely to be performed by a driver or the like in that area. Alternatively, the determination criterion may indicate the motion operation to be adopted as the motion operation that the driver is likely to perform in the area.

The case where two or more motion operations are detected is, for example, a case where one motion operation cannot be specified from the motion of a driver or the like. When the motion determination unit 12 detects the motion of the driver or the like, the motion determination unit 12 extracts the motion operation associated with the detected motion. At this time, the motion determination unit 12 may extract a plurality of motion operations instead of extracting only one specific motion operation. Information about the motion operation associated with the extracted motion may be stored in advance in a memory or the like in the motion determination device 10.

As described above, when the motion determination device 10 detects two or more motion operations in a specific area, the motion determination device 10 can specify one motion operation according to the determination criteria associated with the specific area. In addition, the determination criteria indicate that motion operations that are likely to be performed by the driver or the like are preferentially adopted in a specific area. Therefore, when the motion determination device 10 detects two or more motion operations, it is possible to improve the probability of selecting the motion operation according to the intention of the driver or the like.

Subsequently, a configuration example of the motion determination device 10 according to the second embodiment will be described with reference to FIG. The motion determination device 10 has an external device interface 61, a processor 62, and a memory 63.

The external device interface 61 may include a baseband processor that performs digital baseband signal processing. Alternatively, the external device interface 61 may include a network interface card (NIC) compliant with the IEEE 802.3 series. Alternatively, the external device interface 61 may be a USB (Universal Serial Bus) interface, a Bluetooth (registered trademark) interface, a wireless LAN (Local Area Network) interface, or the like.

The external device interface 61 is connected to the TOF (Time Of Flight) camera 21 and the monitor 22. The TOF camera is a camera that can measure the distance to the image pickup target. In other words, the TOF camera can calculate the distance data to the image pickup target for each pixel. The image generated by the TOF camera 21 is referred to as a distance image.

The monitor 22 displays the image data output from the external device interface 61. For example, the monitor 22 may display the distance image data generated by the TOF camera 21. Alternatively, the monitor 22 may display the image data after processing or analyzing the distance image data in the motion determination device 10. For example, the monitor 22 may display the 3D point cloud data generated by 3D converting the distance image in the motion determination device 10.

The processor 62 reads software (computer program) from the memory 63, and performs the processing described below using a flowchart or the like. The processor 62 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 62 may include a plurality of processors.

The memory 63 is composed of a combination of a volatile memory and a non-volatile memory. The memory 63 may include storage located away from the processor 62. In this case, the processor 62 may access the memory 63 via an I / O interface (not shown). The processor 62 is used to execute a function or process in the area detection unit 11 and the motion determination unit 12 of FIG. Further, the processor 62 may generate 3D point cloud data by 3D converting the distance image generated by the TOF camera 21.

The memory 63 is used to store software or software modules. The processor 62 reads out these software or software modules from the memory 63 and executes them.

Subsequently, with reference to FIG. 3, an outline of the motion determination process executed by the motion determination device 10 will be described. FIG. 3 shows the steering in the vehicle and the areas defined for each of the top, bottom, left, and right of the steering. FIG. 3 shows a plane on the H-axis and the W-axis. A shaded circular area indicates steering.

Area A includes the area above the steering. Area B includes the area to the right of the steering. Area C includes the area under the steering. Area D includes the area to the left of the steering. Areas A to D are shown as rectangular areas having a size of a distance W1 in the W-axis direction and a distance H1 in the H-axis direction, respectively. In FIG. 3, areas A to D are shown as rectangular areas, but the shape of the area is not limited to a rectangle. The shape of the area may include, for example, a certain area including the steering. Further, the size of the area is not limited to the size shown in FIG. For example, the larger the area, the wider the area to which the motion determination process for distinguishing the paying motion and the tap motion is applied.

Here, the motion determination device 10 executes a motion determination process for distinguishing between pay motion and tap motion. The paying motion is, for example, a motion in which the hand moves at a speed equal to or higher than a predetermined speed after the hand has stopped, and the hand has stopped again. The term "resting" includes not only the case where the hand is completely stationary but also the case where the movement of the hand becomes a speed equal to or lower than a predetermined speed. The tap motion is a motion of tapping a part of the steering included in the areas A to D. Tap may be paraphrased as tap.

Paying motions further include uppaying motions, downpaying motions, left paying motions, and right paying motions. The pay-as-you-go motion is a motion in which the driver moves his / her hand upward at a speed equal to or higher than a predetermined speed. The prepaid motion is a motion in which the driver moves his / her hand downward at a speed equal to or higher than a predetermined speed. The left-handed motion is a motion in which the driver moves his / her hand to the left at a speed equal to or higher than a predetermined speed. The right-handed motion is a motion in which the driver moves his / her hand to the right at a speed equal to or higher than a predetermined speed.

The tap motion in the area A is a motion in which the driver moves his / her hand downward and taps the steering. The tap motion in the area B is a motion in which the driver moves his / her hand to the left and taps the steering wheel. The tap motion in the area C is a motion in which the driver moves his / her hand upward and taps the steering. The tap motion in the area D is a motion in which the driver moves his / her hand to the right and taps the steering wheel.

Both the tap motion and the subpaid motion in the area A include a motion in which the driver moves his / her hand downward. That is, the tap motion in the area A and the prepaid motion are similar to each other. Therefore, the motion determination device 10 may simultaneously detect the tap motion in the area A and the prepaid motion. Further, the motion determination device 10 may simultaneously detect the tap motion in the area B and the left-handed motion. Further, the motion determination device 10 may simultaneously detect the tap motion in the area C and the pay-as-you-go motion. Further, the motion determination device 10 may simultaneously detect the tap motion in the area D and the right-handed motion. Here, the simultaneous detection includes not only the case where the detection is performed at exactly the same timing but also the case where the tap motion and the paying motion are detected with a predetermined period difference. That is, simultaneous detection also includes timing that is considered to be substantially simultaneous detection.

Here, it is unlikely that the driver intentionally executes the paying motion around the steering. When the driver makes a paying motion while driving, he / she takes his / her hand away from the steering to make the paying motion. Further, it is assumed that the driver performs the paying motion in a space that does not come into contact with the steering during the paying motion. Therefore, when the motion determination device 10 simultaneously detects the tap motion and the subpaid motion in the area A, the motion determining device 10 rejects the subpaid motion and determines that the tap motion operation has been performed.

Further, when the motion determination device 10 simultaneously detects the tap motion and the left-paying motion in the area B, the motion determining device 10 rejects the left-paying motion and determines that the tap motion operation has been performed. Further, when the motion determination device 10 simultaneously detects the tap motion and the upper payment motion in the area C, the motion determination device 10 rejects the upper payment motion and determines that the tap motion operation has been performed. Further, when the motion determination device 10 simultaneously detects the tap motion and the right-paying motion in the area D, the motion determining device 10 rejects the right-paying motion and determines that the tap motion operation has been performed.

Subsequently, the flow of the motion determination process in the motion determination device 10 will be described with reference to FIGS. 4 to 7. First, the motion determination unit 12 simultaneously detects pay motion and tap motion (S11). Next, the area detection unit 11 determines whether or not the area in which the pay motion and the tap motion are detected by the motion determination unit 12 is the area A (S12). For example, the area detection unit 11 may detect the steering in the imaging area and specify the positions of the areas A to D with reference to the steering position. For example, the area detection unit 11 identifies a rectangular area determined by using the distance W1 and the distance H1 existing at a position moved upward (+ H direction) by W1 / 2 from the center point of the steering as the position of the area A. You may. Further, the area detection unit 11 identifies a rectangular area determined by using the distance W1 and the distance H1 existing at a position W1 / 2 moved to the right (+ W direction) from the center point of the steering as the position of the area B. You may. The area detection unit 11 also specifies the positions of the area C and the area D in the same manner as the area A and the area B.

Next, when the area detection unit 11 determines that the detected pay motion and tap motion are detected in the area A, the motion determination unit 12 determines whether or not the detected pay motion is a subpaid motion. (S13).

When the motion determination unit 12 determines that the detected pay motion is a subpaid motion, the motion determination unit 12 identifies that the motion operation performed by the driver is a tap motion (S14). That is, the motion determination unit 12 rejects the detected prepaid motion and specifies that the motion operation performed by the driver is a tap motion.

When the motion determination unit 12 determines in step S13 that the detected payment motion is not a subpayment motion, the motion determination unit 12 identifies it as an erroneous detection (S15). In this case, since it is unlikely that the payment motion other than the subpayment and the tap motion are simultaneously detected in the area A, the motion determination unit 12 identifies it as an erroneous detection. Alternatively, the motion determination unit 12 may determine that a predetermined tap motion or a predetermined payment motion has been detected without specifying it as an erroneous detection in step S15. Hereinafter, the same applies to step S24, step S34, and step S44.

If it is determined in step S12 that the area in which the detected paying motion and tap motion are detected is not the area A, the process of FIG. 5 is executed. Hereinafter, the process of FIG. 5 will be described.

The area detection unit 11 determines whether or not the area in which the pay motion and the tap motion are detected by the motion determination unit 12 is the area B (S21).

Next, when the area detection unit 11 determines that the detected pay motion and tap motion are detected in the area B, the motion determination unit 12 determines whether or not the detected pay motion is a left pay motion. (S22).

When the motion determination unit 12 determines that the detected pay motion is a left pay motion, the motion determination unit 12 identifies that the motion operation performed by the driver is a tap motion (S23). That is, the motion determination unit 12 rejects the detected left-paid motion and specifies that the motion operation performed by the driver is a tap motion.

When the motion determination unit 12 determines in step S22 that the detected paying motion is not a left-paying motion, it identifies it as an erroneous detection (S24).

If it is determined in step S21 that the area in which the detected paying motion and tap motion are detected is not the area B, the process of FIG. 6 is executed. Hereinafter, the process of FIG. 6 will be described.

The area detection unit 11 determines whether or not the area in which the pay motion and the tap motion are detected by the motion determination unit 12 is the area C (S31).

Next, when the area detection unit 11 determines that the detected pay motion and tap motion are detected in the area C, the motion determination unit 12 determines whether or not the detected pay motion is an upper payment motion. (S32).

When the motion determination unit 12 determines that the detected pay motion is a top pay motion, the motion determination unit 12 identifies that the motion operation performed by the driver is a tap motion (S33). That is, the motion determination unit 12 rejects the detected prepaid motion and specifies that the motion operation performed by the driver is a tap motion.

When the motion determination unit 12 determines in step S22 that the detected paying motion is not an upper payment motion, the motion determination unit 12 identifies it as an erroneous detection (S34).

If it is determined in step S31 that the area in which the detected paying motion and tap motion are detected is not the area C, the process of FIG. 7 is executed. Hereinafter, the processing of FIG. 7 will be described.

The area detection unit 11 determines whether or not the area in which the pay motion and the tap motion are detected by the motion determination unit 12 is the area D (S41).

Next, when the area detection unit 11 determines that the detected pay motion and tap motion are detected in the area D, the motion determination unit 12 determines whether or not the detected pay motion is a right-pay motion. (S42).

When the motion determination unit 12 determines that the detected pay motion is a right pay motion, the motion determination unit 12 identifies that the motion operation performed by the driver is a tap motion (S43). That is, the motion determination unit 12 rejects the detected right-handed motion and specifies that the motion operation performed by the driver is a tap motion.

When the motion determination unit 12 determines in step S22 that the detected paying motion is not a right-paying motion, it identifies it as an erroneous detection (S44). Further, when the area where the pay motion and the tap motion are detected is not determined to be the area D in step S41, the motion determination unit 12 identifies it as an erroneous detection (S44). That is, since it is unlikely that the paying motion and the tap motion are detected at the same time in an area different from the areas A to D, the motion determination unit 12 identifies it as an erroneous detection.

As described above, when the motion determination device 10 simultaneously detects a pay motion and a tap motion in the areas A to D, the motion determination device 10 rejects the pay motion in a predetermined direction in each area. That is, the motion determination device 10 preferentially adopts tap motion as the motion operation performed by the driver. From this, the motion determination device 10 can specify the motion operation performed by the driver even when the paying motion and the tap motion are simultaneously detected around the steering.

(Embodiment 3)
Subsequently, the motion determination process in the motion determination device 10 according to the third embodiment will be described. In the third embodiment, the motion determination device 10 executes a motion determination process for distinguishing between a paying motion and a motion naturally performed by the driver. The motion that the driver naturally performs includes, for example, a motion of releasing the steering from the state of holding the steering, and a motion of holding the steering from the state of releasing the steering.

In area A of FIG. 3, the driver is expected to move his hand upward when the steering is released. In this case, the motion determination device 10 may erroneously identify a natural motion in which the driver tries to release the steering as a top-paying motion. Further, it is assumed that the driver moves his / her hand downward when he / she grips the steering wheel in the area A. In this case, the motion determination device 10 may erroneously identify the natural motion in which the driver tries to grip the steering as the subpaid motion.

Further, the motion determination device 10 may erroneously identify a natural motion in which the driver tries to grasp or release the steering in the area B as a left-handed motion or a right-handed motion. Further, the motion determination device 10 may erroneously identify a natural motion in which the driver tries to grasp or release the steering in the area C as an upper-payment motion or a lower-payment motion. Further, the motion determination device 10 may erroneously identify a natural motion in which the driver tries to grasp or release the steering in the area D as a right-handed motion or a left-handed motion.

Here, it is unlikely that the driver intentionally executes the paying motion around the steering. When the driver makes a paying motion while driving, he / she takes his / her hand away from the steering to make the paying motion. Further, it is assumed that the driver performs the paying motion in a space that does not come into contact with the steering during the paying motion.

Therefore, when the motion determination unit 12 according to the third embodiment detects an upper payment motion or a lower payment motion in the area A, the detected payment motion is rejected. Further, when the motion determination unit 12 detects a left-paying motion or a right-paying motion in the area B, the motion determining unit 12 rejects the detected paying motion. Further, when the motion determination unit 12 detects a prepaid motion or a prepaid motion in the area C, the motion determination unit 12 rejects the detected paying motion. Further, when the motion determination unit 12 detects a right-paying motion or a left-paying motion in the area D, the motion determining unit 12 rejects the detected paying motion.

As described above, when the motion determination device 10 detects a paying motion in the areas A to D, the motion determining device 10 rejects the paying motion in a predetermined direction in each area. From this, the motion determination device 10 can consider the motion detected as the paying motion in the areas A to D as a natural motion performed by the driver. That is, the motion determination device 10 can determine the motion detected as the paying motion in the areas A to D as a natural motion performed by the driver. As a result, the motion determination device 10 can reduce the erroneous detection of detecting the paying motion in the areas A to D.

In the above example, the program can be stored and supplied to the computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks). Further, examples of non-temporary computer-readable media include CD-ROM (Read Only Memory), CD-R, and CD-R / W. Further, examples of non-temporary computer readable media include semiconductor memory. The semiconductor memory includes, for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (Random Access Memory). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

The present disclosure is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. Further, the present disclosure may be carried out by appropriately combining the respective embodiments.

10 Motion judgment device 11 Area detection unit 12 Motion judgment unit 21 TOF camera 22 Monitor 61 External device interface 62 Processor 63 Memory

Claims (9)

An area detector that detects a specific area in the imaging area,
When at least two or more motion operations using a specific part of the human body are detected in the specific area, the two or more detected motion operations are performed based on the criteria associated with the specific area. It is equipped with a motion determination unit that determines that any motion operation has been performed .
The specific area
The area around the steering in the vehicle,
The motion determination unit is
The motion operation of tapping the steering and the paying motion operation indicating that the specific part has moved at a predetermined speed and then stopped are detected, and the paying motion operation is performed in the area around the steering in the vehicle. Motion judgment device to reject . The specific area
The motion determination device according to claim 1 , which is on a plane including the steering in the vehicle and is an area around the steering. The motion determination unit is
The motion determination device according to claim 1 or 2 , wherein it is determined that the motion operation of tapping the steering is performed in the area around the steering in the vehicle. The motion judgment device
Detects a specific area in the imaging area and
When at least two or more motion operations using a specific part of the human body are detected in the specific area, the two or more detected motion operations are performed based on the criteria associated with the specific area. From among, it is determined that one of the motion operations has been performed,
The specific area
The area around the steering in the vehicle,
When making a determination regarding the motion operation
The motion operation of tapping the steering and the paying motion operation indicating that the specific part has moved at a predetermined speed and then stopped are detected, and the paying motion operation is performed in the area around the steering in the vehicle. Motion judgment method to reject . The specific area
The motion determination method according to claim 4 , which is on a plane including the steering in the vehicle and is an area around the steering. When making a determination regarding the motion operation
The motion determination device
The motion determination method according to claim 4 or 5 , wherein it is determined that the motion operation of tapping the steering is performed in the area around the steering in the vehicle. Detects a specific area in the imaging area and
When at least two or more motion operations using a specific part of the human body are detected in the specific area, the two or more detected motion operations are performed based on the criteria associated with the specific area. From among, it is determined that one of the motion operations has been performed,
The specific area
The area around the steering in the vehicle,
When making a determination regarding the motion operation
The motion operation of tapping the steering and the paying motion operation indicating that the specific part has moved at a predetermined speed and then stopped are detected, and the paying motion operation is performed in the area around the steering in the vehicle. A program that causes a computer to reject . The specific area
The program according to claim 7 , which is a plane including a steering wheel in a vehicle and is an area around the steering wheel. When making a determination regarding the motion operation
The program according to claim 7 or 8 , wherein it is determined that the motion operation of tapping the steering is performed in the area around the steering in the vehicle.

Images