JP2019179389A - Motion determination device, motion determination method, and program - Google Patents

Abstract

To provide a motion determination device capable of stably detecting the specific motion performed by a driver or the like.SOLUTION: A motion determination device includes: a detection part 11 for detecting stillness of a specific part of a human body in a specific area within an imaging area; and a motion determination part 12 for determining whether or not the motion operation using the specific part has been performed according to the detection result. The motion determination part 12 maintains a state where the stillness of the specific part for a predetermined period is detected when the specific part is no longer detected after the stillness of the specific part is detected in the specific area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a motion determination device, a motion determination method, and a program.

  A driver who drives a vehicle is required to perform these operations safely when it becomes necessary to perform an audio operation or a car navigation operation while driving. 2. Description of the Related Art In recent years, in order to reduce the influence of operations such as audio on driving operations, studies on techniques for performing audio operations or the like according to specific behaviors of the driver have been advanced. The specific behavior may be, for example, that the hand is stationary in a specific space in the vehicle. In this case, for example, a motion determination device installed in the vehicle detects a specific motion of the driver, such as selecting a specific position on the operation screen.

  Patent Document 1 describes that a behavior detection space is defined and a behavior such as the movement of a human hand or a finger is recognized in the behavior detection space.

JP 2017-97608 A

  A driver or a passenger (hereinafter referred to as a driver or the like) cannot visually confirm the behavior detection space, and thus it is difficult to accurately recognize the behavior detection space. Therefore, a specific motion may be executed near the boundary of the behavior detection space. In addition, even when the driver or the like holds the hand still and executes a specific motion, the hand may not move completely and may move slightly near the boundary of the behavior detection space. In such a case, the motion determination device repeats detection and non-detection output as the detection result of the motion executed by the driver or the like in a short time, and there is a problem that the detection result cannot be stabilized. is there.

  In view of the above-described problems, an object of the present disclosure is to provide a motion determination device, a motion determination method, and a program that can stably detect a specific motion performed by a driver or the like.

  The motion determination apparatus according to the first aspect of the present invention includes a detection unit that detects the stasis of a specific part of a human body in a specific area in an imaging area, and a motion that uses the specific part according to the detection result. A motion determination unit that determines whether or not an operation has been performed, and the motion determination unit no longer detects the specific part after the stationary part of the specific part is detected in the specific area. If this occurs, the state where the stationary state of the specific part is detected for a predetermined period is maintained.

  The motion determination method according to the second aspect of the present invention detects a stationary part of a specific part of a human body in a specific area within an imaging area, and performs a motion operation using the specific part according to the detection result. If the specific part is not detected after the stationary part is detected in the specific area, the stationary part is detected for a predetermined period. maintain.

  The program according to the third aspect of the present invention detects whether a specific part of a human body is detected in a specific area within an imaging area, and whether a motion operation using the specific part is performed according to the detection result. If the specific part is no longer detected after the specific part is detected in the specific area, the state in which the specific part is detected for a predetermined period is maintained. Let the computer do that.

  According to the present invention, it is possible to provide a motion determination device, a motion determination method, and a program capable of stably detecting a specific motion performed by a driver or the like.

1 is a configuration diagram of a motion determination apparatus according to a first embodiment. It is a block diagram of the motion determination apparatus concerning Embodiment 2. It is a figure explaining the outline | summary of the motion determination which the motion determination apparatus concerning Embodiment 2 performs. It is a figure which shows the state transition regarding the area selection state concerning Embodiment 2. FIG. It is a figure which shows the flow of the motion determination process concerning Embodiment 2. FIG. It is a figure which shows the flow of the motion determination process concerning Embodiment 2. FIG. It is a figure which shows the flow of the motion determination process concerning Embodiment 2. FIG.

(Embodiment 1)
Embodiments of the present invention will be described below with reference to the drawings. A configuration example of the motion determination apparatus 10 according to the first exemplary embodiment will be described with reference to FIG. The motion determination device 10 may be a computer device that operates when a processor executes a program stored in a memory. The motion determination device 10 may be a server device or a personal computer device.

  The motion determination device 10 includes a detection unit 11 and a motion determination unit 12. The detection unit 11 and the motion determination unit 12 may be software or a module in which processing is executed by a processor executing a program stored in a memory. Alternatively, the detection unit 11 and the motion determination unit 12 may be hardware such as a circuit or a chip.

  The detection unit 11 detects the stasis of a specific part of the human body in a specific area within the imaging area. The imaging area is an area that appears in an image captured using a camera. The specific area is a partial area included in the imaging area. The specific part of the human body may be, for example, a finger, a hand, a head, a foot, or the like. Detecting stillness means not only when a specific part of the human body is completely stationary, but also when a specific part of the human body moves within a predetermined distance within a predetermined time, Good. That is, the stationary state detected by the detection unit 11 includes a state regarded as substantially stationary.

  The motion determination unit 12 determines whether a motion operation using a specific part has been performed according to the detection result in the detection unit 11. For example, the motion determination unit 12 may determine that the driver or the like has performed a motion operation when detecting that the hand is stationary in a specific area.

  Also, the motion determination unit 12 maintains the state where the stationary part of the specific part is detected for a predetermined period when the specific part is not detected after the stationary part of the specific part is detected in the specific area. That is, in the state where the motion determination unit 12 determines that the motion operation using the specific part is performed, the motion operation is not performed for a predetermined period even when the specific part is not detected. No determination is made that there is no. The motion determination unit 12 maintains a state where it is determined that a motion operation is being performed for a predetermined period.

  As described above, the motion determination device 10 detects the stasis of a specific part in a specific area, and after that, even when the stasis of the specific part can no longer be detected, The detected state can be maintained. As a result, the detection unit 11 can prevent repeated detection and non-detection outputs as detection results within a short period of time as a stationary detection result of a specific part.

(Embodiment 2)
Next, a configuration example of the motion determination apparatus 10 according to the second embodiment will be described with reference to FIG. The motion determination device 10 includes an external device interface 61, a processor 62, and a memory 63.

  The external device interface 61 may include a baseband processor that performs digital baseband signal processing. Alternatively, the external device interface 61 may include a network interface card (NIC) compliant with IEEE 802.3 series. Alternatively, the external device interface 61 may be a USB (Universal Serial Bus) interface, a Bluetooth (registered trademark) interface, a wireless LAN (Local Area Network) interface, or the like.

  The external device interface 61 is connected to a TOF (Time Of Flight) camera 21 and a monitor 22. The TOF camera is a camera that can measure a distance to an imaging target. In other words, the TOF camera can calculate the distance data to the imaging target for each pixel. An image generated by the TOF camera 21 is referred to as a distance image.

  The monitor 22 displays the image data output from the external device interface 61. For example, the monitor 22 may display distance image data generated by the TOF camera 21. Alternatively, the monitor 22 may display the image data after processing or analyzing the distance image data in the motion determination device 10. For example, the monitor 22 may display 3D point cloud data generated by performing 3D conversion of the distance image in the motion determination device 10.

  The processor 62 reads software (computer program) from the memory 63, and performs processing described below using flowcharts and the like. The processor 62 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 62 may include a plurality of processors.

  The memory 63 is configured by a combination of a volatile memory and a nonvolatile memory. Memory 63 may include storage located remotely from processor 62. In this case, the processor 62 may access the memory 63 via an I / O interface not shown. The processor 62 is used to execute functions or processes in the detection unit 11 and the motion determination unit 12 in FIG. Further, the processor 62 may generate 3D point cloud data by performing 3D conversion on the distance image generated in the TOF camera 21.

  The memory 63 is used for storing software or software module groups. The processor 62 reads out these software or software module groups from the memory 63 and executes them.

  Next, an outline of motion determination performed by the motion determination unit 12 will be described with reference to FIG. FIG. 3 shows a 3D point group generated by 3D conversion of the distance image generated by the TOF camera 21. The 3D point group in FIG. 3 uses the L axis, the W axis, and the H axis. The H axis is an axis perpendicular to the ground. The L axis is an axis in the traveling direction of the vehicle driven by the human body. The W axis is an axis perpendicular to the H axis and the L axis.

  The L axis, the W axis, and the H axis are orthogonal to each other. FIG. 3 shows a 3D point cloud including a driver in the vehicle. FIG. 3 shows a state where the driver is holding the steering wheel. Further, the 3D point cloud of FIG. 3 is generated based on imaging data or a distance image taken from the left front of the driver.

  The motion determination unit 12 detects a hand indicated by the 3D point group and determines whether a motion operation using the hand is performed. For example, the motion determination unit 12 detects that the hand is stationary in the setting area shown in FIG. The motion determination unit 12 may determine that the driver or the like has performed a motion operation when detecting that the hand is stationary within the setting area. The motion operation in which the driver or the like stops his / her hand in a specific area may be referred to as area selection. That is, the driver selects a specific space in the vehicle with his / her hand. Depending on the space (that is, the setting area) selected by the driver using the hand, the operation content to be executed may be different. The setting area is a space recognized by the detection unit 11, and in fact, the driver or the like cannot visually recognize the setting area. The setting area corresponds to the specific area described in the first embodiment.

  Moreover, although FIG. 3 shows that there is one setting area, there may be two or more setting areas. For example, it is assumed that a new route is proposed from a car navigation device installed in a vehicle while the driver is driving. In such a case, if you place your hand in one of the two setting areas, the new route is selected, and if you place your hand in the other setting area, the conventional route is selected. May be associated.

  Next, the transition of the determination state managed by the motion determination unit 12 will be described with reference to FIG. First, it is assumed that the motion determination unit 12 has been initialized and has not determined the motion of the driver or the like. Here, the motion determination unit 12 determines that the hand of the driver or the like is not detected in the setting area and the motion operation is not performed. This state is a state in which the driver or the like has not made an area selection, or no selection.

  Next, the motion determination unit 12 determines that the motion of the hand is detected by detecting the stillness of the hand within the setting area. This state is a state in which the driver or the like has made an area selection and is being selected. That is, it can be said that the determination state managed by the motion determination unit 12 has transitioned from the state of no selection to the state of selection.

  The motion determination unit 12 starts or starts the monitoring timer when the determination state transitions to the selected state. The timer value or count value of the monitoring timer increases with time. Further, when the monitoring timer is reset, the value is returned to 0. The motion determination unit 12 measures a period during which the determination state managed by the motion determination unit 12 is maintained using a monitoring timer. In addition, when the determination state is selected, the motion determination unit 12 resets the monitoring timer when the stillness of the hand is detected in the setting area.

  Next, the motion determination unit 12 determines that the stillness of the hand cannot be detected in the setting area in the selected state, that is, the stillness of the hand in the setting area has not been detected. This state is assumed to be a state in which the driver or the like continues to perform area selection and is extending. That is, it can be said that the determination state managed by the motion determination unit 12 has transitioned from the selected state to the extended state. The extended state is a state in which the motion determination unit 12 determines that a motion operation has been performed. In other words, the determination state managed by the motion determination unit 12 is not extended to the state of no selection, but is extended even when the stationary state of the hand in the setting area cannot be detected in the selected state. Transition to the state of.

  Next, it is assumed that the motion determination unit 12 determines that the stillness of the hand cannot be detected in the setting area before the predetermined duration expires or ends in the extended state. In this case, the determination state managed by the motion determination unit 12 transitions from the extended state to the unselected state when the duration time has expired. In addition, the motion determination unit 12 determines that the stillness of the hand can be detected in the setting area before the predetermined duration expires or ends in the extended state. In this case, the determination state managed by the motion determination unit 12 transitions from the extended state to the selected state.

  The motion determination unit 12 may initialize, for example, a state recognized by the motion determination unit 12 when the motion determination process is finished in a state of no selection, selection, or extension. The initialization state may be a state in which no state is set.

  Next, the flow of the motion determination process according to the second embodiment will be described with reference to FIGS. 5 to 7 are diagrams in which the state transition diagram in FIG. 4 is replaced with a processing flow. First, the processing in the “no selection” determination state will be described with reference to FIG. The motion determination unit 12 determines whether or not stillness of the hand has been detected in the setting area in the “no selection” determination state (S11).

  In the determination state “no selection”, the motion determination unit 12 transitions the managed determination state to “selected” when the stillness of the hand is detected in the setting area (S12). The motion determination unit 12 maintains the determination state of “no selection” when the stillness of the hand is not detected in the setting area in the determination state of “no selection” (S13).

  Next, the process in the “selected” determination state will be described with reference to FIG. The motion determination unit 12 determines whether or not stillness of the hand is detected in the setting area in the “selected” determination state (S21).

  In the selected state, when the stillness of the hand is not detected in the setting area, the motion determination unit 12 shifts the managed state to “extending” (S22). The motion determination unit 12 resets the monitoring timer when the stillness of the hand is detected in the setting area in the selected state (S23). Next, the motion determination unit 12 maintains the “selected” determination state (S24).

  Next, the processing in the “extended” determination state will be described with reference to FIG. The motion determination unit 12 determines whether or not the duration time has expired in the extended state (S31). The duration may be started when the state transitions to the selected state, or may be started when the state transitions to the extending state. The motion determination unit 12 determines whether the timer value or the count value indicated by the monitoring timer has passed the duration.

  When determining that the duration has expired in the “extended” determination state, the motion determination unit 12 transitions the managed determination state to “no selection” (S32). If it is determined that the duration has not expired and is within the duration, the motion determination unit 12 determines whether or not stillness of the hand has been detected within the setting area (S33).

  If the motion determination unit 12 is being extended and a stillness of the hand is detected within the duration, the motion determination unit 12 shifts the managed determination state to being selected (S34). Also, the motion determination unit 12 maintains the extended state when the extension is being performed and the stillness of the hand is not detected within the duration (S35).

  As described above, the motion determination unit 12 is in the selected state where the driver has performed the area selection, even if the stillness of the hand is not detected in the area, before the transition to the no selection state, Transition to the extending state. The extended state is a state in which the driver maintains the state where the area is selected. For this reason, the motion determination unit 12 maintains the state in which the driver is performing the area selection for a predetermined duration even when the stillness of the hand is no longer detected in the selected state. be able to. That is, the motion determination unit 12 can prevent switching between a state where the area is selected and a state where the area is not selected in a short time.

  In the above example, the program can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks). Furthermore, examples of non-transitory computer-readable media include CD-ROM (Read Only Memory), CD-R, and CD-R / W. Further examples of non-transitory computer readable media include semiconductor memory. The semiconductor memory includes, for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (Random Access Memory). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  Note that the present disclosure is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present disclosure. In addition, the present disclosure may be implemented by appropriately combining the respective embodiments.

DESCRIPTION OF SYMBOLS 10 Motion determination apparatus 11 Detection part 12 Motion determination part 21 TOF camera 22 Monitor 61 External apparatus interface 62 Processor 63 Memory

Claims (22)

A detection unit for detecting the rest of a specific part of the human body in a specific area within the imaging area;
A motion determination unit that determines whether a motion operation using the specific part is performed according to the detection result,
The motion determination unit
A motion determination device that maintains a state in which the stationary part of the specific part is detected for a predetermined period when the specific part is not detected after the stationary part of the specific part is detected in the specific area. The motion determination unit
When the specific part is no longer detected after the specific part is detected in the specific area, a monitoring timer is started and a period for maintaining the state where the specific part is detected is maintained. The motion determination apparatus according to claim 1, which measures. The motion determination unit
A state in which the specific area is selected using the specific part, a state in which the specific area is selected using the specific part, and an extended state in which the state in which the specific area is selected is maintained. The motion determination apparatus according to claim 1, wherein a state without selection indicating that the specific area is not selected is managed using a specific part. The motion determination unit
The motion determination device according to claim 3, wherein, in the selected state, when it is not possible to detect stillness of the specific part, the motion determination apparatus transitions to the extended state. 5. The motion determination unit according to claim 3, wherein, when the stationary part cannot be detected in the extended state, the motion determination unit shifts to a non-selected state after the end of the period for maintaining the extended state. Motion judgment device. The motion determination unit
6. The motion determination device according to claim 3, wherein, when the stationary state of the specific part is detected in the extended state, the state is changed to the selected state. 6. The imaging area includes a plurality of the specific areas,
The motion determination unit
The motion determination apparatus according to any one of claims 1 to 6, wherein a process executed when a motion operation is performed is associated with each specific area and managed. The imaging area is
The motion determination apparatus according to claim 1, wherein a 3D point cloud is used to indicate a position of the imaging target by performing 3D conversion on a distance image indicating a distance to the imaging target. Detects stillness of a specific part of the human body in a specific area within the imaging area,
It is determined whether a motion operation using the specific part is performed according to the detection result,
A motion determination method that maintains a state in which the stasis of the specific part is detected for a predetermined period when the specific part is not detected after the stasis of the specific part is detected in the specific area. When determining whether the motion operation has been performed,
When the specific part is no longer detected after the specific part is detected in the specific area, a monitoring timer is started and a period for maintaining the state where the specific part is detected is maintained. The motion determination method according to claim 9, wherein measurement is performed. When determining whether the motion operation has been performed,
A state in which the specific area is selected using the specific part, a state in which the specific area is selected using the specific part, an extended state in which the state in which the specific area is selected is maintained, The motion determination method according to claim 9 or 10, wherein a state without selection indicating that the specific area is not selected is managed using a specific part. When determining whether the motion operation has been performed,
The motion determination method according to claim 11, wherein, when the stationary state of the specific part cannot be detected in the selected state, the state is shifted to the extended state. When determining whether the motion operation has been performed,
13. The motion determination method according to claim 11 or 12, wherein, in the extended state, when it is not possible to detect the stationary state of the specific part, the state is changed to a non-selected state after the period for maintaining the extended state is ended. When determining whether the motion operation has been performed,
The motion determination method according to any one of claims 11 to 13, wherein when the stationary state of the specific part is detected in the extending state, the state is changed to the selected state. The imaging area is
The motion determination method according to claim 9, wherein a 3D point cloud is used to indicate a position of the imaging target by performing 3D conversion on a distance image indicating a distance to the imaging target. Detects stillness of a specific part of the human body in a specific area within the imaging area,
It is determined whether a motion operation using the specific part is performed according to the detection result,
When the specific part is no longer detected after the specific part is detected in the specific area, the computer is caused to maintain the state where the specific part is detected for a predetermined period. program. When determining whether the motion operation has been performed,
When the specific part is no longer detected after the specific part is detected in the specific area, a monitoring timer is started, and a period for maintaining the state where the specific part is detected is maintained. The program according to claim 16, which measures. When determining whether the motion operation has been performed,
A state in which the specific area is selected using the specific part, a state in which the specific area is selected using the specific part, and an extended state in which the state in which the specific area is selected is maintained. The program according to claim 16 or 17, which manages a state without selection indicating that the specific area is not selected using a specific part. When determining whether the motion operation has been performed,
19. The program according to claim 18, wherein, when the stationary state of the specific part cannot be detected in the selected state, the program makes a transition to the extended state. When determining whether the motion operation has been performed,
20. The program according to claim 18, wherein, in the extension state, when it is not possible to detect stillness of the specific part, the program is shifted to a non-selection state after the end of the period for maintaining the extension state. When determining whether the motion operation has been performed,
The program according to any one of claims 18 to 20, wherein when the stationary state of the specific part is detected in the extending state, the state is changed to the selected state. The imaging area is
The program according to any one of claims 16 to 21, wherein a 3D point cloud is used to indicate a position of the imaging target by performing 3D conversion on a distance image indicating a distance to the imaging target.

Images