WO2024201878A1 - 動作分析装置、動作分析方法、及び動作分析プログラム - Google Patents

動作分析装置、動作分析方法、及び動作分析プログラム Download PDF

Info

Publication number
WO2024201878A1
WO2024201878A1 PCT/JP2023/013122 JP2023013122W WO2024201878A1 WO 2024201878 A1 WO2024201878 A1 WO 2024201878A1 JP 2023013122 W JP2023013122 W JP 2023013122W WO 2024201878 A1 WO2024201878 A1 WO 2024201878A1
Authority
WO
WIPO (PCT)
Prior art keywords
trajectory
target
time
motion analysis
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/013122
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勝大 草野
尚吾 清水
孝之 小平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to JP2023541030A priority Critical patent/JP7350222B1/ja
Priority to PCT/JP2023/013122 priority patent/WO2024201878A1/ja
Publication of WO2024201878A1 publication Critical patent/WO2024201878A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • This disclosure relates to a motion analysis device, a motion analysis method, and a motion analysis program.
  • Patent Literature 1 proposes a method for automating the measurement of a worker's cycle time and the analysis of the work content in order to reduce the burden of measuring the cycle time of a worker and analyzing the work content.
  • the time series transition of posture information obtained from a video is defined as a movement trajectory, and the work content is identified by comparing it with a trajectory learned in advance.
  • this method by using posture information based on data acquired by a camera and a sensor, no burden is placed on the worker who is the subject of the analysis.
  • Patent Literature 1 in order to analyze the work time and work procedure of a line worker in a factory, the work time and work procedure are detected by matching the trajectories of the joints before and after the analysis target time including the analysis target time with the trajectories of the joints in the registered routine work.
  • a movement different from the movement in the routine work such as skipping a work or non-routine work
  • the trajectories of the joints of the worker are different from the registered trajectories, so there is a possibility that the trajectories of the joints may be erroneously matched.
  • Patent Literature 1 does not disclose a technology to reduce the possibility of erroneously matching the trajectories of the joints.
  • the present disclosure aims to reduce the possibility of erroneously matching the trajectory of a target body part of a target worker with the trajectory of the target body part during routine work in a technique for matching the trajectory of the target body part of a target worker with the trajectory of the target body part during routine work.
  • the motion analysis device comprises: The system is provided with a matching unit that performs a third matching process to extract a trajectory of a target body part of a target worker from a trajectory indicated by reference trajectory information that indicates the trajectory of the body part corresponding to the target body part when the target worker is performing the work to be performed by the target worker, the trajectory being a third target trajectory derived based on the input video, as a third similar trajectory from a trajectory indicated by reference trajectory information that indicates the trajectory of the body part corresponding to the target body part when the target worker is performing the work to be performed by the target worker.
  • the matching unit performs matching in a subsequent period based on the target frame. Therefore, according to the present disclosure, even if an action different from that in routine work occurs in a period before and after the target frame, matching can be performed appropriately as long as routine work is performed in the latter half of the period before and after. Therefore, according to the present disclosure, in a technology that matches the trajectory of a target body part of a target worker with the trajectory of a target body part in routine work, it is possible to reduce the possibility of erroneously matching the trajectory of the target body part.
  • FIG. 1 is a diagram showing an example of the configuration of a motion analysis apparatus 100 according to a first embodiment.
  • 4 is a diagram for explaining processing of the motion analysis apparatus 100 according to the first embodiment.
  • 5A to 5C are diagrams for explaining the processing of a matching unit 130 according to the first embodiment.
  • FIG. 2 is a diagram showing an example of a hardware configuration of the motion analysis apparatus 100 according to the first embodiment.
  • 4 is a flowchart showing the operation of the motion analysis apparatus 100 according to the first embodiment.
  • 1A and 1B are diagrams illustrating the effect of the motion analysis apparatus 100 according to the first embodiment, where FIG. 1A illustrates matching in a preceding and succeeding periods, and FIG. 1B illustrates matching in a previous period.
  • FIG. 1A and 1B are diagrams illustrating the effect of the motion analysis apparatus 100 according to the first embodiment, where FIG. 1A illustrates matching in a preceding and succeeding periods, and FIG. 1B illustrates matching in a previous period.
  • FIG. 1A illustrates
  • FIG. 13 is a diagram showing an example of a hardware configuration of a motion analysis apparatus 100 according to a modification of the first embodiment.
  • 11 is a flowchart showing the operation of the motion analysis apparatus 100 according to the second embodiment.
  • 13 is a flowchart showing the operation of the motion analysis apparatus 100 according to the third embodiment.
  • motion analysis apparatus 100 includes a skeleton extraction unit 110, a trajectory extraction unit 120, a matching unit 130, and a similarity comparison unit 140.
  • motion analysis apparatus 100 stores reference trajectory information 190.
  • the skeleton extraction unit 110 acquires an input video from an external source, extracts the skeleton of the target worker in each frame included in the acquired input video, and generates skeleton information indicating the skeleton extracted in each frame.
  • the skeleton extraction unit 110 identifies the joint positions of the target worker in each frame included in the acquired input video, and generates information indicating the identified joint positions in each frame as skeleton information.
  • the input video is an image output by a camera, and is an image showing the target worker performing work. The image is typically a video. It is assumed that the target worker should be performing routine work.
  • the input video may be information output by a 3D (3-Dimensions) sensor, and may be information indicating the movements of the target worker. Data acquired by a 3D sensor may also be called an image. Information indicating the posture of the target worker acquired by the 3D sensor at each time may also be called a frame.
  • the trajectory extraction unit 120 extracts three patterns of trajectories for the target body part of the target worker based on the target frame from the skeleton information generated by the skeleton extraction unit 110, and generates trajectory information indicating the trajectories of each extracted pattern.
  • the target body part is, for example, a joint or a part of the skeleton.
  • the target frame is a frame selected from the frames included in the input video.
  • the three patterns of trajectories of the target part based on the target frame are composed of a trajectory of the target part in a pre- and post-period based on the target frame, a trajectory of the target part in a previous period based on the target frame, and a trajectory of the target part in a subsequent period based on the target frame.
  • the pre- and post-period based on the target frame is a continuous period from a time earlier than the target time to a time later than the target time.
  • the previous period based on the target frame is a continuous period from a time earlier than the target time to the target time.
  • the subsequent period based on the target frame is a continuous period from a time earlier than the target time to a time later than the target time.
  • the target frame corresponds to a frame that is a target for recognizing the work content of the target worker.
  • the target time is a time corresponding to the target frame, and as a specific example, it is the time when the target frame is photographed by a camera or the time when the posture of the target worker is observed by a sensor.
  • the start time and end time of the pre- and post-period may be determined in any manner.
  • the start time of the previous period may be determined in any manner.
  • the end time of the subsequent period may be determined in any manner.
  • the first half of the trajectory of the target part in the previous and subsequent periods may be set as the trajectory of the target part in the previous period, and the second half of the trajectory of the target part in the previous and subsequent periods may be set as the trajectory of the target part in the subsequent period.
  • the trajectory of the target part in the previous period and the trajectory of the target part in the subsequent period may be combined as the trajectory of the target part in the previous period.
  • the trajectory of the target part in the previous period is also referred to as the first object trajectory.
  • the trajectory of the target part in the previous and subsequent periods is also referred to as the second object trajectory.
  • the trajectory of the target part in the subsequent period is also referred to as the third object trajectory.
  • the first object trajectory is the trajectory of the target part in the previous period and is a trajectory derived based on the input video.
  • the second object trajectory is the trajectory of the target part in the previous and subsequent periods and is a trajectory derived based on the input video.
  • the third object trajectory is the trajectory of the target part and is a trajectory derived based on the input video.
  • the trajectory extraction unit 120 extracts each of the first object trajectory, the second object trajectory, and the third object trajectory from the skeleton information generated by the skeleton extraction unit 110.
  • the trajectory extraction unit 120 extracts three patterns of trajectories from the skeletal information for each target part.
  • the matching unit 130 performs a first matching process, a second matching process, and a third matching process.
  • the first matching process is a process of extracting a trajectory similar to a first object trajectory as a first similar trajectory from a trajectory indicated by the reference trajectory information 190.
  • the second matching process is a process of extracting a trajectory similar to a second object trajectory as a second similar trajectory from a trajectory indicated by the reference trajectory information 190.
  • the third matching process is a process of extracting a trajectory similar to a third object trajectory as a third similar trajectory from a trajectory indicated by the reference trajectory information 190.
  • the matching unit 130 finds a similar section corresponding to the trajectory of each pattern by matching the trajectory of each pattern indicated by the trajectory information generated by the trajectory extraction unit 120 with the trajectory indicated by the reference trajectory information 190. At this time, the matching unit 130 calculates a similarity corresponding to the similar section corresponding to the trajectory of each pattern for the trajectory of each pattern.
  • the matching process is a process of extracting a portion similar to or corresponding to the trajectory of each pattern indicated by the trajectory information from the trajectory indicated by the reference trajectory information 190.
  • the similar section is a continuous section indicated by the reference trajectory information 190 and corresponds to a certain time range. In the similar section corresponding to the trajectory of the target pattern, the similarity between the trajectory of the target pattern and the trajectory indicated by the reference trajectory information 190 is relatively high.
  • the similarity may be calculated in any way.
  • the similar section corresponding to the target period may be a section corresponding to the work content that the target worker should originally be doing.
  • the matching unit 130 identifies a frame identifier corresponding to the target frame in each similar section based on the reference trajectory information 190.
  • the frame identifier corresponding to the target frame is a frame identifier identified based on the matching result, and is a frame identifier corresponding to the frame corresponding to the target frame among the frames corresponding to each time point of the trajectory indicated by the reference trajectory information 190.
  • the frame identifier is an identifier that identifies a frame corresponding to each time point of the trajectory indicated by the reference trajectory information 190, and is a frame number as a specific example.
  • the frame corresponding to each time point of the trajectory indicated by the reference trajectory information 190 is, as a specific example, a frame that is the source of generation of the trajectory of the target part at each time point.
  • the matching unit 130 may infer the work content that the target worker should perform at the time corresponding to the target frame based on the matching result using a frame corresponding to a time earlier than the time corresponding to the target frame and the reference trajectory information 190 or information indicating the work content that the target worker should perform, estimate the position of the similar section based on the inferred work content, and perform matching taking into consideration the inferred result.
  • the similarity comparison unit 140 outputs information corresponding to the work content corresponding to the estimated work trajectory.
  • the estimated work trajectory is a trajectory corresponding to the highest similarity among the similarity between the first object trajectory and the first similar trajectory, the similarity between the second object trajectory and the second similar trajectory, and the similarity between the third object trajectory and the third similar trajectory, and is a trajectory indicated by the reference trajectory information 190.
  • the similarity comparison unit 140 compares the similarities corresponding to the trajectories of each pattern found by the matching unit 130, and outputs a frame identifier in a similar section corresponding to the trajectory of the pattern with the highest corresponding similarity, the frame identifier identified by the matching unit 130.
  • the reference trajectory information 190 is information indicating the trajectory of a part corresponding to the target part when a worker is performing routine work, and is information indicating a frame identifier corresponding to each time point of the trajectory of the target part. Routine work corresponds to work that the worker should perform.
  • a specific example of the part corresponding to the target part is a part that is the same as the target part, or a part that is equivalent to the target part.
  • the trajectory of the target part indicated by the reference trajectory information 190 may indicate a change in the relative position of the target part with respect to a part other than the target part or a work target, or may indicate the range of the position of the target part at each time point.
  • the information indicating the frame identifier corresponds to information corresponding to the work content corresponding to each time point of the trajectory indicated by the reference trajectory information 190.
  • the reference trajectory information 190 may be time-series data that is skeletal information for multiple cycles acquired in advance.
  • a cycle is a unit indicating one period of work that is performed periodically.
  • the reference trajectory information 190 may be time-series data in which multiple cycles of skeletal information are compiled into one cycle by, for example, calculating an average value of the multiple cycles of skeletal information, or may be a model in which the trajectory of the target part is machine-learned using the multiple cycles of skeletal information.
  • Fig. 2 is a diagram for explaining the process of the motion analysis device 100.
  • the skeleton extraction unit 110 extracts a target joint as a target part of the target worker from the input video, and then the trajectory of the target joint is extracted by the trajectory extraction unit 120.
  • the input trajectory information is trajectory information generated by the trajectory extraction unit 120, and is information indicating the trajectory of the target joint of the target worker.
  • the reference trajectory information 190 is information indicating the trajectory of a target joint, which is a target part of a worker shown in a reference video.
  • the reference trajectory information 190 is generated by extracting a target joint of the worker shown in the reference video and extracting the trajectory of the extracted target joint.
  • the reference video is the same as the input video.
  • the trajectory of the target joint is divided into a plurality of sections according to the corresponding work content, and a label is assigned to each section.
  • the matching unit 130 matches the trajectory of the target joint indicated by the input trajectory information with the trajectory of the section corresponding to label 2 of the trajectory of the target joint indicated by the reference trajectory information 190.
  • the matching unit 130 specifies a frame identifier corresponding to the target frame and of a frame included in the reference video, based on the joint position corresponding to the target frame and the joint position indicated by the reference trajectory information 190.
  • the task content corresponding to label 2 has been specified as the task content corresponding to the target frame.
  • FIG. 3 is a diagram for explaining the process of the matching unit 130.
  • the input trajectory information and the reference trajectory information 190 are each represented by a graph.
  • the input trajectory information according to this example is information showing the trajectory of the target part in the period before and after the target frame.
  • the horizontal axis of the graph shows each frame included in the input video or the reference video in the order of the shooting time corresponding to each frame, and the vertical axis of the graph shows the coordinates of the position of the target part.
  • the circles in the graph show the position of the target part in each frame included in the input video or the reference video.
  • the solid lines in each graph are lines drawn by complementing the position of the target part in each frame.
  • the dashed lines in the graph corresponding to the reference trajectory information 190 show the shape of the graph corresponding to the input trajectory information. Note that, although the position of the target part is expressed one-dimensionally in this example, the position of the target part may be expressed two-dimensionally or three-dimensionally.
  • the matching unit 130 performs matching based on the shape of the graph and the application of the coordinates of the target portion, as indicated by the dashed line in the graph corresponding to the reference trajectory information 190. After that, the matching unit 130 acquires a frame identifier corresponding to the target frame based on the matching result.
  • FIG. 4 shows an example of the hardware configuration of the motion analysis device 100 according to this embodiment.
  • the motion analysis device 100 is composed of a computer.
  • the motion analysis device 100 may be composed of multiple computers.
  • the motion analysis device 100 is a computer equipped with hardware such as a processor 11, a storage device 12, a communication device 13, and an input/output interface 14. These pieces of hardware are appropriately connected via signal lines 19.
  • the processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), and a graphics processing unit (GPU).
  • the motion analysis apparatus 100 may include a plurality of processors that replace the processor 11. The plurality of processors share the role of the processor 11.
  • the memory device 12 includes a volatile memory device and a non-volatile memory device.
  • a specific example of the volatile storage device is a RAM (Random Access Memory).
  • Specific examples of the non-volatile storage device include a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a flash memory.
  • the communication device 13 is a receiver and a transmitter.
  • a specific example of the communication device 13 is a communication chip or a NIC (Network Interface Card).
  • the input/output interface 14 is a port to which an input device and an output device are connected.
  • a specific example of the input/output interface 14 is a USB (Universal Serial Bus) terminal.
  • a specific example of the input device is a keyboard and a mouse.
  • a specific example of the output device is a display.
  • Each part of the motion analysis device 100 may use the communication device 13 and the input/output interface 14 as appropriate when communicating with other devices, etc.
  • the storage device 12 stores a motion analysis program.
  • the motion analysis program is a program that causes a computer to realize the functions of each part of the motion analysis device 100.
  • the motion analysis program is executed by the processor 11.
  • the functions of each part of the motion analysis device 100 are realized by software.
  • Data used when executing the motion analysis program and data obtained by executing the motion analysis program are appropriately stored in the storage device 12.
  • Each part of the motion analysis device 100 uses the storage device 12 as appropriate. Note that the terms data and information may have the same meaning.
  • the storage device 12 may be independent of the computer.
  • the motion analysis program may be recorded on a computer-readable non-volatile recording medium.
  • Specific examples of the non-volatile recording medium include an optical disk or a flash memory.
  • the motion analysis program may be provided as a program product.
  • FIG. 5 is a flowchart showing an example of the operation of the motion analysis device 100. The operation of the motion analysis device 100 will be explained using FIG. 5.
  • Step S101 An input image is input from the outside to the skeleton extraction unit 110.
  • the input image shows a target worker.
  • the skeleton extraction unit 110 extracts the skeleton of the target worker in each frame included in the input video, and generates skeleton information indicating the skeleton extracted in each frame.
  • the trajectory extraction unit 120 selects a target frame, and extracts three patterns of trajectories for the target part based on the selected target frame from the skeletal information generated by the skeleton extraction unit 110. After that, the trajectory extraction unit 120 generates trajectory information indicating the trajectory of each extracted pattern.
  • Step S104 The matching unit 130 finds similar sections corresponding to the trajectories of each pattern by matching the trajectory of each pattern indicated by the trajectory information generated by the trajectory extraction unit 120 with the trajectory indicated by the reference trajectory information 190. At this time, the matching unit 130 calculates the similarity corresponding to the similar section corresponding to the trajectory of each pattern for the trajectory of each pattern. Furthermore, the matching unit 130 identifies a frame identifier corresponding to the target frame in each similar section based on the reference trajectory information 190.
  • Step S105 The similarity comparing unit 140 compares the similarities corresponding to the trajectories of the patterns obtained by the matching unit 130, and outputs a frame identifier corresponding to the trajectory of the pattern having the highest corresponding similarity.
  • matching is performed with the trajectory indicated by the reference trajectory information 190 using three patterns of trajectories based on the target frame.
  • at least one of the trajectory in the period immediately before the operation skip or non-routine operation occurs and the trajectory immediately after the operation skip or non-routine operation ends is usually considered to be similar to a part of the trajectory indicated by the reference trajectory information 190. Therefore, according to this embodiment, even if an operation skip or non-routine operation occurs, at least one of the three patterns of trajectories can usually be appropriately matched with a part of the trajectory indicated by the reference trajectory information 190.
  • the frame identifier corresponding to the target frame can be identified with relatively high accuracy.
  • the previous period is a period in which the target worker performed routine work. That is, according to this embodiment, even if a movement different from the routine work occurs in the latter half of the preceding and following periods, if matching is performed using the trajectory in the previous period, the similarity in the matching result will be high because the trajectory in the latter half of the preceding and following periods is not used.
  • the similarity in the matching result will be high. Therefore, according to this embodiment, even if a task is skipped or non-routine task occurs, the task corresponding to the target frame can be identified with relatively high accuracy, and the accuracy of the work content being performed by a worker who should be performing routine tasks can be analyzed with relatively high accuracy.
  • the matching unit 130 When performing matching for a target frame, the matching unit 130 according to this modification uses an absolute coordinate system or a relative coordinate system as the coordinate system of the trajectory indicated by the input trajectory information, instead of using a coordinate system having the coordinate corresponding to the target frame as the origin. In particular, the matching unit 130 uses a relative coordinate system or an absolute coordinate system as the coordinate system of the third target trajectory. In addition, the matching unit 130 appropriately stores the matching results, particularly the matching results for a future period. When matching results for a period overlapping with the target period are stored, the matching unit 130 reuses the stored matching results when performing matching for the target period.
  • the target period is typically a previous period based on the target frame or a period before and after the target frame.
  • the matching unit 130 reuses the stored matching results as the matching results for the previous period.
  • the trajectory extraction unit 120 may extract only a trajectory in the subsequent period instead of extracting three patterns of trajectories.
  • the matching unit 130 may use the matching result in the subsequent period corresponding to a time earlier than the time corresponding to the target frame when performing matching in the previous period and the preceding and following periods.
  • the matching results in the subsequent period based on the target frame can be reused for matching in periods other than the subsequent period. Therefore, according to this modified example, the matching results in the subsequent period based on the target frame can be reused for matching corresponding to a time later than the time corresponding to the target frame.
  • the matching results in the first subsequent period based on the first target frame can be reused for matching in the second previous period and the second previous and subsequent periods based on the second target frame.
  • the second target frame is a frame corresponding to a time later than the time when the first target frame was acquired. There is an overlapping period between the first subsequent period and the second previous period, and there is also an overlapping period between the first subsequent period and the second previous and subsequent periods.
  • FIG. 7 shows an example of the hardware configuration of a motion analysis apparatus 100 according to this modified example.
  • the motion analysis device 100 includes a processing circuit 18 instead of the processor 11 or the processor 11 and the memory device 12 .
  • Processing circuitry 18 is hardware that realizes at least a portion of each component of motion analysis device 100.
  • the processing circuitry 18 may be dedicated hardware, or may be a processor that executes a program stored in the storage device 12 .
  • processing circuitry 18 When processing circuitry 18 is dedicated hardware, processing circuitry 18 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.
  • the motion analysis device 100 may include a plurality of processing circuits replacing the processing circuit 18. The plurality of processing circuits share the role of the processing circuit 18.
  • motion analysis device 100 some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.
  • Processing circuitry 18 is illustratively implemented in hardware, software, firmware, or a combination thereof.
  • Processor 11, memory device 12, and processing circuit 18 are collectively referred to as “processing circuitry.”
  • processing circuitry the functions of each of the functional components of motion analysis device 100 are realized by the processing circuitry.
  • Motion analysis apparatus 100 according to other embodiments may also have a similar configuration to this modified example.
  • Embodiment 2 The following mainly describes the differences from the above-described embodiment with reference to the drawings.
  • the configuration of motion analysis apparatus 100 according to the present embodiment is similar to the configuration of motion analysis apparatus 100 according to the first embodiment.
  • the matching unit 130 according to the present embodiment performs a first matching process, and when the similarity between the first object trajectory and the first similar trajectory is lower than a first threshold, performs each of the second matching process and the third matching process.
  • the first threshold may be determined in any manner. Specifically, when performing matching, the matching unit 130 first performs matching in a previous period. Furthermore, when the similarity in the previous period is sufficiently high, the matching unit 130 does not perform matching in either the previous or following period or the subsequent period.
  • Fig. 8 is a flowchart showing an example of the operation of motion analysis apparatus 100. The operation of motion analysis apparatus 100 will be described with reference to Fig. 8. Note that in this flowchart, each period is a period based on a target frame.
  • Step S201 The trajectory extraction unit 120 extracts the trajectory of the target part in the previous period from the skeletal information.
  • Step S202 The matching unit 130 performs matching in a previous period by using the trajectory extracted by the trajectory extraction unit 120 and the reference trajectory information 190 .
  • Step S203 If the similarity corresponding to the matching result in step S202 is lower than the first threshold, motion analysis apparatus 100 proceeds to step S204. Otherwise, motion analysis apparatus 100 ends the process of this flowchart. It is preferable that the first threshold value is set to a relatively high value in order to prevent erroneous determination.
  • the trajectory extraction unit 120 extracts the trajectories of the target part in each of the before-and-after period and the after-period from the skeletal information.
  • Step S205 The matching unit 130 uses the trajectory extracted by the trajectory extraction unit 120 in step S204 and the reference trajectory information 190 to perform matching in each of the previous and following periods.
  • Step S206 The similarity comparing unit 140 outputs a frame identifier corresponding to the highest similarity among the similarity corresponding to the previous period, the similarity corresponding to the previous and following periods, and the similarity corresponding to the following period.
  • Embodiment 3 The following mainly describes the differences from the above-described embodiment with reference to the drawings.
  • the configuration of motion analysis apparatus 100 according to the present embodiment is similar to the configuration of motion analysis apparatus 100 according to the first embodiment.
  • matching is first performed in a previous period.
  • the similarity corresponding to the previous period is low, it is considered that a missed task or a non-routine task occurred in the previous period. Therefore, when the target frame is considered to be a frame acquired within the target period, only matching is performed in a subsequent period based on the target frame.
  • the target period is a period in which a missed task or a non-routine task occurs in the work of the target worker.
  • the similarity corresponding to the subsequent period based on the target frame at the time when the time corresponding to the target frame has left the target period is higher than the similarity corresponding to the subsequent period based on the target frame corresponding within the target period. Therefore, when the similarity corresponding to the subsequent period becomes sufficiently high after the time corresponding to the target frame has entered the target period, the process returns to performing only matching in the previous period. Note that the target frame is advanced to the next frame in order.
  • the matching unit 130 performs a third matching process if, when the first matching process is performed, the similarity between the first object trajectory and the first similar trajectory is lower than the second threshold. Furthermore, the matching unit 130 performs a first matching process if, when the third matching process is performed, the similarity between the third object trajectory and the third similar trajectory is lower than the third threshold.
  • the second threshold and the third threshold may each be determined in any manner.
  • the similarity comparison unit 140 in this embodiment outputs a frame identifier corresponding to the matching result.
  • Fig. 9 is a flowchart showing an example of the operation of the motion analysis apparatus 100.
  • the operation of the motion analysis apparatus 100 will be described with reference to Fig. 9.
  • each period is a period based on the target frame.
  • the target frame is sequentially advanced to a frame corresponding to a future time, and the process of this flowchart is executed each time the target frame is advanced.
  • Step S301 If the matching target period is the previous period, the motion analysis apparatus 100 proceeds to step S302. Otherwise, the motion analysis apparatus 100 proceeds to step S306.
  • Step S303 This step is similar to step S202.
  • Step S304 If the degree of similarity in the matching result in step S303 is lower than the second threshold, motion analysis apparatus 100 proceeds to step S305. Otherwise, motion analysis apparatus 100 ends the processing of this flowchart.
  • Step S306 The trajectory extraction unit 120 extracts the trajectory of the target part in the subsequent period from the skeletal information.
  • Step S308 If the degree of similarity in the matching result in step S307 is lower than the third threshold, motion analysis apparatus 100 proceeds to step S309. Otherwise, motion analysis apparatus 100 ends the processing of this flowchart.
  • Step S309 The motion analysis apparatus 100 changes the matching target period to the previous period. Note that, when the similarity corresponding to the previous period is lower than the similarity corresponding to the subsequent period, the motion analysis apparatus 100 does not need to change the matching target period to the previous period.
  • Processor 11 Processor, 12 Storage device, 13 Communication device, 14 Input/output interface, 18 Processing circuit, 19 Signal line, 100 Motion analysis device, 110 Skeleton extraction unit, 120 Trajectory extraction unit, 130 Matching unit, 140 Similarity comparison unit, 190 Reference trajectory information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
PCT/JP2023/013122 2023-03-30 2023-03-30 動作分析装置、動作分析方法、及び動作分析プログラム Ceased WO2024201878A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023541030A JP7350222B1 (ja) 2023-03-30 2023-03-30 動作分析装置、動作分析方法、及び動作分析プログラム
PCT/JP2023/013122 WO2024201878A1 (ja) 2023-03-30 2023-03-30 動作分析装置、動作分析方法、及び動作分析プログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/013122 WO2024201878A1 (ja) 2023-03-30 2023-03-30 動作分析装置、動作分析方法、及び動作分析プログラム

Publications (1)

Publication Number Publication Date
WO2024201878A1 true WO2024201878A1 (ja) 2024-10-03

Family

ID=88099194

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/013122 Ceased WO2024201878A1 (ja) 2023-03-30 2023-03-30 動作分析装置、動作分析方法、及び動作分析プログラム

Country Status (2)

Country Link
JP (1) JP7350222B1 (https=)
WO (1) WO2024201878A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026083587A1 (ja) * 2024-10-18 2026-04-23 富士通株式会社 周期作業認識プログラム、方法、及び装置

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008108008A (ja) * 2006-10-24 2008-05-08 Omron Corp 移動パターン特定装置、移動パターン特定方法、移動パターン特定プログラム、およびこれを記録した記録媒体

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009015529A (ja) * 2007-07-03 2009-01-22 Toshiba Corp 作業分析装置および方法
US8913791B2 (en) * 2013-03-28 2014-12-16 International Business Machines Corporation Automatically determining field of view overlap among multiple cameras
CN112037245B (zh) * 2020-07-22 2023-09-01 杭州海康威视数字技术股份有限公司 一种确定追踪目标相似度的方法和系统
CN114898307B (zh) * 2022-07-11 2022-10-28 浙江大华技术股份有限公司 一种对象跟踪方法、装置、电子设备和存储介质

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008108008A (ja) * 2006-10-24 2008-05-08 Omron Corp 移動パターン特定装置、移動パターン特定方法、移動パターン特定プログラム、およびこれを記録した記録媒体

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
TERADA KENJI , HIROYUKI MIYAHARA, YASUSHI KUBO: "An Experiment of Quantitative Evaluation of Awa Odori Dance", THE JOURNAL OF THE INSTITUTE OF IMAGE ELECTRONICS ENGINEERS OF JAPAN, vol. 34, no. 3, 1 January 2005 (2005-01-01), JP , pages 220 - 227, XP093218589, ISSN: 0285-9831, DOI: 10.11371/iieej.34.220 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026083587A1 (ja) * 2024-10-18 2026-04-23 富士通株式会社 周期作業認識プログラム、方法、及び装置

Also Published As

Publication number Publication date
JPWO2024201878A1 (https=) 2024-10-03
JP7350222B1 (ja) 2023-09-25

Similar Documents

Publication Publication Date Title
US10366307B2 (en) Coarse-to-fine search method, image processing device and recording medium
JP6672712B2 (ja) 異常作業検出システムおよび異常作業検出方法
CN109543705B (zh) 模板创建装置及方法、物体识别处理装置以及记录介质
CN112446363A (zh) 一种基于视频抽帧的图像拼接与去重方法及装置
KR101969623B1 (ko) 병렬 검출과 추적, 및/또는 그룹화된 피처 모션 시프트 추적을 이용한 얼굴 인식
JP2020077231A (ja) 位置検出プログラム、位置検出方法及び位置検出装置
US10614336B2 (en) Method, system, and computer-readable recording medium for image-based object tracking
CN110991292A (zh) 动作识别比对方法、系统、计算机存储介质和电子装置
JP6786015B1 (ja) 動作分析システムおよび動作分析プログラム
US20160110840A1 (en) Image processing method, image processing device, and robot system
JP7350222B1 (ja) 動作分析装置、動作分析方法、及び動作分析プログラム
JP6777150B2 (ja) 劣化検出装置、劣化検出方法、及びプログラム
US10467745B2 (en) Abnormality detection device, abnormality detection method and non-transitory computer-readable recording medium
US9569661B2 (en) Apparatus and method for neck and shoulder landmark detection
JP2010146522A (ja) 顔画像追跡装置及び顔画像追跡方法並びにプログラム
CN118215945A (zh) 作业识别装置、作业识别方法以及作业识别程序
JP6456567B2 (ja) オプティカルフロー精度算出装置およびオプティカルフロー精度算出方法
US10996235B2 (en) System and method for cycle duration measurement in repeated activity sequences
CN114841908A (zh) 异物检测方法，装置，电子设备及存储介质
WO2022162844A1 (ja) 作業推定装置、作業推定方法、及び、作業推定プログラム
JP7831031B2 (ja) 作業認識装置、作業認識方法、及び作業認識プログラム
JP7376446B2 (ja) 作業分析プログラム、および、作業分析装置
JP6218102B2 (ja) 情報処理システム、情報処理方法及びプログラム
JP2005228274A (ja) 画像解析装置、および画像解析方法、並びにコンピュータ・プログラム
CN107589834B (zh) 终端设备操作方法及装置、终端设备

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2023541030

Country of ref document: JP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23930516

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23930516

Country of ref document: EP

Kind code of ref document: A1