JP6928880B2 - Motion analysis device, motion analysis method, motion analysis program and motion analysis system - Google Patents

Description

The present disclosure relates to a motion analyzer, a motion analysis method, a motion analysis program and a motion analysis system.

Conventionally, one or more cameras may be installed on a product manufacturing line to record an operator's movement as an image or a moving image. In addition, image recognition processing may be performed on captured images and moving images to analyze the movements of workers.

The following Patent Document 1 includes a camera that captures the movement of an operator and an operation inspection device that analyzes a moving image taken by the camera and inspects the operation performed by the operator. After generating worker movement data that digitally records the movements performed by the manufacturing worker, pattern matching is used to compare the reference movement pattern that digitally records the reference movement with the worker movement data. , A work motion inspection system that executes a process of identifying all sections of worker motion data that conform to the reference motion data is described.

Further, in Patent Document 2 below, the feature data of the first stage is calculated based on the statistics of the local motion information extracted from the moving image data obtained by capturing the motion of the motion analysis target, and the feature data of the first stage is calculated. The moving image data is divided from the time-series changes, and the second-stage time-series feature data is calculated for each divided section from the divided first-stage feature data, and the second-stage time-series features calculated for each divided section. A motion analysis method is described in which the split section column data is acquired based on at least one of classifying the split sections based on the data, and the motion of the motion analysis target is analyzed based on the acquired split section column data. ..

Japanese Unexamined Patent Publication No. 2016-423332 Japanese Unexamined Patent Publication No. 2009-205282

When various operations are performed on the production line, it may be desired to confirm the scene where the operator performs a specific operation. In order to meet such demands, one or more cameras may be installed on the production line to continuously capture the motion of the operator by moving images. For example, according to the technique described in Patent Document 1, it is possible to specify all the sections of the worker motion data that match the reference motion data, and the sections of the worker motion data in which an undesired motion such as a mistake is performed can be specified. Can be identified. Further, according to the technique described in Patent Document 2, the moving image can be divided and the operation of the target related to the divided section can be analyzed.

However, even if one or more sensors such as cameras are installed on the production line to acquire time-series information indicating the worker's movement, if the types of elemental movements performed by the worker increase, the operation is erroneous. Even if the section in which the above is executed is specified, it becomes difficult to understand what kind of element operation the section is related to, and it becomes difficult to understand what kind of element operation the error has occurred. In addition, since multiple types of erroneous operations may be performed for each element operation, even if the section is simply specified, it is clear until the data content is confirmed what type of erroneous operation was performed. Therefore, it becomes difficult to formulate countermeasures efficiently. This situation applies not only to erroneous actions, but also to identify the section where standard actions are performed. That is, as the types of element operations performed by the operator increase, it becomes difficult to understand what kind of element operation the standard operation was performed on, even if the section in which the standard operation was executed is specified. Moreover, even if the section is simply specified, it is not clear what kind of standard operation was performed until the contents of the data are confirmed.

Therefore, the present invention provides a motion analysis device, a motion analysis method, a motion analysis program, and a motion analysis system that can easily confirm what kind of motion is executed for what element motion.

The analyzer according to one aspect of the present disclosure includes an acquisition unit that acquires time-series information indicating an operator's operation, and a plurality of partial information corresponding to a plurality of elemental operations executed by the operator. It includes a first classification unit for classifying, and a second classification unit for classifying at least one partial information into any of a plurality of groups according to one or a plurality of criteria regarding the movement of the worker. Here, the time-series information indicating the operation of the worker may be information acquired continuously or intermittently, and may include numerical information and image information. In addition, the first classification unit may divide the time series information into a plurality of partial information, or attach tags and flags indicating the plurality of partial information to the target time series information, and based on the tags and flags. The time series information may be classified into a plurality of partial information by classifying the time series information. In addition, the second classification unit may divide the partial information into a plurality of groups, or assign the partial information to any one of the plurality of groups by adding tags and flags indicating the plurality of groups to the partial information. It may be classified into. The second classification unit may classify the partial information into one of a plurality of groups so as not to be duplicated, or may classify the partial information into a plurality of groups by allowing duplication.

According to this aspect, the first classification unit classifies the time series information into a plurality of partial information, each corresponding to any of the plurality of element actions, and the second classification unit classifies at least one partial information. By classifying into one of a plurality of groups according to one or a plurality of criteria regarding the movement of the worker, it is possible to easily confirm what kind of movement is executed for what element movement. Further, by classifying from different viewpoints by the first classification unit and the second classification unit, it is possible to perform classification with less noise.

In the above embodiment, a selection unit for selecting one of a plurality of groups is further provided, and the one or more criteria include a first criterion and a second criterion different from the first criterion, and the second classification portion is: Even if at least one partial information is classified into one of a plurality of groups according to the first criterion, and the partial information contained in the group selected by the selection unit is classified into one of a plurality of groups according to the second criterion. good.

According to this aspect, by classifying from different viewpoints of the first criterion and the second criterion by the second classification unit, it is possible to confirm in more detail what kind of operation was executed for what element operation. can do.

In the above aspect, the selection unit may select a group corresponding to the input by the input unit or a group satisfying a predetermined condition among the plurality of groups. Here, the predetermined condition may be a condition relating to the threshold value set for the first reference.

According to this aspect, at least one partial information is classified into one of a plurality of groups according to the first criterion, a group according to input or a group satisfying a predetermined condition is selected, and the group is included in the selected group. By classifying the partial information into one of a plurality of groups according to the second criterion, it is possible to perform a more detailed classification of the group of particular interest among the plurality of groups classified according to the first criterion.

In the above aspect, the time-series information includes the operation information including the coordinate value indicating the operation of the worker, and the first classification unit is the coordinate value indicating the operation information and the reference operation as a reference for comparison of the operation of the worker. The operation information may be classified into a plurality of partial information based on the comparison with the reference operation information including. Here, the coordinate value indicating the movement of the worker may be the coordinate value of the portion whose position changes with the movement of the worker, and the portion may be the portion that characterizes the movement of the worker. More specifically, the coordinate values indicating the movement of the worker are the coordinate values of a plurality of joints of the worker, the coordinate values of the fingertips and the head of the worker, and the tool grasped by the worker. Or it may be the coordinate value of an object worn by the operator.

According to this aspect, by comparing the motion information including the coordinate values indicating the operator's motion with the reference motion information indicating the coordinate values indicating the reference motion, the plurality of partial information can be more accurately combined with the plurality of elements. Time-series information can be categorized to correspond to the operation.

In the above aspect, the time-series information is associated with the motion information including the coordinate values indicating the operator's motion, includes the moving image of the worker, and is associated with the motion information included in the group selected by the selection unit. An extraction unit that extracts at least one moving image from the moving image may be further provided.

According to this aspect, it is possible to extract the moving image associated with the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and what kind of operation is actually performed. Can be confirmed.

In the above aspect, the extraction unit may extract the moving image based on the center of gravity of the motion information included in the selected group in the reference space used for the classification by the second classification unit.

According to this aspect, with respect to the motion information included in the group of particular interest among the plurality of groups classified according to the first criterion, a moving image can be extracted based on the representative motion information, which is typical of the group of interest. It is possible to confirm what kind of operation has been performed.

In the above aspect, the extraction unit may extract a moving image based on the input by the input unit.

According to this aspect, it is possible to extract a moving image based on the input by the user for the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and what kind of operation is actually performed. It is possible to individually confirm each operation information included in the group of interest as to whether or not it has been performed.

In the above aspect, the determination unit that determines the reproduction range of the extracted moving image based on the type of the group selected by the selection unit may be further provided.

According to this aspect, by determining the playback range of the moving image based on the type of the group of particular interest among the plurality of groups classified according to the first criterion, the scene in which the action of interest is performed can be easily performed. You can play the video so you can see it.

In the above aspect, the first criterion may include a criterion regarding the execution time of the worker's movement.

According to this aspect, the case where the execution time of the worker's action is too short or too long can be classified into a non-standard group, and what kind of action is executed can be classified from the viewpoint of execution time. You can check with.

In the above aspect, the first criterion may include a criterion regarding the length of the movement locus of a specific part that moves with the movement of the operator. Here, the specific part may be a part whose position changes with the movement of the worker, and the part may be a part that characterizes the movement of the worker. More specifically, the specific part may be a plurality of joints of the worker, a fingertip or a head of the worker, a tool grasped by the worker, or an object attached to the worker. You can do it.

According to this aspect, the case where the movement locus of the worker's movement is too short or too long can be classified into a non-standard group, and what kind of movement is executed is viewed from the viewpoint of the movement locus. You can check with.

In the motion analysis method according to another aspect of the present disclosure, time-series information indicating the motion of the worker is acquired, and the time-series information is used as a plurality of partial information corresponding to a plurality of elemental motions executed by the worker. It includes classifying at least one partial information into one of a plurality of groups according to one or a plurality of criteria regarding the movement of the worker.

According to this aspect, the time series information is classified into a plurality of partial information, each of which corresponds to one of a plurality of element actions, and at least one partial information is pluralized according to one or a plurality of criteria relating to the worker's action. By classifying into one of the groups of, it is possible to easily confirm what kind of action was executed for what kind of element action. Further, by performing two types of classification from different viewpoints, it is possible to perform classification with less noise.

In the motion analysis program according to another aspect of the present disclosure, the arithmetic unit provided in the motion analysis device is executed by the operator, the acquisition unit for acquiring the time series information indicating the operation of the operator, and the time series information. The first classification unit that classifies into a plurality of partial information corresponding to a plurality of element movements, and the second classification that classifies at least one partial information into one of a plurality of groups according to one or a plurality of criteria related to the worker's movement. Operate as a part.

According to this aspect, the first classification unit classifies the time series information into a plurality of partial information, each corresponding to any of the plurality of element actions, and the second classification unit classifies at least one partial information. By classifying into one of a plurality of groups according to one or a plurality of criteria regarding the movement of the worker, it is possible to easily confirm what kind of movement is executed for what element movement. Further, by classifying from different viewpoints by the first classification unit and the second classification unit, it is possible to perform classification with less noise.

The motion analysis system according to another aspect of the present disclosure includes a measuring unit that measures motion information including coordinate values indicating the operator's motion, and a photographing unit that captures a moving image of the worker associated with the motion information. , The first classification unit that classifies the motion information into a plurality of partial information based on the comparison between the motion information and the reference motion information including the coordinate values indicating the reference motion that serves as a reference for the comparison of the worker's motion, and at least. A second classification unit that classifies one partial information into one of a plurality of groups according to one or a plurality of criteria related to the movement of the worker, a selection unit that selects one of the plurality of groups, and a selection unit that selects one of the plurality of groups. It includes an extraction unit that extracts at least one moving image from the moving image associated with the motion information included in the group, and a display unit that displays the extracted moving image.

According to this aspect, the first classification unit classifies the time series information into a plurality of partial information, each of which corresponds to any of a plurality of element actions, and at least one partial information is related to the worker's action. Alternatively, by classifying into one of a plurality of groups according to a plurality of criteria, it is possible to easily confirm what kind of operation was executed for what element operation. In addition, it is possible to extract the moving image associated with the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and confirm what kind of operation was actually performed. be able to.

According to the present invention, there is provided a motion analysis device, a motion analysis method, a motion analysis program, and a motion analysis system that can easily confirm what kind of motion is executed for what element motion.

It is a figure which illustrates the outline of the motion analysis system which concerns on embodiment of this invention. It is a figure which illustrates the functional block of the motion analysis system which concerns on this embodiment. It is a figure which illustrates the physical structure of the motion analyzer which concerns on this embodiment. It is a figure which shows the 1st example of the motion information measured by the motion analysis system which concerns on this embodiment. It is a figure which shows the 2nd example of the motion information measured by the motion analysis system which concerns on this embodiment. It is a figure which shows the 3rd example of the motion information measured by the motion analysis system which concerns on this embodiment. It is a figure which shows the 4th example of the motion information measured by the motion analysis system which concerns on this embodiment. It is a figure which shows the 5th example of the motion information measured by the motion analysis system which concerns on this embodiment. It is a flowchart of 1st example of the classification process executed by the motion analysis system which concerns on this embodiment. It is a figure which shows the 1st example of the group classified by the motion analysis system which concerns on this embodiment. It is a flowchart of the 2nd example of the classification process executed by the motion analysis system which concerns on this embodiment. It is a figure which shows the 2nd example of the group classified by the motion analysis system which concerns on this embodiment. It is a figure which shows the 1st example of the reproduction range of the moving image determined by the motion analysis system which concerns on this embodiment. It is a figure which shows the 2nd example of the reproduction range of the moving image determined by the motion analysis system which concerns on this embodiment. It is a figure which shows the 3rd example of the reproduction range of the moving image determined by the motion analysis system which concerns on this embodiment. This is an example of a screen displayed by the motion analysis system according to the present embodiment. It is a figure which shows the 3rd example of the group classified by the motion analysis system which concerns on this embodiment. It is a figure which shows the 4th example of the group classified by the motion analysis system which concerns on this embodiment.

Hereinafter, embodiments according to one aspect of the present invention (hereinafter, referred to as “the present embodiment”) will be described with reference to the drawings. In each figure, those having the same reference numerals have the same or similar configurations.

§1 Application example First, an example of a situation in which the present invention is applied will be described with reference to FIG. The motion analysis system 100 according to the present embodiment has a measuring unit 30 that measures motion information including coordinate values indicating the operator's motion, and a first photographing unit that captures a moving image of the worker associated with the motion information. 20a, a second photographing unit 20b, and a third photographing unit 20c are provided. The motion analysis system 100 according to the present embodiment analyzes the motion performed by the operator in the work area R including the entire production line, and the work area R may be an arbitrary area, for example, a predetermined process. It may be an area where a predetermined element operation is performed, or it may be an area where a predetermined element operation is performed. Here, the element operation is a unit operation performed by an operator, and includes, for example, operations such as picking parts, arranging parts, fixing parts, packing products, tightening screws, and inspecting products.

In this example, a case where the first worker A1 and the second worker A2 perform a predetermined operation in the work area R will be described. The first worker A1 executes an operation such as grasping a screw with a finger, inserting a screw into a screw hole of a part, and tightening the screw with a screwdriver, and the second worker A2 inspects a screwed part, for example. You may perform the operation such as. However, the operator may perform an arbitrary operation in an arbitrary place not limited to the work area R.

The motion analysis system 100 includes a motion analysis device 10. The motion analysis device 10 has an acquisition unit that acquires time-series information indicating the operator's motion, and a first classification that classifies the time-series information into a plurality of partial information corresponding to a plurality of elemental motions executed by the operator. It includes a unit and a second classification unit that classifies at least one partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker. Here, the time-series information indicating the operation of the worker may include the operation information including the coordinate values indicating the operation of the worker and the moving image of the worker associated with the operation information. The time-series information is classified into a plurality of partial information corresponding to a plurality of element actions by the first classification unit. Then, the partial information is classified into any of a plurality of groups by the second classification unit. For example, the first classification unit divides the time series information into a plurality of partial information, each of which corresponds to one of a plurality of element actions, and the second classification unit divides at least one partial information into one or a plurality of partial information. It may be clustered into a plurality of groups according to the criteria.

More specifically, the first classification unit is based on the comparison between the motion information included in the time series information and the reference motion information including the coordinate values indicating the reference motion which is the reference motion for the comparison of the worker's motion. The operation information may be classified into a plurality of partial information. Here, the reference operation may be a standard operation that the operator should follow, or may be a mistaken operation or a non-standard operation of the operator. For example, when the worker's operation is screw tightening and inspection of a part, the first classification unit provides operation information with partial information corresponding to the element operation related to screw tightening of the part and a part corresponding to the element operation related to inspection. It may be classified as information. In addition, the first classification unit calculates the degree of deviation between the specific section of the motion information including the coordinate values indicating the worker's motion in chronological order and the reference motion information including the coordinate values indicating the reference motion in chronological order. , The reference operation information having a small degree of deviation may be specified, and the relevant section of the operation information may be classified as partial information corresponding to the element operation represented by the specified reference operation information.

In addition, the second classification unit may classify at least one partial information into any of a plurality of groups according to the first criterion. Here, the first standard is a standard regarding the execution time of the worker's movement, a standard regarding the length of the movement locus of a specific part that moves with the worker's movement, and the content of the worker's movement. It may be a standard. More specifically, when the partial information corresponds to an element operation related to screw tightening of a part, the first reference is, for example, a reference related to the execution time of the screw tightening operation, or is grasped by an operator's hand or an operator. It may be a reference regarding the length of the movement locus of a specific part such as a screw gripped by a screwdriver or an operator. This makes it possible to classify the cases where the execution time of the worker's action is too short or too long into a non-standard group, and confirm what kind of action was executed from the viewpoint of execution time. be able to. In addition, when the movement locus of the worker's movement is too short or too long, it can be classified into a non-standard group, and what kind of movement was executed can be confirmed from the viewpoint of the movement locus. Can be done.

Further, the second classification unit classifies the partial information included in the selected group from the plurality of groups classified by the first criterion into one of the plurality of groups according to the second criterion different from the first criterion. You can. Here, the second standard is a standard that is not used as the first standard, and is a standard related to the execution time of the worker's movement or the length of the movement locus of a specific part that moves with the worker's movement. It may be a standard or a standard regarding the content of the worker's movement. More specifically, if the partial information contained in the selected group corresponds to an elemental action relating to screw tightening of a part and is a group with a relatively long execution time, the second criterion is, for example, the operator screwing. It may be a criterion for classifying whether it took a longer time than the standard to insert the screw into the screw hole or whether it took a longer time than the standard because the operator dropped the screw and picked it up. In this way, by classifying by the second classification unit from different viewpoints of the first criterion and the second criterion, it is possible to confirm in more detail what kind of operation was executed for what element operation. Can be done.

The motion analysis device 10 may extract at least one moving image from the moving image associated with the motion information included in the selected group and display it on the display unit 10f. Here, the playback range of the extracted moving image may be determined based on the type of the selected group. For example, when the partial information included in the selected group corresponds to an unscheduled operation, the playback range of the moving image may be determined so as to include the unscheduled operation and the operation before and after the unscheduled operation. In this way, it is possible to extract the moving image associated with the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and what kind of operation was actually performed can be extracted. You can check. In addition, by determining the playback range of the moving image based on the type of the group of particular interest among the plurality of groups classified according to the first criterion, it is possible to easily confirm the scene in which the action of interest is being performed. , You can play the video.

According to the motion analysis device 10 according to the present embodiment, the time series information is classified into a plurality of partial information, each of which corresponds to any of a plurality of element motions, by the first classification unit, and by the second classification unit. By classifying at least one partial information into one of a plurality of groups according to one or more criteria regarding the worker's movement, it is easy to confirm what kind of movement was executed for what element movement. can do. Further, by classifying from different viewpoints by the first classification unit and the second classification unit, it is possible to perform classification with less noise. As a result, even if the types of element actions performed by the operator increase, it becomes clear what kind of element action-related sections have been specified, and for each element action, standard actions or multiple types of actions can be specified. The type of operation such as erroneous operation becomes clear. Therefore, the calculation load required for the analysis of the operator's operation can be significantly reduced, and measures for preventing erroneous operation can be devised and efficient operation procedures can be examined more efficiently.

For example, the first classification unit compares the operation information included in the time series information with the reference operation information and classifies it into a plurality of partial information, and the second classification unit classifies the partial information into a standard operation group and a standard operation. By classifying it into a group of non-operations, it is possible to check the percentage of workers performing standard operations. As a result, in the short term, it is possible to detect at an early stage that a non-standard operation has been performed, and reduce the possibility that a defective product is manufactured on a production line, for example. Even if a non-standard operation is performed, even if a non-defective product is manufactured, it has technical significance in that it can catch signs of a problem and make subsequent confirmation work more efficient. Such a merit is a merit obtained in real time when the motion analysis system 100 is used.

Further, in the long term, it can be used to support the proficiency of workers and can provide materials for examining improvement of standard operation. For example, it is conceivable that many workers identify element actions that perform non-standard actions and improve the reference actions related to the element actions. Such a merit is a merit obtained when the motion analysis system 100 is continuously used, and is a merit obtained in non-real time.

§2 Configuration example [Functional configuration]
Next, an example of the functional configuration of the motion analysis system 100 according to the present embodiment will be described with reference to FIG. The motion analysis system 100 includes a first imaging unit 20a, a second imaging unit 20b, a third imaging unit 20c, a measuring unit 30, and a motion analysis device 10. Then, the motion analysis device 10 includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, a storage unit 14, a first classification unit 15, a second classification unit 16, a selection unit 17, and an extraction unit 18. It includes a determination unit 19, an input unit 10e, and a display unit 10f.

<Shooting section>
The first photographing unit 20a, the second photographing unit 20b, and the third photographing unit 20c may be respectively composed of a general-purpose camera, and the first worker A1 and the second worker A2 execute an operation in the work area R. You may shoot a video that includes the scene you are in. The first photographing unit 20a, the second photographing unit 20b, and the third photographing unit 20c may each photograph a part of the work area R, and may photograph a moving image in an area narrower than the work area R. Specifically, it may be a moving image of the operation executed by the first worker A1 and the second worker A2, focusing on a specific area. The first photographing unit 20a, the second photographing unit 20b, and the third photographing unit 20c may, for example, photograph a moving image in which the hands of the first worker A1 and the second worker A2 are close-up.

In addition, the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may capture a plurality of moving images in which a plurality of portions constituting the work area R are captured. For example, the first shooting unit 20a mainly shoots a moving image in which the first worker A1 is executing the operation, and the third shooting unit 20c mainly shoots the moving image in which the second worker A2 is executing the operation. The second photographing unit 20b may photograph both the moving image in which the first worker A1 is performing the operation and the moving image in which the second worker A2 is performing the operation. Further, the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may capture moving images in which different steps are executed at a plurality of positions in the work area R.

<Measurement unit>
The measuring unit 30 may be configured by motion capture, and may measure motion information including coordinate values indicating the actions of the first worker A1 and the second worker A2 executed in a certain work area R. The configuration of the measuring unit 30 is arbitrary, but for example, the pattern light is projected onto the first worker A1 and the second worker A2, and the pattern light is projected onto the first worker A1 and the second worker A1 and the second worker. A moving image of A2 may be photographed, and the coordinate values of a plurality of joints of the first worker A1 and the second worker A2 may be measured based on the captured moving image. Further, when the measuring unit 30 measures a plurality of operation information relating to a plurality of workers, the measuring unit 30 may add information for identifying the worker to the operation information. The measuring unit 30 may measure other than the coordinate values of the plurality of joints of the first worker A1 and the second worker A2. For example, the measuring unit 30 of the first worker A1 and the second worker A2. It may measure the coordinate values of a specific part that is not necessarily a joint, such as a fingertip, a head, a tool gripped by an operator, or an object worn by an operator, and may be a measurement of the coordinate values of a specific part other than the joint and the joint. It may measure the coordinate value of a typical position. Further, the measuring unit 30 may measure the coordinate values of the positions of the trackers worn by the first worker A1 and the second worker A2. Further, the measuring unit 30 may analyze the moving images of the first worker A1 and the second worker A2 and measure the operation information including the coordinate values indicating the operation of the worker based on the moving images. ..

The motion analysis system 100 may include a plurality of measuring units 30. When the operation information of a plurality of workers is measured by a plurality of measuring units 30, the operation information of the same worker may be measured in duplicate, but information for identifying the worker is added to the operation information. Then, the duplication may be removed, or the operation information measured by different measuring units 30 may be combined.

The measuring unit 30 may also serve as a fourth photographing unit that captures a moving image in which the first worker A1 and the second worker A2 are executing the operation. The fourth shooting unit may shoot a moving image of the entire work area R. That is, the fourth photographing unit may photograph the state in which the first worker A1 and the second worker A2 are executing the operation so that both the first worker A1 and the second worker A2 are included. .. On the other hand, the first photographing unit 20a, the second photographing unit 20b, and the third photographing unit 20c include the first worker A1 and the second worker A2 so that one of the first worker A1 and the second worker A2 is included. You may shoot a moving image in which A2 is performing the operation.

<Motion analyzer>
The first acquisition unit 11 may acquire operation information including coordinate values indicating the operations of the first worker A1 and the second worker A2 from the measurement unit 30. The operation information acquired by the first acquisition unit 11 is transmitted to the storage unit 14 and stored as the operation information history 14b. When the motion analysis system 100 includes a plurality of measurement units 30, the first acquisition unit 11 may acquire motion information from each of the plurality of measurement units 30, and determines from which measurement unit 30 the motion information is acquired. The operation information may be transmitted to the storage unit 14 by adding the identification information. The motion information may be, for example, information obtained by measuring coordinate values indicating the operator's motion at 1-second intervals.

The second acquisition unit 12 captures a moving image in which the first worker A1 and the second worker A2 are performing an operation in the first shooting unit 20a, the second shooting unit 20b, the third shooting unit 20c, and the measuring unit 30 ( It may be obtained from the 4th photographing section). The moving image acquired by the second acquisition unit 12 is transmitted to the storage unit 14, associated with the operation information, and stored as the moving image history 14a. The second acquisition unit 12 may transmit the moving image to the storage unit 14 by adding information for identifying which of the plurality of photographing units the moving image is acquired from.

The third acquisition unit 13 may acquire reference operation information including coordinate values indicating a reference operation as a reference for comparison with respect to the operation of the operator from the measurement unit 30. The reference operation information acquired by the third acquisition unit 13 is transmitted to the storage unit 14 and stored as the reference operation information 14c. The third acquisition unit 13 acquires the operation information specified by the user from the already stored operation information history 14b as the reference operation information, and adds and stores information indicating that the operation is the reference operation. It may be stored in the part 14. The reference motion information may be, for example, information obtained by measuring coordinate values indicating the operator's motion when a standard motion to be followed by the operator is executed at 1-second intervals. Further, the third acquisition unit 13 may acquire reference operation information from another device via the Internet or the like connected by the communication unit 10d described later.

The storage unit 14 may store the moving image history 14a, the operation information history 14b, and the reference operation information 14c. In addition to these, the storage unit 14 may store the threshold value used by the second classification unit. Further, the moving image history 14a and the operation information history 14b may be stored in association with each other based on the measured time.

The first classification unit 15 may classify the time series information into a plurality of partial information corresponding to a plurality of element actions executed by the operator. Here, the time-series information may include motion information including coordinate values indicating the worker's motion, and a moving image of the worker photographed in association with the motion information. The first classification unit 15 may classify the operation information into a plurality of partial information based on the comparison between the operation information and the reference operation information. The reference operation information may be prepared for each of a plurality of element operations, and the first classification unit 15 determines which element operation the operation information approximates to the reference operation information of which element operation, and obtains the operation information from the plurality of elements. It may be classified into a plurality of partial information corresponding to the operation.

The first classification unit 15 calculates, for example, the difference between the coordinate value at each time included in the operation information and the start coordinate value included in the reference operation information, and the difference between the coordinate values is equal to or less than the threshold value. The time may be specified as the start time. Then, the difference between the coordinate value of each time included in the operation information and the end coordinate value included in the reference operation information is calculated, and the time of the operation information in which the difference between the coordinate values is equal to or less than the threshold value is specified as the end time. It's okay. After that, the first classification unit 15 cuts out the coordinate values from the specified start time to the end time in the operation information. At this time, the number of lines to be cut out does not necessarily match the number of lines of the reference operation information. Therefore, the first classification unit 15 may interpolate or thin out the number of lines of the operation information cut out from the operation information so as to match the number of lines of the reference operation information. The method of interpolation or thinning is arbitrary. For example, when interpolation is performed, the average value of the previous and next rows may be used for interpolation, and when thinning is performed, the line whose coordinate value changes is relatively small. May be thinned out. Next, the first classification unit 15 may calculate the degree of deviation between the coordinate values of the interpolated or thinned operation information and the coordinate values of the reference operation information. The degree of deviation referred to here is a numerical value indicating how much the worker's movement deviates from the reference movement that is the reference for comparison. The degree of divergence may be calculated for each joint, or may be calculated in units of a combination of a plurality of joints (right hand, etc.). For example, the degree of divergence for each joint is the difference between the coordinate values of motion information (x, y, z) and the coordinate values of reference motion information (X, Y, Z) for the three-dimensional coordinate values of joints at a certain time. Is squared to take the square root, the value ((x-X) ² + (y-Y) ² + (z-Z) ² ) ^1/2 is calculated, and this value is summed over the entire time. You can do it. However, the degree of divergence may be calculated by other values. The first classification unit 15 may compare the operation information with the plurality of reference operation information, calculate the degree of divergence, and classify the operation information into a plurality of partial information based on the degree of divergence. Here, the first classification unit 15 specifies the reference operation information having the smallest degree of divergence from the extracted part of the operation information, and the extracted part of the operation information is the specified reference operation information. It may be classified as partial information corresponding to the element operation to be represented. Further, the first classification unit 15 may classify the part of the operation information whose degree of deviation is equal to or more than the threshold value for any of the reference operation information as partial information corresponding to any element operation. The first classification unit 15 may divide the time-series information into a plurality of partial information, or attach tags and flags indicating the plurality of partial information to the time-series information to provide the time-series information. It may be classified into a plurality of partial information.

In this way, by comparing the motion information including the coordinate values indicating the operator's motion with the reference motion information indicating the coordinate value indicating the reference motion, the plurality of partial information can be more accurately converted into a plurality of element motions. Time series information can be categorized accordingly.

The second classification unit 16 may classify at least one partial information into any of a plurality of groups according to one or a plurality of criteria regarding the movement of the worker. Here, the one or more criteria may include a first criterion and a second criterion different from the first criterion. In this case, the second classification unit 16 classifies at least one partial information into one of a plurality of groups according to the first criterion, and the partial information included in the group selected by the selection unit 17 is classified according to the second criterion. It may be classified into any of a plurality of groups. The second classification unit 16 may divide the partial information into a plurality of groups, or by adding tags and flags indicating the plurality of groups to the partial information, the partial information may be assigned to any of the plurality of groups. It may be classified as a crab. Further, the second classification unit 16 may classify the partial information into one of a plurality of groups so as not to be duplicated, or may classify the partial information into a plurality of groups by allowing duplication.

The second classification unit 16 classifies at least one of the plurality of partial information classified by the first classification unit 15 into one of a plurality of groups according to the first criterion regarding the operation of the worker, and a plurality of groups. Partial information included in the selected group among the groups may be classified into one of a plurality of groups according to a second criterion different from the first criterion regarding the operation of the worker. In this way, by classifying by the second classification unit 16 from different viewpoints of the first criterion and the second criterion, it is confirmed in more detail what kind of operation was executed for what element operation. be able to.

The first criterion may include a criterion regarding the execution time of the worker's movement. In this case, the second classification unit 16 creates a histogram regarding the execution time of the worker's operation for at least one partial information among the plurality of partial information, and for example, a group of the top 10% and a group of the bottom 10%. And other 80% groups. Here, the top 10% of the groups are groups in which the work time is considered to be too long, and there is a possibility that unnecessary work is performed or the work is redoed by mistake. In addition, the bottom 10% of the groups are groups in which the work time is considered to be too short, and there is a possibility that the work may be skipped or different work may be performed. In addition, the other 80% of the groups are considered to perform standard operations at least in terms of working hours. In this way, by classifying the partial information according to the criteria related to the execution time of the operation, it is possible to classify the case where the execution time of the operation of the worker is too short or too long into a non-standard group. It is possible to confirm whether various types of operations have been executed from the viewpoint of execution time.

In addition, the first criterion may include a criterion regarding the length of the movement locus of a specific part that moves with the movement of the operator. In this case, the second classification unit 16 creates a histogram regarding the length of the movement locus of a specific part for at least one partial information among the plurality of partial information, and for example, the top 10% group and the bottom 10%. Group and other 80% group may be classified. Here, the top 10% group is a group in which the movement locus of a specific part is considered to be too long, and there is a possibility that unnecessary work is performed or the work is redoed by mistake. In addition, the lower 10% group is a group in which the movement locus of a specific part is considered to be too short, and there is a possibility that the work may be skipped or a different work may be performed. In addition, the other 80% of the groups are considered to perform standard movements at least from the viewpoint of the movement locus of a specific part. In this way, by classifying the partial information according to the criteria related to the movement locus of a specific part, it is possible to classify the case where the movement locus of the worker's movement is too short or too long into a non-standard group. It is possible to confirm what kind of operation is executed from the viewpoint of the movement locus.

The selection unit 17 may select any of a plurality of groups classified by the second classification unit 16. The selection unit 17 may select a group corresponding to the input by the input unit 10e or a group satisfying a predetermined condition from the plurality of groups. Here, the input by the input unit 10e may be the input received from the user of the motion analysis device 10. Further, the predetermined condition may be a condition relating to comparison with the threshold value set for the first criterion. For example, when the first criterion is a criterion related to the execution time of an operation of an operator, it is a condition whether or not the execution time is shorter than the threshold for setting the lower limit of the execution time of the operation, or the upper limit of the execution time of the operation is set. It may be a condition of whether or not the execution time is longer than the threshold value. Further, for example, when the first criterion is a criterion relating to the length of the movement locus of a specific part that moves with the movement of the operator, the condition is whether or not the movement locus is shorter than the threshold value that defines the lower limit of the length of the movement locus. It may be a condition that the movement locus is longer than the threshold value that sets the upper limit of the length of the movement locus. In this way, at least one partial information is classified into one of a plurality of groups according to the first criterion, a group corresponding to the input or a group satisfying a predetermined condition is selected, and the partial information included in the selected group is selected. Is classified into any of a plurality of groups according to the second criterion, and more detailed classification can be performed for the group of particular interest among the plurality of groups classified according to the first criterion.

The extraction unit 18 may extract at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit 17. Here, the moving image may be extracted from the moving image history 14a of the storage unit 14. As a result, it is possible to extract the moving image associated with the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and confirm what kind of operation was actually performed. can do.

The extraction unit 18 may extract a moving image based on the center of gravity of the motion information included in the selected group in the reference space used for the classification by the second classification unit 16. For example, when the second classification unit 16 classifies the partial information into one of a plurality of groups according to the reference regarding the execution time of the movement and the reference regarding the length of the movement locus of a specific part, the execution time of the movement and the movement locus It is possible to plot the motion information included in a plurality of groups on the two-dimensional space regarding the length. In this case, the extraction unit 18 may calculate the center of gravity of the motion information included in the selected group in the two-dimensional space, and extract, for example, the moving image associated with the motion information closest to the center of gravity. As a result, with respect to the motion information included in the group of particular interest among the plurality of groups classified according to the first criterion, it is possible to extract a moving image based on the representative motion information, which is typically in the group of interest. It is possible to confirm whether such an operation has been performed.

The extraction unit 18 may extract a moving image based on the input by the input unit 10e. For example, with respect to the operation information included in the group selected by the selection unit 17, the input by the input unit 10e may be further received, and the moving image associated with the selected operation information may be extracted. As a result, it is possible to extract a moving image based on the input by the user for the operation information included in the group of particular interest among the plurality of groups classified according to the first criterion, and what kind of operation was actually performed. It is possible to individually check each operation information included in the group of interest.

The determination unit 19 may determine the reproduction range of the extracted moving image based on the type of the group selected by the selection unit 17. Here, the type of the group may be the type of the operation of the worker represented by the partial information included in the group. For example, if the partial information included in the group represents a standard operation for a certain element operation, the type of the group may be "standard". In this case, the determination unit 19 may determine the reproduction range of the moving image so as to include the scene in which the worker is executing the element operation. Further, for example, if the partial information included in the group represents a non-standard operation for a certain element operation, the type of the group may be "non-standard". In this case, the determination unit 19 may determine the playback range of the moving image so as to include a scene in which the operator is performing a non-standard operation. The motion analysis device 10 may store in the storage unit 14 a correspondence table in which the correspondence between the group type and the reproduction range of the moving image is determined, and the determination unit 19 refers to the correspondence table and selects the selection unit 17. The playback range of the moving image corresponding to the group type selected by may be determined. The determination of the reproduction range of the moving image by the determination unit 19 will be described in more detail with reference to FIGS. 4 to 8. In this way, by determining the playback range of the moving image based on the type of the group of particular interest among the plurality of groups classified according to the first criterion, it is possible to easily confirm the scene in which the action of interest is being performed. You can play the video as you like.

[Hardware configuration]
Next, an example of the hardware configuration of the motion analysis device 10 according to the present embodiment will be described with reference to FIG. The motion analysis device 10 includes a CPU (Central Processing Unit) 10a corresponding to a computing device, a RAM (Random Access Memory) 10b corresponding to a storage unit 14, and a ROM (Read only Memory) 10c corresponding to the storage unit 14. It has a communication unit 10d, an input unit 10e, and a display unit 10f. Each of these configurations is connected to each other via a bus so that data can be transmitted and received. In this example, the case where the motion analysis device 10 is composed of one computer will be described, but the motion analysis device 10 may be realized by using a plurality of computers.

The CPU 10a is a control unit that controls execution of a program stored in the RAM 10b or ROM 10c, calculates data, and processes data. The CPU 10a classifies the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker, and sets at least one partial information into a plurality of groups according to one or a plurality of criteria related to the worker's actions. It is an arithmetic unit that executes a program (motion analysis program) classified into any of the above. The CPU 10a receives various input data from the input unit 10e and the communication unit 10d, displays the calculation result of the input data on the display unit 10f, and stores it in the RAM 10b or ROM 10c.

The RAM 10b is a storage unit 14 in which data can be rewritten, and may be composed of, for example, a semiconductor storage element. The RAM 10b stores data such as an operation analysis program executed by the CPU 10a, a moving image history 14a, an operation information history 14b, and a reference operation information 14c.

The ROM 10c can read data from the storage unit 14, and may be composed of, for example, a semiconductor storage element. The ROM 10c stores, for example, an operation analysis program and data that is not rewritten.

The communication unit 10d is an interface for connecting the motion analysis device 10 to an external device. The communication unit 10d is connected to the first imaging unit 20a, the second imaging unit 20b, the third imaging unit 20c, and the measuring unit 30 by, for example, a LAN (Local Area Network), and the first imaging unit 20a and the second imaging unit 20b. And the moving image may be received from the third photographing unit 20c, and the moving image and the operation information may be received from the measuring unit 30. Further, the communication unit 10d may be connected to the Internet and receive a moving image or operation information via the Internet. Further, the communication unit 10d may transmit the moving image extracted by the extracting unit 18 out of the moving images taken via the Internet to an external device.

The input unit 10e receives data input from the user, and may include, for example, a keyboard, a mouse, and a touch panel.

The display unit 10f visually displays the calculation result by the CPU 10a, and may be configured by, for example, an LCD (Liquid Crystal Display). An example of the screen displayed on the display unit 10f will be described in detail later.

The motion analysis program may be stored in a storage medium readable by a computer such as RAM 10b or ROM 10c and provided, or may be provided via a communication network connected by the communication unit 10d. In the motion analysis device 10, the CPU 10a executes the motion analysis program to realize various operations described with reference to FIG. It should be noted that these physical configurations are examples and do not necessarily have to be independent configurations. For example, the motion analysis device 10 may include an LSI (Large-Scale Integration) in which the CPU 10a and the RAM 10b or ROM 10c are integrated.

§3 Operation example FIG. 4 is a diagram showing a first example D1 of operation information measured by the operation analysis system 100 according to the present embodiment. In the figure, the vertical axis shows the position or speed of a specific part of the worker, and the horizontal axis shows the time. Here, the specific part of the worker may be, for example, a hand. The graph is an example and is a simplification of the actually measured operation information.

The first example D1 of the operation information shows an example of the operation information when the worker performs a standard operation for a certain element operation. In this case, the operator moves the position or speed of the specific part from the initial position (initial speed) of the time "0.0" to the minus side at the time "1.0", and then moves the position or speed to the minus side at the time "2." When it is "0", it is returned to the initial position (initial speed), and the same position (speed) is maintained until the time "5.0". The position or speed of the specific part may be the position or speed in a specific direction, and the same applies to other examples of the operation information shown in FIGS. 5 to 8.

FIG. 5 is a diagram showing a second example D2 of motion information measured by the motion analysis system 100 according to the present embodiment. Also in the figure, the vertical axis shows the position or speed of a specific part of the worker, and the horizontal axis shows the time.

The second example D2 of the operation information shows an example of the operation information when the worker performs a non-standard operation for a certain element operation, and more specifically, an extra operation (waste operation) is performed. An example of operation information in the case is shown. In this case, the operator moves the position or speed of the specific part from the initial position (initial speed) of the time "0.0" to the minus side at the time "1.0", and then moves the position or speed to the minus side at the time "2." When it is "0", it moves to a different position (speed) without returning to the initial position (initial speed), and when the time is "3.0", it returns to the initial position (initial speed) and remains the same until the time "5.0". The position (speed) is maintained. Here, the operation performed between the times "2.0" and "3.0" is an extra operation as compared with the standard operation (see FIG. 4).

FIG. 6 is a diagram showing a third example D3 of motion information measured by the motion analysis system 100 according to the present embodiment. Also in the figure, the vertical axis shows the position or speed of a specific part of the worker, and the horizontal axis shows the time.

The third example D3 of the operation information shows an example of the operation information when the worker performs a non-standard operation for a certain element operation, and more specifically, the work is skipped (forgot to work). An example of operation information in the case is shown. In this case, the operator does not move the position or speed of the specific part from the initial position (initial speed) of the time "0.0", and keeps the same position (speed) until the time "5.0" as it is. .. That is, the operation that should be performed between the time "1.0" and "2.0" is not performed as standard.

FIG. 7 is a diagram showing a fourth example D4 of motion information measured by the motion analysis system 100 according to the present embodiment. Also in the figure, the vertical axis shows the position or speed of a specific part of the worker, and the horizontal axis shows the time.

The fourth example D4 of the operation information shows an example of the operation information when the worker performs a non-standard operation for a certain element operation, and more specifically, the operation information when the operation is redone. Is shown as an example. In this case, the operator moves the position or speed of the specific part from the initial position (initial speed) of the time "0.0" to the minus side at the time "1.0", and then moves the position or speed to the minus side at the time "2." When it is "0", it returns to the initial position (initial speed), when it is time "3.0", it moves to the minus side, then when it is time "4.0", it returns to the initial position (initial speed), and then. The same position (speed) is maintained until the time "5.0". Here, the operation performed between the times "3.0" and "4.0" is the same operation as the operation performed between the times "1.0" and "2.0", and the operation is performed. Is being redone.

FIG. 8 is a diagram showing a fifth example D5 of motion information measured by the motion analysis system 100 according to the present embodiment. Also in the figure, the vertical axis shows the position or speed of a specific part of the worker, and the horizontal axis shows the time.

The fifth example D5 of the operation information shows an example of the operation information when the worker performs a non-standard operation for a certain element operation, and more specifically, when a different operation (work error) is performed. An example of the operation information of is shown. In this case, the operator moves the position or speed of the specific part from the initial position (initial speed) of the time "0.0" to the plus side near the time "0.5", and then moves the position or speed to the plus side at the time "1.0". At this time, it returns to the initial position (initial speed) and keeps the same position (speed) until the time "5.0". That is, normally, the operation that should be performed between the times "1.0" and "2.0" is not performed, and different operations are performed between the times "0.5" and "1.0". It has been.

FIG. 9 is a flowchart of a first example of the classification process executed by the motion analysis system 100 according to the present embodiment. First, the measuring unit 30 measures the operation information including the coordinate values indicating the operation of the operator (S10). The measured operation information may be stored in the storage unit 14 as the operation information history 14b.

Next, the motion analysis device 10 classifies the motion information into a plurality of partial information by the first classification unit 15 based on the comparison between the motion information and the reference motion information (S11). Subsequently, the motion analyzer 10 classifies at least one partial information into any of a plurality of groups according to one or a plurality of criteria by the second classification unit 16 (S12). Here, the classification based on one or a plurality of criteria may be performed independently, or the classification based on one criterion may be subdivided by the classification according to another criterion.

The motion analyzer 10 determines whether a plurality of groups satisfy a predetermined condition (S13). Then, the motion analysis device 10 displays the frequencies and determination results of the plurality of groups on the display unit 10f (S14). With the above, the first example of the classification process is completed.

FIG. 10 is a diagram showing a first example of the group G1 classified by the motion analysis system 100 according to the present embodiment. In the figure, a plurality of groups G1 classified by the second classification unit 16 are shown with frequencies on the vertical axis, and the execution time of the operation, which is the first criterion used for classification by the second classification unit 16, is shown on the horizontal axis. Shown, the histogram shows the distribution of multiple groups G1. Further, in the figure, the operation information included in the time series information is classified into a plurality of partial information by the first classification unit 15, and the partial information corresponding to the element operation of "screw tightening" is a plurality of partial information by the second classification unit 16. The case where it is classified into the group G1 of is shown. Further, the classification by the second classification unit 16 is performed according to the first criterion regarding the execution time of the worker's movement. In the figure, the first criterion regarding the execution time of the worker's movement is shown as a "feature amount" that characterizes the partial information.

In the figure, the execution time on the horizontal axis shows a first threshold value and a second threshold value. The first threshold is the lower limit of the standard execution time of the work, and the group having the execution time shorter than the first threshold is highlighted by hatching as the group having the short work time. Further, the second threshold value is the upper limit value of the standard execution time of the work, and the group having the execution time longer than the second threshold value is highlighted by hatching as the group having the long work time.

The motion analysis device 10 may select a group corresponding to the input by the input unit 10e or a group satisfying a predetermined condition from the plurality of groups G1 by the selection unit 17. FIG. 10 shows a case where the user selects one group having a second threshold value or more by using the input unit 10e. In this case, the selection unit 17 may select a group according to the input by the input unit 10e. However, even if there is no input by the input unit 10e, the selection unit 17 selects, for example, a group below the first threshold value as a group satisfying a predetermined condition, or a group having a second threshold value or more satisfying a predetermined condition. May be selected as. The number of groups selected by the selection unit 17 may be plural.

FIG. 11 is a flowchart of a second example of the classification process executed by the motion analysis system 100 according to the present embodiment. The second example of the classification process may be performed in parallel with the first example of the classification process. The motion analysis system 100 captures a moving image of an operator associated with motion information by the first imaging unit 20a, the second imaging unit 20b, the third imaging unit 20c, and the measuring unit 30 (fourth imaging unit) ( S20). The captured moving image may be stored in the storage unit 14 as the moving image history 14a in association with the motion information measured at the same time. The motion analysis system 100 may measure motion information by the measuring unit 30 in parallel with shooting a moving image.

After that, the motion analyzer 10 classifies the motion information into a plurality of partial information by the first classification unit 15, and at least one partial information is classified into one of a plurality of groups according to the first criterion by the second classification unit 16. Classify and display a histogram. Then, the motion analysis device 10 selects a group corresponding to the input by the input unit 10e or a group satisfying a predetermined condition from the plurality of groups classified according to the first criterion by the selection unit 17 (S21).

The motion analyzer 10 classifies the partial information included in the selected group into one of a plurality of groups according to the second criterion by the second classification unit 16 (S22). Then, the display unit 10f displays the frequencies of the plurality of groups classified according to the second criterion (S23).

Further, the motion analysis device 10 selects a group corresponding to the input by the input unit 10e from a plurality of groups classified by the second criterion by the selection unit 17 (S24). Then, the motion analysis device 10 extracts at least one moving image from the moving image associated with the motion information included in the selected group by the extraction unit 18 (S25).

The motion analysis device 10 determines the playback range of the extracted moving image based on the type of the selected group by the determination unit 19 (S26). This completes the second example of the classification process.

FIG. 12 is a diagram showing a second example of the group G2 classified by the motion analysis system 100 according to the present embodiment.
In the figure, among the plurality of groups G1 shown in FIG. 10, the ninth group (the ninth group counted in ascending order of working time in the histogram of FIG. 10) selected according to the input by the input unit 10e is vertically arranged. The frequency is shown on the axis, the type of motion is shown on the horizontal axis, and the distribution of a plurality of groups G2 is shown by the histogram. Further, the figure shows a list L1 of moving images associated with the operation information included in the group selected according to the input by the input unit 10e among the plurality of groups G2.

In the plurality of groups G2, the frequency of "work skipping" is the highest, followed by the frequency of "screw retightening". In this way, with respect to the operation information corresponding to the element operation of "screw tightening" classified by the first classification unit 15, the operation information whose working time is too long is classified by the second classification unit 16, and further, the second classification unit According to 16, it is possible to classify the operation contents of the operation information whose working time is too long. This makes it possible to easily confirm what kind of operation was executed for what kind of element operation.

The moving image list L1 includes items of a start date / time, an end date / time, and a feature amount. To give a specific example, when the start date and time is "2017/11/6 9:35" and the end date and time is "2017/11/6 9:36", the feature amount is "F1. Here, the feature. The quantity may be an quantity related to the first criterion used for classification by the second classification unit 16, for example, the execution time of the worker's movement, or the length of the movement locus of a specific part that moves with the worker's movement. In this example, the start date and time is "2017/11/8 11:10", the end date and time is "2017/11/8 11:11", and the feature amount is "F3" depending on the input unit 10e. The video that is is selected. The extraction unit 18 may extract the selected moving image from the storage unit 14 and display it on the display unit 10f. In addition, the determination unit 19 may determine the reproduction range of the extracted moving image based on the type of the selected group.

FIG. 13 is a diagram showing a first example of a moving image reproduction range determined by the motion analysis system 100 according to the present embodiment. The first example is a case where the selected group is a group containing non-standard operation information, and the type of the selected group is a type in which extra work is performed. As shown in FIG. 5, the types of such groups can be classified according to whether or not the waveform of the motion information includes the waveform of another motion in addition to the waveform of the motion that should be performed as standard.

If the selected group type is a type that has performed extra work, in addition to "work 1", "work 2", and "work 3" that should be performed as standard, "work 2" and "work" "Work X" is performed between "3". Since this "work X" is an extra work, the determination unit 19 starts from the latter half of the "work 2" and includes the entire "work X" so that the state in which the "work X" is executed can be confirmed. , The playback range of the moving image may be determined so as to end in the first half of "Work 3".

FIG. 14 is a diagram showing a second example of a moving image reproduction range determined by the motion analysis system 100 according to the present embodiment. The second example is a case where the selected group is a group containing non-standard operation information, and the type of the selected group is a type in which the wrong work is performed. As shown in FIG. 8, the types of such groups can be classified according to whether or not the waveform of the motion information does not include the waveform of the motion that should be performed as standard, but includes the waveform of another motion.

If the selected group type is of the type that did the wrong work, then "work 1" and "work 3" instead of the standard "work 1", "work 2" and "work 3" "Work 2'" is being performed between ". Since this "work 2'" is an erroneous work, the determination unit 19 plays the video so as to include the entire "work 2'" so that the state in which the "work 2'" is executed can be confirmed. The range may be determined.

FIG. 15 is a diagram showing a third example of a moving image reproduction range determined by the motion analysis system 100 according to the present embodiment.
The third example is a case where the selected group is a group including non-standard operation information, and the type of the selected group is a type in which work is skipped. As shown in FIG. 6, the types of such groups can be classified according to whether or not the waveform of the motion information does not include the waveform of the motion that should be performed as standard and does not include other waveforms.

If the selected group type is the type in which work is skipped, "work 2" is skipped out of "work 1", "work 2", and "work 3" that should be performed as standard. .. In this case, the determination unit 19 determines the playback range of the moving image so that it starts in the middle of "work 1" and ends in the middle of "work 3" so that it can be confirmed that "work 2" is skipped. You can do it.

FIG. 16 is an example of the screen DP displayed by the motion analysis system 100 according to the present embodiment. The screen DP is an example in the case where the moving image list L1 shown in FIG. 12 is selected according to the input by the input unit 10e and the extracted moving image is displayed on the display unit 10f.

The screen DP includes an overview DP1, an overall video DP2, and a hand video DP3. Outline DP1 is information which shows the outline of the extracted operation information. Outline In DP1, the operation indicated by the extracted operation information is "non-standard operation", and the start time is "2017/11/8 11:10", that is, at 11:10 am on November 8, 2017. Yes, the end time is "2017/11/8 11:11", that is, 11:11 am on November 8, 2017, and the process indicated by the extracted operation information is the "packing process", which is the required time. Is shown to be "1 min" (1 minute). Note that the outline DP1 may indicate information that identifies the process indicated by the extracted operation information, instead of the name of the process indicated by the extracted operation information. For example, an ID such as a serial number assigned to each step may be indicated.

The entire moving image DP2 is the first moving image in which the entire work area R is photographed by the measuring unit 30 (fourth photographing unit), and "2017/11/8 11:10" is shown as the start time at the lower right of the moving image screen. ing. With the whole moving image DP2, it is possible to grasp the state in which a plurality of workers are executing the operation as a whole. Further, in the whole moving image DP2, the positions of the joints of the plurality of workers detected by the measuring unit 30 are shown by the skeleton model. From this, it can be confirmed that the coordinates of the joints of a plurality of workers are measured at appropriate positions.

In the case of this example, the hand video DP3 is a second video in which a region narrower than the work area R is shot by the third shooting unit 20c, and the start time is "2017/11/8 11:10" at the lower right of the video screen. It is shown. The details of the operation actually performed by the worker can be confirmed by the hand video DP3 which is a close-up of the worker's hand.

By displaying the entire moving image DP2 and the hand moving image DP3 in this way, the overall state of the specific operation and the details can be confirmed together. In addition, it is possible to confirm the position and time when a specific operation is performed, and to confirm the cooperation of a plurality of workers.

FIG. 17 is a diagram showing a third example of a group classified by the motion analysis system 100 according to the present embodiment. In the figure, the vertical axis shows the working time (execution time of the worker's movement), and the horizontal axis shows the locus length (the length of the movement locus of a specific part that moves with the worker's movement). It shows how the groups are distributed in the space on both axes. The plurality of groups shown in the figure move the partial information corresponding to the element movement of the screw tightening by the second classification unit 16 based on the reference regarding the execution time of the worker's movement and the movement of a specific part that moves with the worker's movement. The results of classification into one of multiple groups according to the criteria for the length of the locus are shown.

FIG. 17 shows a group C1 representing a standard operation and groups C11, C12, C13, and C14 representing a non-standard operation. Group C1 representing a standard motion has a medium motion execution time and movement locus length, and is located near the center of the graph. On the other hand, the groups C11, C12, C13, and C14 representing non-standard movements are located at the edge of the graph because at least one of the movement execution time and the length of the movement locus is too long or too short. ing.

Group C11 representing non-standard operation is a group including operation information when "screws do not fit in the screw holes and is stuck", and the trajectory length is about the same as group C1 representing standard operation, but operation. Execution time is longer than that of group C1, which represents a standard operation.

Group C12 representing non-standard operation is a group including operation information when "screw is dropped and picked up", and both the execution time and trajectory length of the operation are longer than group C1 representing standard operation. ..

Group C13, which represents a non-standard operation, is a group that includes operation information in the case of "forgot to tighten a screw", has a trajectory length shorter than group C1 that represents a standard operation, and has a standard operation execution time. It is shorter than the group C1 representing the operation.

Group C14 representing non-standard operation is a group including operation information in the case of "insufficient screw tightening", and the locus length is identified as group C1 representing standard operation, but the execution time of the operation. Is shorter than group C1, which represents standard behavior.

For example, when the partial information corresponding to the element operation of screw tightening is classified only by the standard regarding the length of the movement locus of a specific part, the group C1, the group C11, and the group C14 representing the standard operation are distinguished. Not done. As described above, even if the partial information is classified according to one criterion, it may not be possible to distinguish between the operation information corresponding to the standard operation and the operation information corresponding to the non-standard operation. Further, when the partial information corresponding to the element operation of screw tightening is classified only by the reference regarding the execution time of the operation, the group C11 and the group C12 are not distinguished, and the group C13 and the group C14 are not distinguished. As described above, even if the partial information is classified according to one criterion, it may not be possible to distinguish and classify the operation information corresponding to different types of non-standard operations. As in the second classification unit 16 according to the present embodiment, by classifying the partial information corresponding to the element operation into one of a plurality of groups according to a plurality of criteria, the operation information corresponding to the standard operation and the operation information corresponding to the standard operation can be obtained. It is possible to distinguish and classify the operation information corresponding to the non-standard operation, and further, it is possible to distinguish and classify the operation information corresponding to the different types of non-standard operation.

FIG. 18 is a diagram showing a fourth example of a group classified by the motion analysis system 100 according to the present embodiment. In the figure, the vertical axis shows the working time (execution time of the worker's movement), and the horizontal axis shows the locus length (the length of the movement locus of a specific part that moves with the worker's movement). It shows how the groups are distributed in the space on both axes. In the plurality of groups shown in the figure, the partial information corresponding to the element operation of the inspection is subjected to the reference regarding the execution time of the operation of the worker and the movement locus of the specific part that moves with the operation of the worker by the second classification unit 16. The results of classification into one of multiple groups according to the criteria for the length of the crab are shown.

FIG. 18 shows a group C2 representing a standard operation and groups C21, C22, C23, and C24 representing a non-standard operation. Group C2, which represents a standard motion, has a medium motion execution time and movement locus length, and is located near the center of the graph. On the other hand, the groups C21, C22, C23, and C24 representing non-standard operations are located at the edge of the graph because they are too long or too short for at least one of the operation execution time and the length of the movement locus. ing.

For group C21 representing non-standard operation, the operation information p included in group C21 is plotted, and the average (center of gravity) C when the distribution of operation information p is approximated by a mixed normal distribution and the standard deviation σ1 on the major axis. , The standard deviation σ2 on the minor axis is shown. The extraction unit 18 may extract a moving image based on the center of gravity C of the motion information p included in the selected group in the reference space used for the classification by the second classification unit 16. For example, when the group C21 is selected by the selection unit 17, the extraction unit 18 may extract the moving image corresponding to the operation information S1 closest to the center of gravity C, which is shown by the double circle. By extracting the moving image corresponding to the motion information S1 closest to the center of gravity C, the average motion included in the group C21 can be confirmed.

Further, the extraction unit 18 extracts a moving image based on the variance (standard deviation) σ1 and σ2 of the operation information p included in the selected group in the reference space used for the classification by the second classification unit 16. good. For example, when the group C21 is selected by the selection unit 17, the extraction unit 18 extracts the moving images corresponding to the operation information S2 and the operation information S3 shown by the double circles located at both ends of the long axis of the group C21. good. In this way, by extracting the moving images corresponding to the operation information S2 and S3 separated from the average C by about σ1 or σ2 based on the variances (standard deviations) σ1 and σ2, the operations of various modes included in the group C21 are performed. Can be confirmed.

Group C22, which represents a non-standard operation, is a group that includes operation information when "a part that has been confirmed once is confirmed again", and both the execution time and the trajectory length of the operation are higher than those of group C2, which represents a standard operation. Is also long. When the parts that have been confirmed once are confirmed in duplicate, the execution time and trajectory length of the operation will be longer by the amount of the duplicated confirmation.

Group C23, which represents a non-standard operation, is a group that includes operation information when "the confirmation point is skipped", the trajectory length is shorter than group C2, which represents a standard operation, and the execution time of the operation is standard. It is shorter than the group C1 representing the operation. If the confirmation point is skipped, there are some operations that are not executed, so the execution time and trajectory length of the operations are shortened by the amount of the operations that were not executed.

Group C24, which represents non-standard operation, is a group that includes operation information when "all parts have been confirmed but the confirmation time is insufficient", and the trajectory length is about the same as group C2, which represents standard operation. However, the execution time of the operation is shorter than that of group C2, which represents a standard operation. If all the parts are checked but the confirmation time is insufficient, the trajectory length of the operation is standard because there is no operation that was not executed, but the execution time of the operation is shortened because the confirmation time is insufficient.

For example, when the partial information corresponding to the element operation of the inspection is classified only by the standard regarding the length of the movement locus of a specific part, the group C2 representing the standard operation, a part of the group C21, and the group C22 Some are indistinguishable from group C24. Further, when the partial information corresponding to the element operation of the inspection is classified only by the reference regarding the execution time of the operation, a part of the group C21 and the group C22 is not distinguished, and the group C23 and the group C24 are not distinguished. Even in such a case, as in the second classification unit 16 according to the present embodiment, by classifying the partial information corresponding to the element operation into one of a plurality of groups according to a plurality of criteria, it is standard. It is possible to distinguish between the operation information corresponding to various operations and the operation information corresponding to the non-standard operation, and further to distinguish and classify the operation information corresponding to different types of non-standard operations. .. For example, by classifying the partial information corresponding to the elemental operation of the inspection using not only the standard related to the execution time of the operation but also the standard related to the trajectory length of the operation, there are cases where "the confirmation part is skipped" and "all". It is possible to distinguish from the case where "I checked the part but the confirmation time was insufficient", and whether it is necessary to take measures for overlooking the confirmed part or whether it is necessary to take measures for thorough confirmation time. Detailed analysis can be performed. Further, for example, by classifying the partial information corresponding to the elemental operation of the inspection by using not only the standard related to the locus length of the operation but also the standard related to the execution time of the operation, "the part confirmed once was confirmed again". It is possible to distinguish between the case and the case where "the confirmation time was insufficient even though all the parts were confirmed", and whether the work procedure is such that duplicate confirmation is induced, or the thorough confirmation time. It is possible to perform a detailed analysis such as whether measures need to be taken.

The embodiments described above are for facilitating the understanding of the present invention, and are not for limiting and interpreting the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those exemplified, and can be changed as appropriate. In addition, the configurations shown in different embodiments can be partially replaced or combined.

Embodiments of the present invention may also be described as the following appendices. However, the embodiments of the present invention are not limited to the embodiments described in the following appendices. Moreover, the embodiment of the present invention may be a form in which the description between the appendices is replaced or combined.

[Appendix 1]
An acquisition unit (11) that acquires time-series information indicating the operation of the worker, and
A first classification unit (15) that classifies the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker.
A second classification unit (16) that classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
Motion analyzer (10).

[Appendix 2]
A selection unit (17) for selecting one of the plurality of groups is further provided.
The one or more criteria include a first criterion and a second criterion different from the first criterion.
The second classification unit (16) classifies at least one of the partial information into one of a plurality of groups according to the first criterion, and the partial information included in the group selected by the selection unit (17). Is classified into any of a plurality of groups according to the second criterion.
The motion analyzer (10) according to Appendix 1.

[Appendix 3]
The selection unit (17) selects a group corresponding to the input by the input unit (10e) or a group satisfying a predetermined condition from the plurality of groups.
The motion analyzer (10) according to Appendix 2.

[Appendix 4]
The time-series information includes operation information including coordinate values indicating the operation of the worker.
The first classification unit (15) obtains the plurality of operation information based on the comparison between the operation information and the reference operation information including the coordinate value indicating the reference operation as the reference for comparison of the operation of the worker. Classify into partial information of
The motion analyzer (10) according to Appendix 2 or 3.

[Appendix 5]
The time-series information is associated with operation information including coordinate values indicating the operation of the worker, and includes a moving image of the worker.
An extraction unit (18) for extracting at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit (17) is further provided.
The motion analyzer (10) according to any one of Appendix 2 to 4.

[Appendix 6]
The extraction unit (18) extracts the moving image based on the center of gravity of the operation information included in the selected group in the reference space used for the classification by the second classification unit (16).
The motion analyzer (10) according to Appendix 5.

[Appendix 7]
The extraction unit (18) extracts the moving image based on the input by the input unit (10e).
The motion analyzer (10) according to Appendix 6.

[Appendix 8]
A determination unit (19) for determining a playback range of the extracted moving image based on the type of group selected by the selection unit (17) is further provided.
The motion analyzer (10) according to Appendix 6 or 7.

[Appendix 9]
The first criterion includes a criterion relating to the execution time of the worker's movement.
The motion analyzer (10) according to any one of Appendix 2 to 8.

[Appendix 10]
The first criterion includes a criterion regarding the length of the movement locus of a specific part that moves with the movement of the worker.
The motion analyzer (10) according to any one of Appendix 2 to 9.

[Appendix 11]
Acquiring time-series information indicating the operation of the worker and
To classify the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker.
To classify at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
Behavioral analysis methods including.

[Appendix 12]
The arithmetic unit provided in the motion analyzer,
Acquisition unit (11), which acquires time-series information indicating the operation of the worker,
The first classification unit (15) that classifies the time-series information into a plurality of partial information corresponding to the plurality of element actions executed by the worker, and at least one said partial information is related to the action of the worker. Second classification unit (16), which classifies into one of a plurality of groups according to one or a plurality of criteria,
A motion analysis program that functions as.

[Appendix 13]
A measuring unit (30) that measures motion information including coordinate values indicating the operator's motion, and
Shooting units (20a, 20b, 20c) for shooting a moving image of the worker associated with the operation information, and
The first classification unit (1st classification unit) that classifies the operation information into a plurality of partial information based on the comparison between the operation information and the reference operation information including the coordinate values indicating the reference operation that serves as a reference for comparison of the operation of the worker. 15) and
A second classification unit (16) that classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
A selection unit (17) for selecting one of the plurality of groups, and
An extraction unit (18) that extracts at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit (17).
A display unit (10f) that displays the extracted moving image, and
Motion analysis system (1).

[Appendix 14]
The time-series information includes operation information including coordinate values indicating the operation of the worker.
The first classification unit (15) obtains the plurality of operation information based on the comparison between the operation information and the reference operation information including the coordinate value indicating the reference operation as the reference for comparison of the operation of the worker. Classify into partial information of
The motion analyzer (10) according to Appendix 1.

[Appendix 15]
The time-series information is associated with operation information including coordinate values indicating the operation of the worker, and includes a moving image of the worker.
An extraction unit (18) for extracting at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit (17) is further provided.
The motion analyzer (10) according to Appendix 1 or 14.

[Appendix 16]
The extraction unit (18) extracts the moving image based on the center of gravity of the operation information included in the selected group in the reference space used for the classification by the second classification unit (16).
The motion analyzer (10) according to Appendix 15.

[Appendix 17]
The extraction unit (18) extracts the moving image based on the input by the input unit (10e).
The motion analyzer (10) according to Appendix 16.

[Appendix 18]
A determination unit (19) for determining a playback range of the extracted moving image based on the type of group selected by the selection unit (17) is further provided.
The motion analyzer (10) according to Appendix 16 or 17.

10 ... Motion analyzer, 10a ... CPU, 10b ... RAM, 10c ... ROM, 10d ... Communication unit, 10e ... Input unit, 10f ... Display unit, 11 ... First acquisition unit, 12 ... Second acquisition unit, 13 ... 3 Acquisition unit, 14 ... Storage unit, 14a ... Video history, 14b ... Operation information history, 14c ... Reference operation information, 15 ... First classification unit, 16 ... Second classification unit, 17 ... Selection unit, 18 ... Extraction unit, 19 ... determination unit, 20a ... first imaging unit, 20b ... second imaging unit, 20c ... third imaging unit, 30 ... measurement unit, 100 ... motion analysis system, C1, C2 ... group representing standard motion, C11 , C12, C13, C14, C21, C22, C23, C24 ... Group representing non-standard operation, D1 ... First example of operation information, D2 ... Second example of operation information, D3 ... Third example of operation information, D4 ... 4th example of operation information, D5 ... 5th example of operation information, G1 ... 1st example of group, G2 ... 2nd example of group, L1 ... video list, DP ... screen, DP1 ... overview, DP2 ... whole Video, DP3 ... Video at hand, R ... Work area

Claims (10)

An acquisition unit that is associated with operation information including coordinate values indicating the operator's operation and acquires time-series information including a moving image of the worker.
A first classification unit that classifies the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker.
A second classification unit that classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
A selection unit that selects one of the plurality of groups,
It is provided with an extraction unit that extracts at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit.
The plurality of criteria regarding the movement of the worker include a first criterion and a second criterion different from the first criterion.
The second classification unit classifies at least one of the partial information into one of a plurality of groups according to the first criterion, and classifies the partial information included in the group selected by the selection unit into the second criterion. Classified into one of multiple groups by
The extraction unit is an operation analyzer that extracts the moving image based on the center of gravity of the operation information included in the selected group in the reference space used for classification by the second classification unit. The selection unit selects a group corresponding to input by the input unit or a group satisfying a predetermined condition from the plurality of groups.
The motion analyzer according to claim 1. The time-series information includes operation information including coordinate values indicating the operation of the worker.
The first classification unit obtains the operation information into the plurality of partial information based on the comparison between the operation information and the reference operation information including the coordinate value indicating the reference operation as the reference for comparison of the operation of the worker. Classify into
The motion analyzer according to claim 1 or 2. The extraction unit extracts the moving image based on the input by the input unit.
The motion analyzer according to claim 1. A determination unit for determining a playback range of the extracted moving image based on the type of group selected by the selection unit is further provided.
The motion analyzer according to claim 1 or 4. The first criterion includes a criterion relating to the execution time of the worker's movement.
The motion analyzer according to any one of claims 1 to 5. The first criterion includes a criterion regarding the length of the movement locus of a specific part that moves with the movement of the worker.
The motion analyzer according to any one of claims 1 to 6. The computer is associated with the operation information including the coordinate values indicating the operation of the worker, and acquires the time series information including the moving image of the worker.
A computer classifies the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker.
The computer classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
When the computer selects one of the plurality of groups,
The computer comprises extracting at least one video from the video associated with the motion information included in the selected group.
The plurality of criteria regarding the movement of the worker include a first criterion and a second criterion different from the first criterion.
Classification into any of the plurality of groups means that at least one of the partial information is classified into one of the plurality of groups according to the first criterion, and the partial information contained in the selected group is classified into the above-mentioned partial information. Classify into one of multiple groups according to the second criterion
The extraction is an motion analysis that extracts the moving image based on the center of gravity of the motion information included in the selected group in the reference space used when classifying into one of the plurality of groups. Method. The arithmetic unit provided in the motion analyzer,
An acquisition unit that is associated with motion information including coordinate values indicating the worker's motion and acquires time-series information including a moving image of the worker.
A first classification unit that classifies the time-series information into a plurality of partial information corresponding to a plurality of element actions executed by the worker.
A second classification unit, which classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
It functions as a selection unit that selects one of the plurality of groups and an extraction unit that extracts at least one video from the video associated with the operation information included in the group selected by the selection unit. ,
The plurality of criteria regarding the movement of the worker include a first criterion and a second criterion different from the first criterion.
The second classification unit classifies at least one of the partial information into one of a plurality of groups according to the first criterion, and classifies the partial information included in the group selected by the selection unit into the second criterion. Classified into one of multiple groups by
The extraction unit is an operation analysis program that extracts the moving image based on the center of gravity of the operation information included in the selected group in the reference space used for classification by the second classification unit. A measuring unit that measures motion information including coordinate values indicating the operator's motion,
A shooting unit that shoots a moving image of the worker associated with the operation information,
A first classification unit that classifies the operation information into a plurality of partial information based on the comparison between the operation information and the reference operation information including the coordinate values indicating the reference operation that serves as a reference for comparison of the operation of the worker. ,
A second classification unit that classifies at least one of the partial information into one of a plurality of groups according to one or a plurality of criteria regarding the operation of the worker.
A selection unit that selects one of the plurality of groups,
An extraction unit that extracts at least one moving image from the moving image associated with the operation information included in the group selected by the selecting unit, and an extraction unit.
A display unit that displays the extracted video and
With
The plurality of criteria regarding the movement of the worker include a first criterion and a second criterion different from the first criterion.
The second classification unit classifies at least one of the partial information into one of a plurality of groups according to the first criterion, and classifies the partial information included in the group selected by the selection unit into the second criterion. Classified into one of multiple groups by
The extraction unit is an operation analysis system that extracts the moving image based on the center of gravity of the operation information included in the selected group in the reference space used for classification by the second classification unit.

Images