WO2019138877A1

WO2019138877A1 - Motion-analyzing system, motion-analyzing device, motion analysis method, and motion analysis program

Info

Publication number: WO2019138877A1
Application number: PCT/JP2018/047812
Authority: WO
Inventors: 浩臣音田; 修一後藤; 直浩河合
Original assignee: オムロン株式会社
Priority date: 2018-01-12
Filing date: 2018-12-26
Publication date: 2019-07-18
Also published as: JP2019125023A; JP6951685B2

Abstract

Provided is a motion-analyzing system with which it is possible to easily extract a scene in which a specific motion has been executed. The motion-analyzing system is provided with a measurement unit for measuring motion information that indicates the motion of one or more workers which has been executed in some work area; an imaging unit for capturing a moving-image that includes a scene in which the workers are executing a motion; a storage unit for storing reference motion information that indicates a reference motion that serves as a reference for comparison with regard to the motion of the workers; an extraction unit for comparing the motion information with the reference motion information and extracting motion information that satisfies a prescribed condition; and a display unit for displaying, using the moving-image, a scene in which the workers are executing the motion indicated by the extracted motion information.

Description

Motion analysis system, motion analysis apparatus, motion analysis method, and motion analysis program

The present invention relates to a motion analysis system, a motion analysis device, a motion analysis method, and a motion analysis program.

Conventionally, one or more cameras are installed on a product manufacturing line, and an operator's operation may be recorded as an image or a moving image. In recent years, motion capture that can measure the three-dimensional position and movement speed of joints without mounting a tracker has begun to spread, and in addition to images and moving pictures, motion information that quantitatively indicates the motion of the worker You may get

In Patent Document 1 below, an operation signal representing a change in the feature amount of the worker's moving image and the worker's operation is acquired, and the operation signal and the operation result when the operation result is good are poor. A work movement analysis system has been described which compares a movement signal of a case to extract a difference and displays a moving image corresponding to the difference.

JP, 2009-110239, A

When various operations are performed on a manufacturing line, it may be desirable to confirm a scene where an operator has performed a specific operation. In order to meet such a demand, one or more cameras may be installed on the production line to continuously capture the motion of the worker by moving pictures. Here, for example, according to the technology described in Patent Document 1, the difference between the case where the work result is good and the case where the work result is bad in order to extract the problem when the result of the work is bad The video corresponding to the point is displayed. However, when the work result is not always clear, the technology described in Patent Document 1 can not be applied, and it is also impossible to extract a scene in which the operator has performed a specific operation.

In addition, even if one or more cameras are installed on the manufacturing line and the operator's action is continuously photographed as a moving image, the number of installed moving cameras increases and the number of moving images requiring confirmation increases. When the length of an individual moving picture becomes long, even if a scene where a specific action is performed is recorded in any part of the taken moving picture, the specific action is performed It becomes difficult to identify the scene in which a specific action has been performed.

Therefore, the present invention provides a motion analysis system, a motion analysis device, a motion analysis method, and a motion analysis program that can easily extract a scene in which a specific motion is performed.

A motion analysis system according to an aspect of the present disclosure includes: a measurement unit configured to measure motion information indicating a motion of one or more workers performed in a work area; and a moving image including a scene where the worker is performing a motion An operation that satisfies a predetermined condition by comparing an imaging unit that captures an image, a storage unit that stores reference operation information indicating a reference operation that is a reference for comparison with the operation of the worker, the operation information, and the reference operation information The information processing apparatus includes an extraction unit that extracts information, and a display unit that displays a scene in which an operator is performing an operation indicated by the extracted operation information using a moving image. Here, the motion information may be information indicating the movement of the worker's body, and may be information indicating the displacement of a representative position of the worker's body. The representative position of the worker's body may be one position of the body, but typically there may be more than one. The motion information can be calculated, for example, by projecting the pattern light to the worker and extracting feature points from the captured moving image. The reference operation may be a standard operation that the operator should follow or a non-standard operation of the operator such as a mistake case. When the reference operation is a standard operation, the predetermined condition may be a condition that the difference between the operation information and the reference operation information is equal to or more than a threshold. When the reference operation is an operation other than the standard, the predetermined condition may be a condition that the difference between the operation information and the reference operation information is equal to or less than a threshold. The reference operation information may be information that can be compared with the operation information, and may not necessarily be information of the same data format as the operation information. The reference operation information and the operation information may be time-series data. Also, “a scene where the worker is executing the motion indicated by the extracted motion information” refers to a scene of a time zone in which the worker is executing the motion indicated by the motion information extracted from one moving image. It may be a scene of a part of moving images in which the operator is executing an operation indicated by the extracted operation information out of a plurality of moving images. Further, the operation indicated by the operation information may be an operation corresponding to quantitative information on the operation of the worker indicated by the operation information.

According to this aspect, of the operation information quantitatively indicating the operation of the worker, the operation information satisfying the predetermined condition in comparison with the reference operation information is extracted, and the extracted operation information is extracted using the moving image. By displaying a scene in which the worker is executing the operation to be shown, it is possible to easily extract a scene in which a specific operation satisfying a predetermined condition is performed. By extracting a scene in which a specific operation satisfying a predetermined condition is performed, for example, regardless of whether the result of the work by the worker is good or bad, the scene in which the worker performed the specific operation is extracted can do. Also, for example, even when the number of moving images requiring confirmation increases or the length of each moving image increases, it is possible to easily extract a scene in which a specific operation is performed, and the extraction is performed. The burden of work can be reduced.

In the above aspect, the measurement unit may measure a plurality of pieces of operation information respectively indicating the operations of the plurality of workers, and the display unit may display information identifying the worker for which the extracted operation information is measured. .

According to this aspect, among the plurality of pieces of operation information respectively indicating the operations of the plurality of workers, the operation information satisfying the predetermined condition in comparison with the reference operation information is extracted, and the extracted operation information is measured. By displaying the information for identifying the worker, it is possible to identify the worker who has executed the specific operation satisfying the predetermined condition among the plurality of workers. Therefore, for example, it is possible to reduce the burden of confirming who has performed a particular operation.

In the above aspect, the imaging unit includes a first imaging unit that captures a first moving image that captures an image of the work area, and a second imaging unit that captures a second moving image that captures a portion of the work area, The scene of the time zone in which the worker is executing the operation indicated by the extracted operation information may be displayed by using the first moving image and the second moving image.

According to this aspect, a specific condition satisfying a predetermined condition is displayed by displaying a scene of a time zone in which the worker is executing the operation indicated by the extracted operation information among the first moving images obtained by photographing the work area. By displaying the scene of the time zone in which the operator can execute the operation indicated by the extracted operation information among the second moving images obtained by photographing a part of the work area, the operation can be confirmed as a whole. It is possible to confirm the details of a specific operation that satisfies a predetermined condition. Therefore, for example, even when the length of the moving image becomes long, the scene of the time zone in which the specific operation is performed can be easily extracted, and the load of the extraction operation can be reduced.

In the above aspect, the second imaging unit captures a plurality of second moving images obtained by capturing a plurality of parts of the work area, and estimates a position in the work area where the operation indicated by the extracted motion information is performed. The display unit may display a second moving image captured at an estimated position among the plurality of second moving images.

According to this aspect, by estimating the position in the work area where the operation indicated by the extracted motion information is performed, it is possible to estimate the position where the specific operation satisfying the predetermined condition is performed, and the work area By displaying the second moving image captured at the estimated position among the plurality of second moving images captured at multiple positions in, it is possible to confirm the details of the operation performed at the estimated position. . Therefore, for example, it is possible to reduce the burden of confirming where a particular operation has been performed.

In the above aspect, the reference operation information is determined for each of the plurality of steps, and the extraction unit compares the operation information with the reference operation information for each of the plurality of steps and determines the operation information satisfying the predetermined condition. The extraction unit may display information identifying a process indicated by the extracted operation information.

According to this aspect, among the motion information indicating the motion of the worker, the motion information satisfying the predetermined condition is extracted in comparison with the reference motion information determined for each of the plurality of steps, and the extracted motion information is By displaying the information for identifying the indicated process, it is possible to confirm in which process the specific operation satisfying the predetermined condition is the operation performed. Therefore, for example, it is possible to reduce the burden of confirming in which process a specific operation has been performed.

In the above aspect, the estimation unit may estimate the position in the work area where the worker is executing the operation indicated by the extracted operation information, based on the process indicated by the extracted operation information.

According to this aspect, the plurality of positions in the work area are estimated by estimating the position in the work area where the worker is executing the operation indicated by the extracted operation information based on the process indicated by the extracted operation information. The second moving image photographed at the position where the process indicated by the extracted operation information is performed among the plurality of second moving images photographed in step can be displayed, and the details of the operation performed in the step are confirmed can do. Therefore, for example, it is possible to reduce the burden of confirming where and for which process a specific operation has been performed.

In the above aspect, the motion information and the reference motion information respectively include coordinate values of the joint of the worker, and the extraction unit compares the coordinate value included in the motion information with the coordinate value included in the reference motion information, You may extract the operation information which satisfy | fills a predetermined condition.

According to this aspect, by comparing the coordinate values of the joints included in the movement information with the coordinate values of the joints included in the reference movement information, the degree to which the movement information deviates from the reference movement information is accurately determined. It is possible to evaluate and to appropriately extract operation information that satisfies a predetermined condition.

In the above aspect, the estimation unit may estimate the position in the work area where the worker is executing the operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information.

According to this aspect, based on the coordinate values of the joints included in the extracted motion information, it is extracted by estimating the position in the work area where the worker is executing the motion indicated by the extracted motion information. It is possible to accurately estimate the position at which the motion indicated by the motion information is performed.

In the above aspect, the reference motion information is determined for each of the plurality of element motions, and the estimation unit is extracted based on the coordinate value included in the extracted motion information and the element motion indicated by the extracted motion information. The position in the work area where the worker is executing the motion indicated by the motion information may be estimated. Here, the element operation is a unit operation performed by the operator, and includes, for example, an operation such as picking of parts, arrangement of parts, fixing of parts, packing of products. The element operation may be a component of the operator's operation, and a combination of one or more element operations may constitute a series of operations of the operator.

According to this aspect, based on the coordinate value of the joint included in the extracted motion information and the element motion indicated by the extracted motion information, a work area in which the operator performs the motion indicated by the extracted motion information By estimating the position in, the second moving image photographed at the position at which the element motion indicated by the extracted motion information is displayed among the plurality of second moving images photographed at the plurality of positions in the work area is displayed. And the details of the element operation can be confirmed. Therefore, for example, it is possible to reduce the burden of confirming where a particular operation has been performed with respect to which element operation.

In the aspect described above, the display unit displays a scene in which the operator is executing the operation indicated by the operation information extracted using the moving image, and a moving image in which the operator is executing the reference operation indicated by the reference operation information. You may display it. For example, the display unit displays one screen of a scene in which the operator is executing the operation indicated by the operation information extracted using the moving image, and a moving image in which the operator is executing the reference operation indicated by the reference operation information. They may be displayed in parallel at the same time, may be superimposed on one screen and may be displayed simultaneously, or may be displayed continuously and alternately.

According to this aspect, the worker displays the scene where the operator is executing the operation indicated by the motion information extracted using the moving image and the moving image where the operator is executing the reference operation. It becomes easy to compare the specific operation that fulfills the predetermined condition that has been executed with the reference operation.

In the above aspect, the display unit may display a graph indicating the extracted operation information and a scene of a moving image in which the operator is executing the operation indicated by the extracted operation information.

According to this aspect, the specific operation satisfying the predetermined condition is displayed by displaying the graph indicating the extracted operation information and the scene of the moving image in which the operator is executing the operation indicated by the extracted operation information. It is possible to confirm the scene where was executed from different viewpoints of animation and graph. Therefore, for example, it is possible to reduce the burden of qualitatively and quantitatively confirming a specific operation.

In the above aspect, the display unit may superimpose a plurality of frames included in a scene in which the worker is executing an operation indicated by the extracted operation information in the moving image and display the frame as a single image.

According to this aspect, a predetermined condition can be obtained by superimposing a plurality of frames included in a scene in which the operator is executing an operation indicated by motion information extracted from the moving image and displaying the frame as a single image. It is possible to see at a glance the whole scene where a specific action to be fulfilled has been performed. Therefore, for example, the entire specific operation can be confirmed in a short time, and the burden of confirmation can be reduced.

A motion analysis apparatus according to another aspect of the present disclosure includes a first acquisition unit that acquires motion information indicating the motion of one or more workers executed in a work area, and a scene where the worker is executing a motion A second acquisition unit for acquiring a moving image including a third acquisition unit for acquiring a reference operation information indicating a reference operation as a reference for comparison of the operation of the worker, the operation information, and the reference operation information An extraction unit that extracts operation information that satisfies a predetermined condition, and an identification unit that specifies a scene in which an operator is executing an operation indicated by the extracted operation information using a moving image.

According to this aspect, of the operation information indicating the operation of the worker, the operation information satisfying the predetermined condition in comparison with the reference operation information is extracted, and the operation indicated by the extracted operation information is displayed using the moving image. By identifying the scene being executed by the worker, it is possible to easily extract the scene in which the specific operation satisfying the predetermined condition is executed. Therefore, for example, regardless of whether the result of the work by the worker is good or bad, it is possible to extract a scene where the worker performed a specific operation. Also, for example, even when the number of moving images requiring confirmation increases or the length of each moving image increases, it is possible to easily extract a scene in which a specific operation is performed, and the extraction is performed. The burden of work can be reduced.

A motion analysis method according to another aspect of the present disclosure includes acquiring motion information indicating a motion of one or more workers performed in a work area, and a moving image including a scene in which the worker is performing a motion Obtaining motion information, obtaining reference motion information indicating a reference motion serving as a comparison reference for the motion of the worker, comparing the motion information with the reference motion information, and obtaining motion information satisfying a predetermined condition Extracting and identifying a scene in which the worker is executing the operation indicated by the extracted operation information using the moving image.

A motion analysis program according to another aspect of the present disclosure includes: a first acquisition unit that acquires operation information indicating an operation of one or more workers executed in a work area, in an arithmetic device provided in a motion analysis device; A second acquisition unit that acquires a moving image including a scene in which an operator is executing an operation; a third acquisition unit that acquires reference operation information indicating a reference operation that is a reference for comparison of the worker's operation; An extraction unit that compares operation information with reference operation information and extracts operation information that satisfies a predetermined condition, and an identification unit that specifies a scene where an operator is executing an operation indicated by the extracted operation information using a moving image Act as.

According to this aspect, of the operation information indicating the operation of the worker, the operation information satisfying the predetermined condition in comparison with the reference operation information is extracted, and the operation indicated by the extracted operation information is displayed using the moving image. By identifying a scene being executed by the worker, it is possible to extract a scene in which a specific operation satisfying a predetermined condition is executed. Therefore, for example, regardless of whether the result of the work by the worker is good or bad, it is possible to extract a scene where the worker performed a specific operation. Also, for example, even when the number of moving images requiring confirmation increases or the length of each moving image increases, it is possible to easily extract a scene in which a specific operation is performed, and the extraction is performed. The burden of work can be reduced.

According to the present invention, it is possible to provide a motion analysis system, a motion analysis device, a motion analysis method, and a motion analysis program capable of easily extracting a scene in which a specific motion is performed.

It is a figure showing the outline of the operation analysis system concerning the embodiment of the present invention. It is a figure showing the functional block of the operation analysis system concerning this embodiment. It is a figure which shows the physical structure of the movement analysis apparatus which concerns on this embodiment. It is a figure which shows an example of the movement information measured by the movement analysis system which concerns on this embodiment. It is a figure which shows the example of the 1st reference | standard operation information memorize | stored by the movement analysis system which concerns on this embodiment. It is a figure which shows the example of the 1st corresponding | compatible table memorize | stored by the movement analysis system which concerns on this embodiment. It is a flow chart of the 1st example of extraction processing of operation information performed by operation analysis system concerning this embodiment. It is a flow chart of the 1st example of the animation display processing performed by the operation analysis system concerning this embodiment. It is an example of the screen displayed by the movement analysis system concerning this embodiment. It is a figure which shows the example of the 2nd corresponding | compatible table memorize | stored by the movement analysis system which concerns on this embodiment. It is a flow chart of the 2nd example of the animation display processing performed by the operation analysis system concerning this embodiment. It is a figure which shows the example of the 2nd reference | standard operation information memorize | stored by the movement analysis system which concerns on this embodiment. It is a flow chart of the 2nd example of extraction processing of operation information performed by operation analysis system concerning this embodiment. It is a flowchart of the 3rd example of the moving image display process performed by the motion analysis system which concerns on this embodiment. It is a figure which shows the example of the 3rd corresponding | compatible table memorize | stored by the movement analysis system which concerns on this embodiment. It is a flowchart of the 4th example of the moving image display process performed by the motion analysis system which concerns on this embodiment. It is a flow chart of selection processing of a display mode performed by a motion analysis system concerning a modification. It is an example of the screen displayed by the movement analysis system concerning a modification. It is a figure showing the functional block of the operation analysis system concerning other embodiments of this indication.

Hereinafter, an embodiment according to one aspect of the present invention (hereinafter referred to as “the present embodiment”) will be described based on the drawings. In addition, what attached the same code | symbol in each figure has the same or same structure.

First, an example of a scene to which the present invention is applied will be described with reference to FIG. The motion analysis system 100 according to the present embodiment takes a measurement unit 30 that measures motion information that quantitatively indicates a worker's motion performed in a certain work area R, and captures a moving image in which the worker is performing a motion And a second imaging unit 20b and a third imaging unit 20c. Although the work area R in this example is an area including the entire manufacturing line, the work area R may be any area, for example, an area where a predetermined process is performed or a predetermined element operation is performed. It may be an area. Here, the element operation is a unit operation performed by the operator, and includes, for example, an operation such as picking of parts, arrangement of parts, fixing of parts, packing of products.

In this example, a case where the first worker A1 and the second worker A2 perform predetermined operations in the work area R will be described. The first worker A1 can perform, for example, an operation such as picking, arranging, and fixing a first part, and the second worker A2 can perform an operation such as picking, arranging, and fixing a second part, for example.

The motion analysis system 100 includes a motion analysis device 10. The motion analysis apparatus 10 compares the motion information and the reference motion information with a storage unit that stores reference motion information that quantitatively indicates a reference motion serving as a reference for comparison of the motion of the worker, and determines predetermined conditions. An extraction unit that extracts operation information to be satisfied, and a display unit 10 f that displays a scene where an operator is performing an operation indicated by the extracted operation information using a moving image. Here, the operation indicated by the operation information is an operation corresponding to quantitative information on the operation of the worker indicated by the operation information. The storage unit may be separate from the motion analysis device 10, as long as it can communicate with the motion analysis device 10.

Here, the reference operation may be a standard operation to be followed by the worker, or may be a worker's mistake operation or a non-standard operation. When the reference operation is a standard operation, the predetermined condition may be a condition that the difference between the operation information and the reference operation information is equal to or more than a threshold. In this case, the extraction unit extracts operation information indicating a nonstandard operation. On the other hand, when the reference operation is a miss operation or an operation outside the standard, the predetermined condition may be a condition that the difference between the operation information and the reference operation information is equal to or less than a threshold. In this case, the extraction unit extracts a misoperation or a nonstandard operation. In the following, the case where the reference operation is a standard operation and the predetermined condition is a condition that the difference between the operation information and the reference operation information is equal to or more than a threshold will be described. Note that the non-standard operation may include an operation that may affect the quality of the product, such as a worker coughing or rubbing his / her nose.

The display unit 10 f may display a scene of a time zone during which the worker is executing the operation indicated by the extracted operation information of one moving image, or the operation information extracted from the plurality of moving images is indicated. You may display the scene of the one part animation which the operator is performing.

The movement analysis system 100 measures movement information indicating displacement of representative positions of the worker's body, such as displacements of coordinate values of a plurality of joints of the worker, by the measurement unit 30. Then, operation information when a standard operation to be followed by the worker is executed is stored in advance in the storage unit as reference operation information.

After that, the measurement unit 30 measures operation information of the worker, and in parallel with the measurement of the operation information by the measurement unit 30, the worker operates by the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c. Shoot the moving image that is being executed and store it in the storage unit. Subsequently, the extraction unit compares the measured operation information with the reference operation information stored in the storage unit to extract operation information having a difference between the operation information and the reference operation information equal to or greater than a threshold. . Here, for comparison of the operation information and the reference operation information, for example, specifying the start time point and the end time point of the operation information indicating a certain operation, and finding the difference between the operation information recorded at the corresponding timing and the reference operation information You may go there.

Then, the worker is executing the operation indicated by the operation information whose divergence is equal to or more than the threshold compared with the extracted operation information, that is, the reference operation information when the standard operation to be followed by the operator is executed. The scene of the moving image is displayed by the display unit 10f.

As described above, among the motion information quantitatively indicating the motion of the worker, the motion information satisfying the predetermined condition in comparison with the reference motion information is extracted, and the motion indicated by the extracted motion information using the moving image By displaying the scene in which the worker is executing, it is possible to easily extract the scene in which the specific operation satisfying the predetermined condition is performed regardless of the number or length of the captured moving images. For example, by displaying a scene of a time zone in which the operation indicated by the extracted motion information in one moving image is executed, a scene suitable for confirming a specific operation is extracted from the one moving image. be able to. In addition, for example, to confirm a specific operation by displaying a scene of a part of moving images executing an operation indicated by the extracted operation information among a plurality of moving images captured by a plurality of imaging units. A moving image captured by a suitable imaging unit can be extracted. Furthermore, the motion information which extracted the operation information which satisfy | fills a predetermined condition, and identifies the scene where the operator is performing the operation | movement which the extracted operation information shows using a moving image, The moving image which image | photographed a worker's operation | movement The calculation load can be significantly reduced as compared with searching for a scene in which a specific operation satisfying a predetermined condition is performed by comparing the motion image captured with the reference operation with the moving image.

For example, when the reference operation is a standard operation and the predetermined condition is a condition that the difference between the operation information and the reference operation information is equal to or greater than a threshold, according to the operation analysis system 100 according to the present embodiment, It is possible to extract moving image scenes in which non-standard operations are performed. As a result, in the short term, it is possible to detect early that a non-standard operation has been performed, and to reduce, for example, the possibility of defective products being produced on the production line. Even if a non-standard operation is performed, even if a non-defective product is manufactured, it has technical significance in that it can catch signs that a problem occurs and make the subsequent confirmation work more efficient. Such a merit is a merit that can be obtained in real time when using the motion analysis system 100.

In addition, in the long term, it can be used for assisting the learning of the worker, or can provide materials for considering improvement of the standard operation. For example, it is conceivable that many workers identify non-standard operating parts to improve the standard operation. Such an advantage is an advantage obtained when the motion analysis system 100 is continuously used, and an advantage obtained in non-real time.

22 Configuration example [Function configuration]
Next, an example of a functional configuration of the operation analysis system 100 according to the present embodiment will be described using FIG. The motion analysis system 100 includes a first imaging unit 20a, a second imaging unit 20b, a third imaging unit 20c, a measurement unit 30, and the operation analysis apparatus 10. The motion analysis apparatus 10 includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, a storage unit 14, an extraction unit 15, an identification unit 16, an estimation unit 17, and a display unit 10f.

<Structure of Shooting Unit>
The first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may be configured by general-purpose cameras, and the first operator A1 and the second operator A2 execute operations in the work area R. You may shoot a movie that includes a scene that you The first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may respectively capture a part of the work area R, and may capture a moving image in an area smaller than the work area R. Specifically, a moving image in which the operations performed by the first worker A1 and the second worker A2 are closed up may be taken. The first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may capture, for example, a moving image in which the hands of the first worker A1 and the second worker A2 are close up.

In addition, the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may capture a plurality of moving images obtained by capturing a plurality of portions of the work area R, respectively. For example, the first imaging unit 20a mainly captures a moving image in which the first operator A1 is performing an operation, and the third imaging unit 20c mainly performs a moving image in which the second operator A2 is performing an operation The second imaging unit 20b may capture both the moving image in which the first operator A1 is performing an operation and the moving image in which the second operator A2 is performing an operation. In addition, the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c may capture a moving image in which different steps are performed at a plurality of positions in the work area R.

<Configuration of Measurement Unit>
The measurement unit 30 may be configured by motion capture, and may measure a plurality of pieces of operation information quantitatively showing the operations of the first worker A1 and the second worker A2 performed in a certain work area R. Although the configuration of the measurement unit 30 is arbitrary, for example, the pattern light is projected to the first worker A1 and the second worker A2, and the first worker A1 and the second worker in a state in which the pattern light is projected A moving image of A2 may be photographed, and coordinate values of a plurality of joints of the first worker A1 and the second worker A2 may be measured based on the photographed moving image. In addition, when measuring a plurality of pieces of operation information regarding a plurality of workers, the measurement unit 30 may add information identifying the worker to the operation information. In addition, the measurement unit 30 may measure other than the coordinate values of the plurality of joints of the first worker A1 and the second worker A2, and, for example, the measurement unit 30 of the first worker A1 and the second worker A2 Coordinate values of representative positions of the body not necessarily joints, such as fingertips and heads, may be measured. Coordinate values of joints and coordinate values of representative positions of bodies other than joints may be measured together. It may be In addition, the measurement unit 30 may measure the coordinate values of the positions of the trackers worn by the first worker A1 and the second worker A2, and in this case, the representative position of the body is It may be a position. Also, the representative position of the body may be reworded as the position of feature points of the body.

The motion analysis system 100 may include a plurality of measurement units 30. When the operation information of a plurality of workers is measured by a plurality of measuring units 30, the operation information of the same worker may be measured in duplicate, but information for identifying the worker is added to the operation information Then, duplicates may be removed, or operation information measured by different measurement units 30 may be combined.

The measurement unit 30 may also serve as a fourth imaging unit that captures a moving image in which the first worker A1 and the second worker A2 are performing an operation. The fourth imaging unit may capture a moving image of the entire work area R. That is, the fourth imaging unit may capture the state in which the first worker A1 and the second worker A2 are performing operations so that both the first worker A1 and the second worker A2 are included. . On the other hand, the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit 20c are the first worker A1 and the second worker so that one of the first worker A1 and the second worker A2 is included. A2 may shoot a moving image in action.

A moving image obtained by shooting the work area R shot by the measuring unit 30 (fourth shooting unit) corresponds to the “first moving image” of the present invention, and the first shooting unit 20a, the second shooting unit 20b, and the third shooting unit A moving image obtained by capturing a part of the work area R, which is captured by 20c, corresponds to the “second moving image” in the present invention. Further, the measurement unit 30 (fourth imaging unit) of the present embodiment corresponds to the "first imaging unit" of the present invention, and the first imaging unit 20a, the second imaging unit 20b, and the third imaging unit of the present embodiment. 20c corresponds to the "second imaging unit" in the present invention.

<Configuration of First Acquisition Unit>
The first acquisition unit 11 acquires, from the measurement unit 30, operation information that quantitatively indicates the operations of the first worker A1 and the second worker A2 performed in the work area R. The operation information acquired by the first acquisition unit 11 is transmitted to the storage unit 14 and stored as an operation information history 14 b. When the operation analysis system 100 includes a plurality of measurement units 30, the first acquisition unit 11 may acquire operation information from each of the plurality of measurement units 30, and it is determined from which measurement unit 30 the operation information is acquired. The operation information may be transmitted to the storage unit 14 by adding information to be identified. A specific example of the operation information will be described in detail later with reference to FIG. The movement information may be, for example, information obtained by measuring coordinate values of representative positions of the worker's body at one-second intervals.

<Configuration of Second Acquisition Unit>
The second acquiring unit 12 is a moving image in which the first worker A1 and the second worker A2 are performing the operation, the first imaging unit 20a, the second imaging unit 20b, the third imaging unit 20c, and the measuring unit 30 ( Acquired from the fourth imaging unit). The moving image acquired by the second acquisition unit 12 is transmitted to the storage unit 14 and stored as a moving image history 14 a. The second acquisition unit 12 may transmit the moving image to the storage unit 14 by adding information that identifies the moving image acquired from any of the plurality of imaging units.

<Configuration of Third Acquisition Unit>
The third acquisition unit 13 acquires, from the measurement unit 30, reference operation information that quantitatively indicates a reference operation serving as a comparison reference for the operation of the worker. The reference operation information acquired by the third acquisition unit 13 is transmitted to the storage unit 14 and stored as reference operation information 14c. Note that the third acquisition unit 13 acquires, as reference operation information, operation information not extracted by the extraction unit 15 described later among the already stored operation information history 14 b, and indicates that it is a reference operation. Information may be added and stored in the storage unit 14. Further, the third acquisition unit 13 acquires the operation information specified by the user among the already stored operation information history 14b as the reference operation information, and adds and stores information indicating that it is the reference operation. It may be stored in the unit 14. A specific example of the reference operation information will be described in detail later with reference to FIG. The reference operation information may be, for example, information obtained by measuring coordinate values of representative positions of the worker's body when the standard operation to be followed by the worker is performed at one second intervals.

<Configuration of Storage Unit>
The storage unit 14 stores at least reference operation information 14 c quantitatively indicating a reference operation which is a reference of comparison for the operation of the worker. In the operation analysis system 100 according to the present embodiment, the storage unit 14 stores a moving image history 14a, an operation information history 14b, a reference operation information 14c, and a correspondence table 14d. Among these, the correspondence table 14 d is used to estimate the position in the work area R in which the operation indicated by the operation information extracted by the extraction unit 15 is performed. The correspondence table 14d will be described later by showing a specific example.

<Configuration of Extraction Unit>
The extraction unit 15 compares the operation information measured by the measurement unit 30 with the reference operation information 14c, and extracts operation information that satisfies a predetermined condition. Here, the motion information and the reference motion information 14c may include the coordinate values of the worker's joints, respectively, and the extraction unit 15 may calculate the coordinate values included in the motion information and the coordinate values included in the reference motion information 14c. The operation information may be extracted to satisfy a predetermined condition by comparison. In this case, the predetermined condition may be a condition that the degree of deviation between the coordinate value included in the operation information and the coordinate value included in the reference operation information 14c is equal to or greater than a threshold.

Further, the extraction unit 15 may extract the operation information satisfying the predetermined condition by comparing the operation information measured by the measurement unit 30 with the reference operation information 14c determined for each of the plurality of steps. In this case, the extracted operation information corresponds to one process. Further, the extraction unit 15 may extract the operation information satisfying the predetermined condition by comparing the operation information measured by the measurement unit 30 with the reference operation information 14c determined for each of the plurality of element operations. . In this case, the extracted operation information corresponds to one element operation.

<Configuration of specific part>
The identifying unit 16 identifies a scene in which the operator is executing the operation indicated by the extracted operation information, using the moving image history 14a. The identifying unit 16 compares the time when the motion information is measured with the time when the moving image is shot, and in the moving picture history 14a, a scene of a time zone in which the operation indicated by the extracted action information is performed. You may specify. In addition, the identifying unit 16 may identify the worker for which the extracted operation information is measured, based on the information identifying the worker added to the extracted operation information. Furthermore, the specification unit 16 may specify a process indicated by the operation information extracted by the extraction unit 15 or may specify an element operation indicated by the operation information extracted by the extraction unit 15.

<Configuration of estimation unit>
The estimation unit 17 estimates the position in the work area R in which the operation indicated by the operation information extracted by the extraction unit 15 is performed. The estimation unit 17 may estimate the position in the work area R corresponding to the operation information extracted by the extraction unit 15 by referring to the correspondence table 14 d stored in the storage unit 14. In the motion analysis system 100 according to the present embodiment, the storage unit 14 indicates the first correspondence table D3 in which the correspondence between the process and the imaging unit is indicated, and the correspondence between the coordinate values of the joints of the worker and the imaging unit. A second correspondence table D4 may be stored, and a third correspondence table D6 in which the coordinate values of the joints of the worker and the correspondence between the element operation and the photographing unit are indicated. The identifying unit 16 may identify a moving image captured at a position estimated by the estimating unit 17 among the plurality of moving images.

The estimation unit 17 may estimate the position in the work area R where the worker is executing the operation indicated by the extracted operation information based on the process indicated by the operation information extracted by the extraction unit 15. In this case, the estimation unit 17 refers to the first correspondence table D3 in which the correspondence between the process and the imaging unit is indicated, and the position in the work area R where the operator performs the operation indicated by the extracted operation information. You may estimate

The estimation unit 17 may estimate the position in the work area R where the worker is executing the operation indicated by the extracted operation information based on the coordinate value included in the operation information extracted by the extraction unit 15 . In this case, the estimation unit 17 refers to the second correspondence table D4 in which the correspondence between the coordinate values of the joint of the worker and the imaging unit is indicated, and the worker executes the operation indicated by the extracted operation information. The position in the working area R may be estimated.

The estimation unit 17 is a task in which the worker is executing the operation indicated by the extracted operation information based on the coordinate value included in the operation information extracted by the extraction unit 15 and the element operation indicated by the extracted operation information. The position in the region R may be estimated. In this case, the estimation unit 17 refers to the third correspondence table D6 in which the coordinate value of the joint of the worker and the correspondence between the element operation and the imaging unit are displayed, and the worker performs the operation indicated by the extracted operation information. The position in the work area R being executed may be estimated.

Note that the motion analysis system 100 may evaluate the worker's tiredness and intimacy based on the frequency at which the worker performs a non-standard motion. The evaluation of fatigue and irritation may be performed, for example, by evaluating how often the frequency of occurrence of non-standard motion detected for a certain worker deviates from the average frequency of occurrence for all the workers. In addition, the motion analysis system 100 may adjust the work load of the worker based on the tiredness and inconsistencies of the evaluated worker. More specifically, adjustment is made to slow down the flow speed of the production line as fatigue and insecurity of the evaluated worker increase, thereby reducing the workload of the worker and preventing the generation of defective products. It may be

[Hardware configuration]
Next, an example of the hardware configuration of the operation analysis device 10 according to the present embodiment will be described using FIG. 3. The operation analysis device 10 includes a central processing unit (CPU) 10 a corresponding to an arithmetic device, a random access memory (RAM) 10 b corresponding to the storage unit 14, and a read only memory (ROM) 10 c corresponding to the storage unit 14. A communication unit 10d, an input unit 10e, and a display unit 10f are provided. Each of these configurations is mutually connected so as to be able to transmit and receive data via a bus. In addition, although the case where the motion analysis apparatus 10 is comprised by one computer is demonstrated in this example, the motion analysis apparatus 10 may be implement | achieved using several computers.

The CPU 10a is a control unit that performs control related to the execution of a program stored in the RAM 10b or the ROM 10c, and performs calculation and processing of data. The CPU 10a compares the operation information with the reference operation information to extract the operation information satisfying the predetermined condition, and identifies a part of the moving image that is executing the operation indicated by the extracted operation information (operation (operation) Is an arithmetic unit that executes an analysis program). The CPU 10a receives various input data from the input unit 10e and the communication unit 10d, and displays the calculation result of the input data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.

The RAM 10 b is a part of the storage unit 14 in which data can be rewritten, and may be formed of, for example, a semiconductor storage element. The RAM 10 b stores an operation analysis program executed by the CPU 10 a and data such as a moving image history 14 a, an operation information history 14 b, a reference operation information 14 c, and a correspondence table 14 d.

The ROM 10 c is one of the storage units 14 that can read data, and may be configured of, for example, a semiconductor storage element. The ROM 10 c stores, for example, an operation analysis program and data not to be rewritten.

The communication unit 10 d is an interface that connects the operation analysis device 10 to an external device. The communication unit 10d is connected to the first imaging unit 20a, the second imaging unit 20b, the third imaging unit 20c, and the measuring unit 30, for example, by a LAN (Local Area Network), and is connected to the first imaging unit 20a and the second imaging unit 20b. And, the moving image may be received from the third imaging unit 20c, and the moving image and operation information may be received from the measuring unit 30. The communication unit 10 d may be connected to the Internet to receive a moving image or receive operation information via the Internet. In addition, the communication unit 10d may transmit, to the external device, a portion of the captured moving image that is performing the operation indicated by the operation information extracted by the extraction unit 15 through the Internet.

The input unit 10 e receives an input of data from the user, and may include, for example, a keyboard, a mouse, and a touch panel.

The display unit 10 f visually displays the calculation result by the CPU 10 a, and may be configured of, for example, an LCD (Liquid Crystal Display). An example of the screen displayed on the display unit 10 f will be described in detail later.

The operation analysis program may be stored in a computer-readable storage medium such as the RAM 10 b or the ROM 10 c and provided, or may be provided via a communication network connected by the communication unit 10 d. In the operation analysis apparatus 10, the CPU 10a executes the operation analysis program to realize various operations described with reference to FIG. Note that these physical configurations are exemplifications and may not necessarily be independent configurations. For example, the operation analysis apparatus 10 may include an LSI (Large-Scale Integration) in which the CPU 10a, the RAM 10b, and the ROM 10c are integrated.

Section 3 Operation Example FIG. 4 is a view showing an example of operation information D1 measured by the operation analysis system 100 according to the present embodiment. The figure shows an example in which the coordinate values of joints of a plurality of workers are measured at intervals of one second by the measuring unit 30.

In the operation information D1 of this example, "time", "worker ID", "right hand (X coordinate)", "right hand (Y coordinate)", "right hand (Z coordinate)" and "left hand (X coordinate)" Items are shown as an example. Here, “time” indicates the date and time when the operation information was measured by the measurement unit 30, and for example, “2017/9/1 8:43:21” is shown in the second and sixth lines from the top. , And it is represented that it is operation information measured at 8:43:21 on September 1, 2017. Similarly, the third line from the top indicates “2017/9/1 8:43:22”, which is operation information measured at 8:43:22 on September 1, 2017 Is represented. In addition, the fourth line from the top indicates "2017/9/1 8:43:23", and it is operation information measured at 8:43:23 on September 1, 2017 Is represented. Note that the description “...” Shown in the fifth and seventh lines is an ellipsis indicating that a plurality of data are included.

The “worker ID” is information for identifying a plurality of workers whose operation information has been measured by the measurement unit 30. In the case of this example, the worker ID is “A1” for the second to fourth lines from the top and “A2” for the sixth line from the top. That is, the operation information on the second to fourth lines from the top is the operation information measured for the first worker A1, and the operation information on the sixth line from the top is the operation measured for the second worker A2 It is shown to be information.

The “right hand (X coordinate)” indicates the position on the X axis of the coordinate system with the measuring unit 30 as the origin, for the right hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the X coordinate value of the right hand is “463” at 8:43:21 for the first worker A1, “533” at 8:43:22, and 8:43:23. It is shown that it is "483" in second. From this, it can be read that the first worker has moved the right hand by 7 cm in the X-axis direction. Also, for the second worker A2, the X coordinate value of the right hand is shown to be “416” at 8:43:21.

The “right hand (Y coordinate)” indicates the position on the Y axis of the coordinate system with the measuring unit 30 as the origin, for the right hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the Y coordinate value of the right hand is “574” at 8:43:21 for the first worker A1, “977” at 8:43:22, and 8:43:23. It is shown that it is "830" in second. From this, it can be read that the first worker moved the right hand by about 40 cm in the Y-axis direction and returned the position by about 15 cm. In addition, for the second worker A2, the Y coordinate value of the right hand is shown to be “965” at 8:43:21.

The “right hand (Z coordinate)” indicates the position of the operator's right joint in the Z axis of the coordinate system with the measuring unit 30 as the origin. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the Z coordinate value of the right hand is “531” at 8:43:21 for the first worker A1, “341” at 8:43:22, and 8:43:23. It is shown that the second is "624". From this, it can be read that the first worker moved the right hand about 9 cm in the minus direction of the Z coordinate and moved the right hand about 28 cm in the plus direction of the Z coordinate. Also, for the second worker A2, the Z coordinate value of the right hand is shown to be “408” at 8:43:21.

The “left hand (X coordinate)” indicates the position on the X axis of the coordinate system with the measuring unit 30 as the origin with respect to the left hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the X coordinate value of the left hand is “327” at 8:43:21 for the first worker A1, “806” at 8:43:22, 8:43:23. It is shown that it is "652" in second. From this, it can be read that the first operator moved the left hand by 7 cm in the X-axis direction and immediately returned to the original position. In addition, for the second worker A2, the X coordinate value of the left hand is shown to be “374” at 8:43:21.

In this example, the description of coordinate values of other joints included in the movement information D1 is omitted, but the movement information D1 is a three-dimensional movement of each joint such as wrist, elbow, shoulder, head, waist, knee, ankle and so on. Coordinate values may be included. Further, in this example, the operation information D1 includes the operation information of two workers, but may include the operation information of three or more workers, and a plurality of measured by the plurality of measurement units 30 Operation information for each worker may be included.

FIG. 5 is a diagram showing an example of the first reference operation information D2 stored by the operation analysis system 100 according to the present embodiment. In the figure, reference operation information defined for each of a plurality of steps is shown, and an example is shown in which coordinate values of joints when an operator performs a standard operation are recorded at one second intervals.

In the first reference motion information D2 of this example, "elapsed time from start", "step", "right hand (X coordinate)", "right hand (Y coordinate)", "right hand (Z coordinate)" and "left hand ( Items such as X coordinate) are shown as an example. Here, the "elapsed time from the start" indicates the elapsed time from the start to the end of one process in seconds. For example, “00:00:00” is shown in the second line from the top, which indicates that the line indicates reference operation information at the start of the process. Similarly, the third line from the top indicates “00:00:01”, which indicates that it is reference operation information one second after the start of the process. Further, the fifth line from the top shows “00:00:00”, which indicates that it is a line indicating reference operation information at the start of a different process. Similarly, the sixth line from the top indicates “00:00:01”, which indicates that it is reference operation information one second after the start of the process. Note that the description “...” Shown in the fourth and seventh lines is an ellipsis indicating that a plurality of data are included.

The “step” is information indicating which step the reference operation information relates to. For example, in the second and third lines from the top, "1. Assembly" is shown, and it is indicated that the reference operation information of these lines relates to the assembly process. Similarly, in the fifth and sixth lines from the top, "5. package" is shown, and it is indicated that the reference operation information of these lines relates to the packing process.

The “right hand (X coordinate)” indicates the position on the X axis of the coordinate system with the measuring unit 30 as the origin, for the right hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right hand X coordinate value is shown to be "463" at the start and "533" after one second for the assembly process. Also, for the packaging process, it is shown that the standard right hand X coordinate value is “416” at the start and “796” after one second.

The “right hand (Y coordinate)” indicates the position on the Y axis of the coordinate system with the measuring unit 30 as the origin, for the right hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right hand Y coordinate value is shown to be "574" at the beginning and "977" after one second for the assembly process. Also, for the packaging process, it is shown that the standard right hand Y coordinate value is "965" at the beginning and "595" after one second.

The “right hand (Z coordinate)” indicates the position of the operator's right joint in the Z axis of the coordinate system with the measuring unit 30 as the origin. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right-hand Z coordinate value is shown to be "531" at the beginning and "341" after one second for the assembly process. Also, for the packaging process, the standard right-hand Z coordinate value is shown to be "408" at the start and "949" after one second.

The “left hand (X coordinate)” indicates the position on the X axis of the coordinate system with the measuring unit 30 as the origin with respect to the left hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard left hand X coordinate value is shown to be "327" at the beginning and "806" after one second for the assembly process. Also, for the packaging process, it is shown that the standard left hand X coordinate value is "374" at the start and "549" after one second.

In this example, the description of coordinate values of other joints included in the first reference motion information D2 is omitted, but the first reference motion information D2 includes a wrist, an elbow, a shoulder, a head, a hip, a knee, an ankle, etc. And three-dimensional coordinate values of representative positions of each joint and body. Further, in the present example, the first reference operation information D2 includes reference operation information of two steps, but may include reference operation information of three or more steps.

FIG. 6 is a diagram showing an example of the first correspondence table D3 stored by the behavior analysis system 100 according to the present embodiment. In the figure, the example which recorded the correspondence of several process and several imaging | photography part is shown.

In the first correspondence table D3 of this example, items such as "process" and "shooting unit" are shown as an example. In the first correspondence table D3, correspondences between a plurality of processes and a plurality of imaging units are shown in one-to-one correspondence. For example, it is shown that the corresponding imaging unit is the "first imaging unit" (the first imaging unit 20a) for the process "1. assembly", and the imaging corresponding to the process "5. packing" It is shown that the unit is a "third imaging unit" (third imaging unit 20c).

The first correspondence table D3 may indicate the correspondence between a plurality of processes and a plurality of imaging units in a one-to-many manner. For example, two or more imaging units may correspond to the assembly process. In that case, two or more imaging units in which the assembly process is taken at different angles and different distances may be shown.

FIG. 7 is a flowchart of a first example of the process of extracting operation information performed by the operation analysis system 100 according to the present embodiment. The first example of the process of extracting the operation information is a process of comparing the operation information D1 and the first reference operation information D2 and extracting the operation information satisfying the predetermined condition.

The motion analysis system 100 first calculates the difference between the coordinate value of each row included in the motion information D1 and the start coordinate value included in the reference motion information of a predetermined step of the first reference motion information D2 (S10) . Here, the start coordinate value is a coordinate value of each joint when “the elapsed time from the start” is “00:00:00”.

Next, the motion analysis system 100 specifies the row of the motion information D1 for which the difference between the coordinate values is equal to or less than the threshold as the start row of the predetermined process (S11). Here, whether or not the difference between coordinate values is equal to or less than the threshold may be determined based on whether the absolute value of the difference between coordinate values is equal to or less than the threshold. Such processing is necessary because it is not clear in advance which coordinate value of each row of the motion information D1 corresponds to.

Subsequently, the motion analysis system 100 calculates a difference between the coordinate value of each row included in the motion information D1 and the end coordinate value included in the reference motion information of the predetermined process in the first reference motion information D2 (S12). ). Here, the end coordinate value is the coordinate value of each joint shown in the last line of the predetermined process. The behavior analysis system 100 specifies the row of the behavior information for which the difference between the coordinate values is equal to or less than the threshold as the end row of the predetermined process (S13). Here, whether or not the difference between coordinate values is equal to or less than the threshold may be determined based on whether the absolute value of the difference between coordinate values is equal to or less than the threshold.

The motion analysis system 100 cuts out from the identified start row to the end row among the plurality of rows included in the motion information D1 (S14). At this time, the number of lines to be cut out does not necessarily match the number of lines of reference operation information of a predetermined process in the first reference operation information D2. Therefore, the motion analysis system 100 performs interpolation or thinning so that the number of rows of motion information extracted from the motion information D1 matches the number of rows of reference motion information of a predetermined process in the first reference motion information D2 (S15) ). As a matter of course, when the number of lines of motion information cut out from the motion information D1 is smaller than the number of lines of reference motion information of a predetermined step in the first reference motion information D2, interpolation is performed and motion information cut out from the motion information D1 If the number of rows is larger than the number of rows of reference operation information of a predetermined process in the first reference operation information D2, the number of rows may be reduced. In addition, although the method of interpolation or thinning-out is arbitrary, when performing interpolation, for example, you may interpolate by the average value of the line of back and front, and when thinning-out, the line with a comparatively small change of a coordinate value is comparatively It is good to thin out the

Next, the motion analysis system 100 calculates the degree of deviation between the coordinate value of the motion information interpolated or thinned out and the coordinate value of the reference motion information of the predetermined process (S16). Here, the degree of deviation is a numerical value indicating how much the operation of the operator deviates from the standard operation to be followed by the operator. The degree of divergence may be calculated for each joint, or may be calculated in a unit (such as the right hand) in which a plurality of joints are combined. For example, the degree of deviation for each joint is the coordinate value (x, y, z) of the motion information and the coordinate value (X, Y, Z) of the reference motion information of a predetermined process for the three-dimensional coordinate value of the joint at a certain time. ) And the square root to obtain the value ((x-X) ² + (y-Y) ² + (z-Z) ² ) ^1/2 , and this value is summed over all time You may calculate by doing. However, the degree of deviation may be calculated by another value.

If the degree of divergence is equal to or greater than the threshold (S17: YES), the behavior analysis system 100 extracts the behavior information from the identified start line to the end line (S18). The operation information to be extracted may be operation information in a state where interpolation or thinning is not performed. Here, when the degree of divergence is calculated for each joint, the threshold may also be set for each joint, and when any one of the degrees of divergence calculated for each joint is equal to or more than the threshold, the motion information is extracted The motion information may be extracted if the degree of divergence calculated for two or more joints is equal to or greater than a threshold.

After the motion information is extracted, a portion of the moving image, which is indicated by the extracted motion information, is displayed (S19). The moving image display process will be described in detail using the following figure. Finally, it is determined whether analysis has been completed for all steps included in the first reference operation information D2 (S20), and if analysis is completed for all steps (S20: YES), the first process of extracting operation information The example ends.

In this example, the predetermined process is performed by calculating the difference between the coordinate value of each row included in the operation information D1 and the start coordinate value included in the reference operation information of the predetermined process of the first reference operation information D2. The specified start line is specified, and the difference between the coordinate value of each row included in the operation information D1 and the end coordinate value included in the reference operation information of the predetermined step in the first reference operation information D2 is calculated. Although the end line of the process is specified, the movement analysis system 100 may cut out movement information by other methods. For example, when the work range is assigned for each process, or when the movement range of the worker is limited for each process, the motion analysis system 100 determines that the coordinate value representing the center of the worker's body is a predetermined range The section of the motion information contained in may be cut out as a section of motion information corresponding to a predetermined process.

FIG. 8 is a flowchart of a first example of the moving image display process executed by the behavior analysis system 100 according to the present embodiment. In the first example of the moving image display process, in the first example of the process of extracting the operation information, after the operation information is extracted, a process of displaying a portion of the moving image that is executing the operation indicated by the extracted operation information (S19 An example).

First, the motion analysis system 100 refers to the first correspondence table D3 to estimate the position at which the process indicated by the extracted motion information is performed (S190). For example, when the process indicated by the extracted operation information is a packing process, “5. packing” is identified in the “process” column of the first correspondence table D3, and the corresponding “imaging unit 20c” is ", To estimate the position at which the process is performed. The position may be estimated based on, for example, the correspondence between a plurality of imaging units and the position in the work area R in which the plurality of imaging units are installed, or the position may be estimated based on the specification of the imaging units. .

Next, the motion analysis system 100 is photographed at a position estimated for a first moving image, which is a moving image of the entire work area R photographed by the measurement unit 30 (fourth image pickup unit), and an area narrower than the work area R. A portion of a time zone in which the operation indicated by the extracted operation information is executed is displayed for the second moving image (S191). Of the first moving image and the second moving image, the portion of the time zone in which the operation indicated by the extracted operation information is being executed is based on the time of the start line and the end line of the extracted operation information. It is specified by cutting out the second moving image. For example, when the process indicated by the extracted operation information is a packing process, using the first moving image, which is a moving image of the entire work area R captured by the measuring unit 30 (fourth imaging unit), and the first correspondence table D3 For the second moving image which is a moving image of a region narrower than the work region R captured by the specified third imaging unit 20c, the behavior analysis system 100 determines the time based on the start row and the end row of the extracted operation information. The first moving image and the second moving image are cut out and displayed on the display unit 10 f.

Furthermore, the motion analysis system 100 displays the process indicated by the extracted motion information (S192), and displays the identification information of the worker for which the extracted motion information is measured (S193). For example, when the process indicated by the extracted operation information is a packing process, and the worker who has executed the process is the first worker A1, the operation analysis system 100 determines the first moving image and the second moving image that have been cut out. At the same time, the display unit 10f displays that the content of the moving image is the packing process and that the worker who executes the process is the first worker A1. In addition, the display of a process may be performed by the information which can identify a several process. For example, in the present example, a serial number is added to the process, and the number "5" may be displayed to represent the packing process. In addition, the display of the worker may be performed by information that can identify a plurality of workers. For example, in the present example, a worker ID may be added to a plurality of workers, and the ID “A1” may be displayed to represent the first worker A1. Thus, the first example of the moving image display process ends.

FIG. 9 is an example of the screen DP displayed by the motion analysis system 100 according to the present embodiment. In the first example of the moving image display process, the screen DP is a scene of a time zone during which the operator performs an operation indicated by the extracted operation information with respect to the first moving image and the second moving image shot at the estimated position. Is displayed (S191).

The screen DP includes a summary DP1, an entire moving picture DP2, and a hand moving picture DP3. The outline DP1 is information indicating an outline of the extracted operation information. In the outline DP1, the operation indicated by the extracted operation information is “non-standard operation”, and the start time is “2017/7/7 10: 10: 44.138”, that is, 10 am on July 7, 2017. 10 minutes 44.138 seconds and the end time is “2017/7/7 10: 12: 44.435”, that is, 10: 12: 44.435 seconds on July 7, 2017, and extracted The process indicated by the operation information is the "packaging process", and the required time is "5.5 s" (5.5 seconds). The outline DP1 may indicate not the name of the process indicated by the extracted operation information but information identifying the process indicated by the extracted operation information. For example, an ID such as a serial number assigned to each process may be indicated.

The whole moving image DP2 is a first moving image obtained by photographing the entire work area R by the measuring unit 30 (fourth image pickup unit), and it is "2017/7/7 10: 10: 44.138" at the lower right of the moving image screen. "It is shown. The entire moving image DP2 makes it possible to generally understand how a plurality of workers are performing an operation. Further, in the general moving image DP2, the positions of joints of a plurality of workers detected by the measurement unit 30 are indicated by a skeletal model. This makes it possible to confirm that the coordinates of joints of a plurality of workers are measured at appropriate positions.

In the case of this example, the hand moving image DP3 is a second moving image obtained by capturing an area narrower than the work area R by the third imaging unit 20c, and as the start time at the lower right of the moving image screen, “2017/7/7 10:10: 44.138 "is shown. In addition, “packaging process” and “worker ID: A1” are displayed on the moving image screen, and information identifying the process indicated by the extracted operation information and the worker whose extracted operation information was measured Identification information is displayed. The details of the operation actually performed by the worker can be confirmed by the hand moving image DP3 in which the worker's hand is closed up.

According to the motion analysis system 100 according to the present embodiment, the motion information satisfying the predetermined condition in comparison with the reference motion information is extracted from the plurality of pieces of motion information quantitatively indicating the motions of the plurality of workers. By displaying information for identifying the worker for whom the extracted motion information has been measured, it is possible to identify a worker who has executed a specific motion satisfying a predetermined condition among a plurality of workers. In the case of this example, it can be confirmed that the first worker A1 has performed an operation other than the standard.

In addition, by displaying a scene of a time zone in which the worker is executing the operation indicated by the extracted operation information in the first moving image obtained by photographing the work area R, the specific operation satisfying the predetermined condition is entirely performed. In the second moving image obtained by capturing an area narrower than the work area R, by displaying the scene of the time zone during which the operator is executing the operation indicated by the extracted operation information, It is possible to confirm the details of a specific operation that meets a predetermined condition.

By estimating the position in the work area R where the operation indicated by the extracted motion information is performed by the estimation unit 17, the position where the specific operation satisfying the predetermined condition is performed can be estimated. By displaying the second moving image captured at the estimated position among the plurality of second moving images captured at multiple positions in, it is possible to confirm the details of the operation performed at the estimated position. .

In addition, among the operation information quantitatively indicating the operation of the worker, the operation information satisfying the predetermined condition is extracted in comparison with the reference operation information determined for each of the plurality of steps, and the extracted operation information indicates By displaying the information identifying the process, it is possible to confirm in which process the specific operation satisfying the predetermined condition is the operation performed. In the case of this example, it can be confirmed that the non-standard operation has been performed in the packing process.

By estimating the position in the work area R executing the operation indicated by the extracted operation information based on the process indicated by the extracted operation information using the first correspondence table D 3 Among the plurality of second moving images shot at the position of the second moving image, the second moving image shot at the position at which the process indicated by the extracted operation information is performed can be displayed, and the details of the operation performed in the step Can be confirmed.

Also, by comparing the coordinate values of the joints included in the movement information with the coordinate values of the joints included in the reference movement information, it is possible to accurately evaluate how much the movement information deviates from the reference movement information. It is possible to properly extract motion information that satisfies a predetermined condition.

FIG. 10 is a diagram showing an example of the second correspondence table D4 stored by the behavior analysis system 100 according to the present embodiment. In the figure, the example which recorded the correspondence of the coordinate range of several joints and several imaging | photography part is shown.

In the second correspondence table D4 of this example, items such as “right hand range (X coordinate)”, “right hand range (Y coordinate)”, and “shooting unit” are shown as an example. In the second correspondence table D4, the correspondence relationship between the range of the three-dimensional coordinates of the plurality of joints and the plurality of imaging units is shown in one-to-one correspondence. For example, in the case of “600 to 700” for “right hand range (X coordinate)” and “200 to 300” for “right hand range (Y coordinate)”, the corresponding imaging unit is “first imaging unit” (first It is shown that the photographing unit 20a). In this example, although the range of the X coordinate and the Y coordinate in the coordinate range of the right hand is shown, the second correspondence table D4 includes a wrist, an elbow, a shoulder, a head, a waist, a knee, an ankle, etc. in one row. A range of three-dimensional coordinate values of each joint may be included.

The second correspondence table D4 may indicate the correspondence between the ranges of the three-dimensional coordinates of the plurality of joints and the plurality of imaging units in a one-to-many manner. For example, two or more imaging units may correspond to the range of three-dimensional coordinates of a plurality of joints. In that case, two or more imaging | photography parts which image | photographed an operator's operation | movement from different angles and different distances may be shown.

FIG. 11 is a flowchart of a second example of the moving image display process executed by the behavior analysis system 100 according to the present embodiment. The second example of the moving image display process is a process of displaying the portion of the moving image that is executing the operation indicated by the extracted operation information after the operation information is extracted in the first example of the operation information extraction process (S19 Another example of).

First, the motion analysis system 100 refers to the second correspondence table D4 to search for a row including the largest number of coordinate values included in the motion information extracted from the second correspondence table D4 (S195). For example, when the X coordinate of the right hand included in the extracted motion information is in the range of 550 to 650, the “right hand range (X coordinate)” shown in the second row from the top of the second correspondence table D4 is extracted It is identified as a line including the largest number of coordinate values included in the selected motion information. Similarly, with regard to the Y coordinate and Z coordinate of the right hand, and the three-dimensional coordinates of other joints, a row with the most overlap is searched as compared with the coordinate range shown in the second correspondence table D4.

Next, the motion analysis system 100 refers to the searched line to estimate the position at which the motion indicated by the extracted motion information is being executed (S196). For example, when the searched line is the second line from the top of the second correspondence table D4, the position at which the operation has been performed is estimated by the “first imaging unit 20a” that is the corresponding imaging unit. The position may be estimated based on, for example, the correspondence between a plurality of imaging units and the position in the work area R in which the plurality of imaging units are installed, or the position may be estimated based on the specification of the imaging units. .

The motion analysis system 100 includes a first moving image, which is a moving image of the entire work area R taken by the measurement unit 30 (fourth imaging unit), and a second moving image taken at a position estimated for an area narrower than the work area R. The portion of the time zone in which the operation indicated by the extracted operation information is executed is displayed (S 197). Of the first moving image and the second moving image, the portion of the time zone in which the operation indicated by the extracted operation information is being executed is based on the time of the start line and the end line of the extracted operation information. It is specified by cutting out the second moving image.

Furthermore, the motion analysis system 100 displays the identification information of the worker for which the extracted motion information has been measured (S198). For example, when the worker who has executed the operation indicated by the extracted operation information is the first worker A1, the operation analysis system 100 performs the operation to execute the operation together with the first moving image and the second moving image which have been cut out. The display unit 10f displays that the worker is the first worker A1. Thus, the second example of the moving image display process ends.

According to the motion analysis system 100 according to the present embodiment, based on the coordinate values of the joints included in the extracted motion information, the position in the work area where the worker is executing the motion indicated by the motion information is By estimating, the position where the motion indicated by the extracted motion information is performed can be accurately estimated.

FIG. 12 is a diagram showing an example of the second reference motion information D5 stored by the motion analysis system 100 according to the present embodiment. In the figure, reference motion information defined for each of a plurality of element motions is shown, and an example is shown in which coordinate values of joints when a worker performs a standard motion are recorded at one second intervals.

In the second reference motion information D5 of this example, "elapsed time from start", "element motion", "right hand (X coordinate)", "right hand (Y coordinate)", "right hand (Z coordinate)" and "left hand" An item such as (X coordinate) is shown as an example. Here, the "elapsed time from the start" indicates the elapsed time from the start to the end of one process in seconds. For example, the second line from the top is indicated as "00:00:00", which represents a line indicating reference operation information at the start of an element operation. Similarly, the third line from the top indicates “00:00:01”, which indicates that it is reference operation information one second after the start of the element operation. Further, the fifth line from the top indicates “00:00:00”, which represents a line indicating reference operation information at the start of the different element operation. Similarly, the sixth line from the top indicates "00:00:01", which indicates that it is reference operation information one second after the start of the element operation. Note that the description “...” Shown in the fourth and seventh lines is an ellipsis indicating that a plurality of data are included.

The “element operation” is information indicating which element operation the reference operation information relates to. For example, in the second and third lines from the top, "part pick" is indicated, and it is indicated that the reference operation information of these lines relates to picking of parts. Similarly, in the fifth and sixth lines from the top, "arrangement" is shown, and it is indicated that the reference operation information of these lines relates to the arrangement of parts.

The “right hand (X coordinate)” indicates the position of the operator's right joint in the X axis of the coordinate system with the start point as the origin. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right-handed X-coordinate value is shown to be "0" at the beginning and "533" one second after the element operation of the part pick. In addition, for the element operation of the arrangement, it is shown that the standard right hand X coordinate value is "0" at the start and "796" after one second.

The “right hand (Y coordinate)” indicates the position on the Y axis of the coordinate system with the start point as the origin for the right hand joint of the worker. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right-handed Y coordinate value is shown to be "0" at the beginning and "977" one second after the element operation of the part pick. Also, for the element operation of the arrangement, it is shown that the standard right hand Y coordinate value is "0" at the start and "595" after one second.

The “right hand (Z coordinate)” indicates the position of the operator's right joint in the Z axis of the coordinate system with the start point as the origin. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard right-hand Z coordinate value is shown to be "0" at the beginning and "341" after one second for the element operation of the part pick. Also, for the element operation of the arrangement, the standard right hand Z coordinate value is shown to be "0" at the start and "949" after one second.

The “left hand (X coordinate)” indicates the position of the operator's left joint in the X axis of the coordinate system with the start point as the origin. In addition, although a unit is arbitrary, it is mm (millimeter) in this example. In the case of this example, the standard left hand X coordinate value is shown to be "0" at the beginning and "806" one second after the element operation of the part pick. Also, for the element operation of the arrangement, it is shown that the standard left hand X coordinate value is "0" at the start and "549" after one second.

In this example, the description of the coordinate values of other joints included in the second reference motion information D5 is omitted, but the second reference motion information D5 includes a wrist, an elbow, a shoulder, a head, a hip, a knee, an ankle, etc. May include three-dimensional coordinate values of each joint. Also, in the present example, the second reference motion information D5 includes reference motion information of two element motions, but may include reference motion information of three or more element motions.

FIG. 13 is a flowchart of a second example of the process of extracting operation information performed by the operation analysis system 100 according to the present embodiment. The second example of the process of extracting the operation information is a process of comparing the operation information D1 and the second reference operation information D5 to extract the operation information satisfying the predetermined condition.

The motion analysis system 100 first calculates the speed of each joint based on the adjacent coordinate values included in the motion information D1 (S40). For example, in the case of coordinate values measured at one-second intervals, the joint velocity may be calculated by dividing the difference between the coordinate values of adjacent rows by one second.

Next, the motion analysis system 100 identifies a row in which the velocity of any joint is less than or equal to the threshold as a row in which element motion changes (S41). For example, when transitioning from an element motion of a part pick to an element motion of placing a part, the speed of a hand joint becomes almost zero when finishing picking a part and moving to placement, so the speed of any joint is thresholded The following line can be identified as a line in which element operation changes.

The motion analysis system 100 specifies all the lines where the element motion changes for the motion information D1, and then cuts out from the motion information D1 from the start line to the end line of one element motion (S42). At this time, the number of lines to be cut out does not necessarily match the number of lines of reference operation information of a predetermined element operation in the second reference operation information D5. Therefore, the motion analysis system 100 performs interpolation or thinning so that the number of rows of motion information extracted from the motion information D1 matches the number of rows of reference motion information of a predetermined element motion in the second reference motion information D5 ( S43).

Next, the motion analysis system 100 calculates the degree of deviation between the coordinate value of the motion information interpolated or thinned out and the coordinate value of the reference motion information of the predetermined element motion (S44). The degree of divergence may be calculated for each joint. The degree of divergence is the difference between the coordinate value (x, y, z) of the motion information and the coordinate value (X, Y, Z) of the reference motion information of a predetermined process for the three-dimensional coordinate value of the joint at a certain time. The square is taken and the square root is taken, the value of ((x-X) ² + (y-Y) ² + (z-Z) ² ) ^1/2 is obtained, and this value is calculated by summing over all time You may However, the degree of deviation may be calculated by another value.

If the divergence degree is equal to or more than the threshold (S45: YES), the behavior analysis system 100 extracts the behavior information from the identified start line to the end line (S46). The operation information to be extracted may be operation information in a state where interpolation or thinning is not performed. Here, when the degree of divergence is calculated for each joint, the threshold may also be set for each joint, and when any one of the degrees of divergence calculated for each joint is equal to or more than the threshold, the motion information is extracted The motion information may be extracted if the degree of divergence calculated for two or more joints is equal to or greater than a threshold.

After the motion analysis system 100 extracts the motion information, the motion analysis system 100 displays a portion of the moving image in which the motion indicated by the extracted motion information is being executed (S47). The moving image display process will be described in detail using the following figure. The motion analysis system 100 finally determines whether analysis of all element motions included in the second reference motion information D5 is completed (S48), and analysis of all element motions is completed (S48: YES), The second example of the operation information extraction process ends.

FIG. 14 is a flowchart of a third example of the moving image display process executed by the behavior analysis system 100 according to the present embodiment. The third example of the moving image display process is a process of displaying the portion of the moving image indicated by the extracted operation information that is executing the operation after the operation information is extracted in the second example of the operation information extraction process (S47 An example).

First, the motion analysis system 100 refers to the second correspondence table D4 to search for a row including the largest number of coordinate values included in the extracted motion information in the second correspondence table D4 (S470). For example, when the X coordinate of the right hand included in the extracted motion information is in the range of 550 to 650, the “right hand range (X coordinate)” shown in the second row from the top of the second correspondence table D4 is extracted It is identified as a line including the largest number of coordinate values included in the selected motion information. Similarly, with regard to the Y coordinate and Z coordinate of the right hand, and the three-dimensional coordinates of other joints, a row with the most overlap is searched as compared with the coordinate range shown in the second correspondence table D4.

Next, the motion analysis system 100 estimates the position at which the motion indicated by the extracted motion information is being executed, with reference to the searched line (S471). For example, when the searched line is the second line from the top of the second correspondence table D4, the position at which the operation has been performed is estimated by the “first imaging unit 20a” that is the corresponding imaging unit. The position may be estimated based on, for example, the correspondence between a plurality of imaging units and the position in the work area R in which the plurality of imaging units are installed, or the position may be estimated based on the specification of the imaging units. .

The motion analysis system 100 includes a first moving image, which is a moving image of the entire work area R taken by the measurement unit 30 (fourth imaging unit), and a second moving image taken at a position estimated for an area narrower than the work area R. The portion of the time zone in which the operation indicated by the extracted operation information is executed is displayed (S 472). Of the first moving image and the second moving image, the portion of the time zone in which the operation indicated by the extracted operation information is being executed is based on the time of the start line and the end line of the extracted operation information. It is specified by cutting out the second moving image.

Furthermore, the motion analysis system 100 displays the element motion indicated by the extracted motion information (S 473), and displays the identification information of the worker for which the extracted motion information is measured (S 474). For example, if the element motion indicated by the extracted motion information is a component pick and the worker who has executed the element motion is the first worker A1, the motion analysis system 100 determines that the first moving image and the second motion image have been cut out. In addition to the moving image, the display unit 10f displays that the content of the moving image is a component pick and that the worker who executes the element operation is the first worker A1. Note that the display of the element operation may be performed by information that can identify a plurality of element operations. In addition, the display of the worker may be performed by information that can identify a plurality of workers. Thus, the third example of the video display process ends.

According to the motion analysis system 100 according to the present embodiment, the worker is caused to exhibit the motion indicated by the extracted motion information based on the coordinate value of the joint included in the extracted motion information and the element motion indicated by the extracted motion information. Is estimated at the position where the element motion indicated by the extracted motion information is performed among the plurality of second moving images shot at the plurality of positions in the work area by estimating the position in the work area being executed by The second moving image can be displayed, and details of the element operation can be confirmed. In addition, by defining the reference operation information for each of a plurality of element operations, it is possible to easily check whether there is a leak occurring in the element operation being executed by the operator, and the operator can perform an appropriate operation. It is possible to reduce the burden of checking whether you are going.

FIG. 15 is a diagram showing an example of the third correspondence table D6 stored by the behavior analysis system 100 according to the present embodiment. In the figure, the example which recorded the correspondence of the coordinate range of several joints, several element operation | movement, and several imaging | photography part is shown.

In the third correspondence table D6 of this example, items such as “right hand range (X coordinate)”, “right hand range (Y coordinate)”, “element operation”, “shooting unit”, and “remarks” are shown as an example. In the third correspondence table D6, the correspondence relationship between the range of the three-dimensional coordinates of the plurality of joints and the plurality of element operations and the plurality of imaging units is shown in a one-on-one manner. For example, for “right hand range (X coordinate),” “600 to 700”, “200 right hand for“ right hand range (Y coordinate) ”and“ 200 to 300 ”, and corresponding element operation is“ part pick ” It is shown that the unit is a "first imaging unit" (first imaging unit 20a). Here, "Remarks" indicates "shooting from diagonally above". In addition, for “right hand range (X coordinate)”, “200 to 300” for “600 to 700” and “right hand range (Y coordinate)”, and the element operation is “arrangement”, the corresponding imaging unit Is shown to be the "second imaging unit" (second imaging unit 20b). Here, "Remarks" indicates that "shooting from right above". As described above, even when the range of the three-dimensional coordinates of the joint is the same, when the appropriate imaging direction and imaging distance differ depending on the element operation, the operator's operation details are further classified by further classifying according to the element operation It is possible to identify videos that are easy to check. In this example, the range of the X coordinate and the Y coordinate in the coordinate range of the right hand is shown, but the third correspondence table D6 includes the wrist, elbow, shoulder, head, hip, knee, ankle, etc. in one row. A range of three-dimensional coordinate values of each joint may be included.

The third correspondence table D6 may indicate the correspondence between the ranges of the three-dimensional coordinates of the plurality of joints and the plurality of element operations and the plurality of imaging units in a one-to-many manner. For example, two or more imaging units may correspond to the range of three-dimensional coordinates of a plurality of joints and element motion. In that case, two or more imaging | photography parts which image | photographed an operator's operation | movement from different angles and different distances may be shown.

FIG. 16 is a flowchart of a fourth example of the moving image display process executed by the behavior analysis system 100 according to the present embodiment. The fourth example of the moving picture display process is a process of displaying the portion of the moving picture that is executing the operation indicated by the extracted action information after the action information is extracted in the second example of the action information extraction process (S47 Another example of).

First, the motion analysis system 100 refers to the third correspondence table D6 to search for a row including the largest number of coordinate values included in the motion information extracted from the third correspondence table D6 (S475). For example, when the X coordinate of the right hand included in the extracted motion information is in the range of 550 to 650, “right hand range (X coordinate) shown in the second and third rows from the top of the third correspondence table D6 “Is identified as the line including the largest number of coordinate values included in the extracted motion information. Similarly, with regard to the Y coordinate and Z coordinate of the right hand, and the three-dimensional coordinates of other joints, a row with the most overlap is searched as compared with the coordinate range shown in the third correspondence table D6.

Next, the motion analysis system 100 specifies a row corresponding to the element motion indicated by the extracted motion information among the searched rows (S471). For example, when the second and third rows from the top of the third correspondence table D6 are searched as a row including the largest number of coordinate values included in the extracted operation information, an element operation indicated by the extracted operation information If is a component pick, the second line from the top of the third correspondence table D6 is specified as a line corresponding to the element operation indicated by the extracted operation information.

The motion analysis system 100 estimates the position at which the motion indicated by the extracted motion information is being executed, with reference to the identified line (S477). For example, when the specified line is the second line from the top of the third correspondence table D6, the “first imaging unit 20a” that is the corresponding imaging unit estimates the position at which the operation has been performed. The position may be estimated based on, for example, the correspondence between a plurality of imaging units and the position in the work area R in which the plurality of imaging units are installed, or the position may be estimated based on the specification of the imaging units. .

The motion analysis system 100 includes a first moving image, which is a moving image of the entire work area R taken by the measurement unit 30 (fourth imaging unit), and a second moving image taken at a position estimated for an area narrower than the work area R. The portion of the time zone in which the operation indicated by the extracted operation information is executed is displayed (S 478). Of the first moving image and the second moving image, the portion of the time zone in which the operation indicated by the extracted operation information is being executed is based on the time of the start line and the end line of the extracted operation information. It is specified by cutting out the second moving image.

Furthermore, the motion analysis system 100 displays the element motion indicated by the extracted motion information (S479), and displays the identification information of the worker for which the extracted motion information is measured (S4710). For example, if the element motion indicated by the extracted motion information is a component pick and the worker who has executed the element motion is the first worker A1, the motion analysis system 100 determines that the first moving image and the second motion image have been cut out. In addition to the moving image, the display unit 10f displays that the content of the moving image is a component pick and that the worker who executes the element operation is the first worker A1. Note that the display of the element operation may be performed by information that can identify a plurality of element operations. In addition, the display of the worker may be performed by information that can identify a plurality of workers. Thus, the fourth example of the video display process ends.

According to the motion analysis system 100 according to the present embodiment, the worker is caused to exhibit the motion indicated by the extracted motion information based on the coordinate value of the joint included in the extracted motion information and the element motion indicated by the extracted motion information. Is estimated at the position where the element motion indicated by the extracted motion information is performed among the plurality of second moving images shot at the plurality of positions in the work area by estimating the position in the work area being executed by The second moving image can be displayed, and details of the element operation can be confirmed.

4 Modified Example FIG. 17 is a flowchart of a process of selecting a display mode performed by the behavior analysis system 100 according to a modified example of the present embodiment. The motion analysis system 100 according to the present embodiment, as shown in FIG. 9, is an outline of the extracted motion information, a first moving image (whole moving image) obtained by capturing the entire work area R, and a second obtained by capturing the hand of the operator. Display videos (videos at hand). The behavior analysis system 100 according to the present modification can change the content displayed on the display unit 10 f according to the user's selection.

The motion analysis system 100 determines whether to display the reference motion together (S50). It may be specified based on the input by the user's input unit 10e whether or not the reference operation is displayed together. When the reference motion is displayed together (S50: YES), the first moving image is displayed, and a portion of the second moving image that is executing the motion indicated by the extracted motion information and the reference indicated by the reference motion information The moving image on which the operation is being performed is displayed (S51). Here, the moving image executing the reference operation indicated by the reference operation information may be one captured in advance and stored in the storage unit 14 or may not be extracted by the extraction unit 15 in the moving image history 14 a It may be selected from videos.

By displaying a portion of the video that is performing the motion indicated by the extracted motion information and the motion video that is performing the reference motion, the comparison between the specific motion and the reference motion satisfying the predetermined condition is performed. It becomes easy.

In addition, the behavior analysis system 100 determines whether to display a graph indicating the behavior information together (S52). It may be specified based on the input by the user's input unit 10e whether or not the graph indicating the operation information is displayed together. When displaying the graph which shows operation information together (S52: YES), while displaying the 1st animation, the scene where the worker is performing the operation which the extracted operation information shows among the 2nd animation, A graph showing the extracted operation information is displayed (S53). The graph showing the extracted motion information may be a graph of any aspect, but may be a graph showing coordinate values of each joint on the vertical axis and an elapsed time on the horizontal axis, for example.

By displaying a graph indicating the extracted motion information and a scene of a moving image in which the operator is executing the motion indicated by the extracted motion information, the scene in which the specific motion satisfying the predetermined condition is executed is displayed. It can be confirmed from different points of view, animation and graph.

In addition, the motion analysis system 100 determines whether to display a superimposed image (S54). Whether or not to display the superimposed image may be designated based on an input from the input unit 10 e of the user. When the superimposed image is displayed (S54: YES), the first moving image is displayed, and a plurality of frames included in the scene where the operator is executing the operation indicated by the extracted operation information in the second moving image are displayed. It is superimposed and displayed as a single image (S55). For the superimposed image, for example, a plurality of frames included in a scene where the operator is executing the motion indicated by the extracted motion information in the second moving image is thinned out and subjected to transmission processing, and synthesized into one image. You may generate by doing.

By superimposing a plurality of frames included in the scene where the operator is executing an operation indicated by the extracted motion information in the moving image and displaying it as a single image, a specific operation satisfying a predetermined condition is executed. You can see at a glance the whole of the scene taken.

When the reference motion is not displayed together (S50: NO), the graph showing the motion information is not displayed together (S52: NO), and the superimposed image is not displayed (S54: NO), the first moving image is displayed. The part which is performing the operation | movement which the extracted operation | movement information shows among 2nd moving images is displayed. An example of the screen in this case is the screen DP shown in FIG. Thus, the selection process of the display mode is completed.

FIG. 18 is an example of a screen DP displayed by the behavior analysis system 100 according to the present modification. The screen DP is an example in the case where a graph indicating operation information is displayed together (S53) in the display mode selection process.

The screen DP includes a graph DP4, an entire moving picture DP2, and a hand moving picture DP3. The graph DP4 of this example is a graph showing the extracted motion information, and the X coordinate value ("right hand X"), Y coordinate value ("right hand Y") and Z coordinate value ("right hand Z") of the right hand It is an operation | movement signal graph which is shown on a vertical axis | shaft and elapsed time is shown on a horizontal axis. In the graph DP4, the X coordinate value (“right hand X”) of the right hand is indicated by a solid line, the Y coordinate value (“right hand Y”) is indicated by a broken line, and the Z coordinate value (“right hand Z”) is indicated by an alternate long and short dash line . In the graph DP4, the time series change of the coordinate value of the right hand is represented for the period of the extracted operation information.

In addition, in the selection process of the display mode, when the reference motion is displayed together (S51), the motion indicated by the extracted motion information is executed for the first motion image and the second motion image captured at the estimated position. While displaying the part of the time slot | zone which is carrying out, the 1st moving image and 2nd moving image which are performing reference operation may be displayed. In that case, the first moving images may be adjacent to each other, and the second moving images may be displayed adjacent to each other to facilitate comparison.

Further, in the case of displaying the superimposed image (S55), the first moving image displays the portion of the time zone in which the operation indicated by the extracted operation information is displayed, and the first image captured at the estimated position. For two moving images, a plurality of frames may be superimposed and displayed as a single image.

The embodiments described above are for the purpose of facilitating the understanding of the present invention, and are not for the purpose of limiting the present invention. The elements included in the embodiment and the arrangement, the material, the conditions, the shape, the size, and the like of the elements are not limited to those illustrated, and can be changed as appropriate. Further, the configurations shown in the embodiments can be partially substituted or combined.

The motion analysis system 100 is not limited to one that measures motion information indicating a worker's motion performed in a certain work area of a manufacturing line. For example, the motion analysis system 100 measures motion information indicating a motion of a person performing sports, such as a swing of a golf club, or a motion of a person performing a performance using a body such as a theater, etc. To compare the motion information with the reference motion information indicating the motion to be followed by the person, and, for example, to extract motion information satisfying a predetermined condition such as deviation from the reference motion information being equal to or greater than a threshold value; You may display the scene of the moving image in which the person is performing the operation | movement which the extracted operation | movement information shows. The motion analysis system 100 can be used when a motion that serves as a model is defined, and stores motion information indicating a motion that serves as a model in the storage unit in advance as reference motion information, and the measured motion information By comparing with, it is possible to display a moving image scene in which an operation deviated from the example is performed.

In the present embodiment and the modification, the motion analysis system 100 includes a plurality of imaging units, and the operator executes an operation indicated by the extracted operation information using moving images captured by the plurality of imaging units. Although the case of displaying the scene being displayed is described, the motion analysis system may extract information other than the moving image.

FIG. 19 is a diagram showing functional blocks of a motion analysis system 100g according to another embodiment of the present disclosure. A motion analysis system 100g according to another embodiment includes a measurement unit 30 that measures motion information indicating a worker's motion performed in a certain work area, a first sensor 40a that senses a worker's motion to be executed, and The extraction unit 15 that extracts operation information that satisfies a predetermined condition by comparing the 2 sensors 40b and the third sensor 40c with the operation information and the reference operation information, a first sensor 40a, a second sensor 40b, and a third sensor The output unit 18 may output a scene in which the worker is performing the operation indicated by the extracted operation information using the sensing data sensed by 40c. That is, the motion analysis system 100g according to the other embodiment may have a configuration in which the imaging unit (that is, the image sensor) of the motion analysis system 100 according to the present embodiment is replaced with another sensor. As a sensor, although any sensor that can sense the operation of the worker can be used, for example, a proximity sensor or a distance measurement sensor can be used. The output unit 18 may output a scene in which the operator is executing the operation indicated by the extracted operation information to the display unit 10 f, but may output the scene to another analyzer.

The second acquisition unit 12 of the motion analysis device 10g included in the motion analysis system 100g may obtain sensing data obtained by sensing a motion performed by the worker. In addition, the storage unit 14 may store a sensing data history 14e which is a history of sensing data sensed by the first sensor 40a, the second sensor 40b, and the third sensor 40c for an operation performed by the worker. In addition, the motion analysis device 10g may have the same configuration as the motion analysis device 10 according to the present embodiment.

When various operations are performed on a manufacturing line, it may be desired to detect that an operator performs an operation other than the standard. For this purpose, it is conceivable to install a plurality of sensors in the manufacturing line to sense the operation of the worker.

However, if the number of sensing data to be confirmed increases as the number of sensors to be installed increases, or if the length of individual sensing data becomes long, an operation specific to any part of the recorded sensing data is performed. Even if the scene where the action was performed is recorded, it becomes difficult to understand when the specific action is performed, and it becomes difficult to extract the scene where the specific action is performed.

Therefore, the operation performed by the operator is sensed by a sensor, and the data sensed by the sensor is recorded by specifying the portion executing the operation indicated by the extracted operation information. A motion analysis system, a motion analysis method, and a motion analysis program can be provided that can extract a scene in which a specific motion is performed regardless of the number or length of sensing data. Here, the scene in which the specific operation is performed may be a scene of a moving image sensed by an image sensor, or may be a scene of sensing data sensed by another sensor.

Embodiments of the present invention may also be described as the following appendices. However, embodiments of the present invention are not limited to the modes described in the following appendices. In addition, the embodiments of the present invention may be in the form of replacing or combining the descriptions of the supplementary notes.

[Supplementary Note 1]
A measurement unit (30) for measuring operation information indicating an operation of one or more workers executed in a certain work area;
A shooting unit (20a, 20b, 20c, 30) for shooting a moving image including a scene in which the worker is executing the operation;
A storage unit (14) for storing reference operation information indicating a reference operation as a reference for comparison of the operation of the worker;
An extraction unit (15) for comparing the operation information and the reference operation information to extract operation information satisfying a predetermined condition;
A display unit (10f) for displaying a scene in which the worker is executing the operation indicated by the extracted operation information using the moving image;
Analysis system with

[Supplementary Note 2]
The measurement unit (30) measures a plurality of the operation information respectively indicating the operations of the plurality of the workers;
The display unit (10f) displays information for identifying the worker for which the extracted operation information has been measured.
The motion analysis system according to appendix 1.

[Supplementary Note 3]
The imaging unit (20a, 20b, 20c, 30) is configured to capture a first imaging unit (30) that captures a first moving image that captures the work area, and a second capturing that captures a portion of the work area 2 including an imaging unit (20a, 20b, 20c),
The display unit (10f) uses the first moving image and the second moving image to display a scene of a time zone in which the worker is executing the operation indicated by the extracted operation information, respectively.
The motion analysis system according to Appendix 1 or 2.

[Supplementary Note 4]
The second imaging unit (20a, 20b, 20c) captures a plurality of second moving images obtained by capturing a plurality of portions of the work area,
It further comprises an estimation unit (17) for estimating the position in the work area where the operation indicated by the extracted operation information has been executed.
The display unit (10f) displays the second moving image captured at an estimated position among the plurality of second moving images.
The motion analysis system according to appendix 3.

[Supplementary Note 5]
The reference operation information is determined for each of a plurality of steps, and the extraction unit (15) compares the operation information with the reference operation information for each of a plurality of steps to determine the predetermined condition. Extract the operation information to satisfy
The display unit (10f) displays information identifying a process indicated by the extracted operation information.
The motion analysis system according to appendix 4.

[Supplementary Note 6]
The estimation unit (17) estimates the position in the work area where the worker is executing the operation indicated by the extracted operation information, based on the process indicated by the extracted operation information.
The motion analysis system according to appendix 5.

[Supplementary Note 7]
The motion information and the reference motion information each include coordinate values of joints of the worker,
The extraction unit (15) compares the coordinate value included in the movement information with the coordinate value included in the reference movement information to extract movement information satisfying the predetermined condition.
The motion analysis system according to appendix 4.

[Supplementary Note 8]
The estimation unit (17) estimates a position in the work area where the worker is executing the operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information. ,
The motion analysis system according to appendix 7.

[Supplementary Note 9]
The reference operation information is determined for each of a plurality of element operations,
The estimation unit (17) is configured such that the worker indicates an operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information and the element operation indicated by the extracted operation information. Estimate the position in the working area being executed,
The motion analysis system according to appendix 7 or 8.

[Supplementary Note 10]
The display unit (10f) causes the worker to execute the reference operation indicated by the reference operation information and a scene in which the operator is executing the operation indicated by the operation information extracted using the moving image. To display videos that include
The motion analysis system according to any one of appendices 1 to 9.

[Supplementary Note 11]
The display unit (10f) displays a graph indicating the extracted operation information and a scene of the moving image in which the worker is executing the operation indicated by the operation information extracted from the moving image. ,
The operation analysis system according to any one of appendices 1 to 10.

[Supplementary Note 12]
The display unit (10f) superimposes a plurality of frames included in a scene in which the operator is executing an operation indicated by the operation information extracted from the moving image and displays the frame as a single image.
The motion analysis system according to any one of appendices 1 to 11.

[Supplementary Note 13]
A first acquisition unit (11) for acquiring operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit (12) for acquiring a moving image including a scene in which the worker is executing the operation;
A third acquisition unit (13) that acquires reference operation information indicating a reference operation that is a reference of comparison for the operation of the worker;
An extraction unit (15) for comparing the operation information and the reference operation information to extract operation information satisfying a predetermined condition;
A specifying unit (16) for specifying a scene in which the operator is executing the operation indicated by the extracted operation information using the moving image;
Operation analysis device comprising:

[Supplementary Note 14]
Obtaining operation information indicating an operation of one or more workers performed in a work area;
Acquiring a moving image including a scene in which the worker is performing the operation;
Obtaining reference operation information indicating a reference operation which is a reference of comparison for the operation of the worker;
Extracting the operation information satisfying a predetermined condition by comparing the operation information and the reference operation information;
Specifying the scene in which the operator is executing the operation indicated by the extracted operation information using the moving image;
Operation analysis method including:

[Supplementary Note 15]
A computing device provided in the motion analysis device,
A first acquisition unit (11) for acquiring operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit (12) for acquiring a moving image including a scene in which the worker is executing the operation;
A third acquisition unit (13) for acquiring reference operation information indicating a reference operation serving as a reference for comparison of the operation of the worker;
An extraction unit (15) for extracting the operation information satisfying a predetermined condition by comparing the operation information with the reference operation information; and the work indicated by the operation information extracted using the moving image Identifying part (16) for identifying the scene where the person is performing,
Operation analysis program to operate as.

[Supplementary Note 16]
A measurement unit (30) for measuring operation information indicating an operation of one or more workers executed in a certain work area;
A sensor (40a, 40b, 40c, 30) that senses the operation performed by the worker;
A storage unit (14) for storing reference operation information indicating a reference operation as a reference for comparison of the operation of the worker;
An extraction unit (15) for comparing the operation information and the reference operation information to extract operation information satisfying a predetermined condition;
An output unit (18) for outputting a scene in which the worker is performing an operation indicated by the extracted operation information using sensing data sensed by the sensor (40a, 40b, 40c, 30);
Analysis system with

[Supplementary Note 17]
The measurement unit (30) measures a plurality of the operation information respectively indicating the operations of the plurality of the workers;
The output unit (18) outputs information for identifying the worker for which the extracted operation information has been measured.
The motion analysis system according to appendix 16.

[Supplementary Note 18]
The sensor (40a, 40b, 40c, 30) is configured to sense a first sensor (30) for sensing a first sensing data sensing the work area, and a second sensing data for sensing a portion of the work area Including two sensors (40a, 40b, 40c),
The output unit (18) outputs, using the first sensing data and the second sensing data, a scene of a time zone in which the worker is executing the operation indicated by the extracted operation information, respectively.
The motion analysis system according to

appendix

16 or 17.

[Supplementary Note 19]
The second sensor (40a, 40b, 40c) senses a plurality of second sensing data obtained by sensing a plurality of portions of the work area, respectively.
It further comprises an estimation unit (17) for estimating the position in the work area where the operation indicated by the extracted operation information has been executed.
The output unit (18) outputs the second sensing data sensed at a position estimated among the plurality of second sensing data,
The motion analysis system according to appendix 18.

[Supplementary Note 20]
The reference operation information is determined for each of a plurality of steps,
The extraction unit (15) compares the operation information with the reference operation information for each of a plurality of steps, and extracts the operation information satisfying the predetermined condition,
The output unit (18) outputs information identifying a process indicated by the extracted operation information.
The motion analysis system according to appendix 19.

[Supplementary Note 21]
The estimation unit (17) estimates the position in the work area where the worker is executing the operation indicated by the extracted operation information, based on the process indicated by the extracted operation information.
The motion analysis system according to appendix 20.

[Supplementary Note 22]
The motion information and the reference motion information each include coordinate values of joints of the worker,
The extraction unit (15) compares the coordinate value included in the movement information with the coordinate value included in the reference movement information to extract movement information satisfying the predetermined condition.
The motion analysis system according to appendix 19.

[Supplementary Note 23]
The estimation unit (17) estimates a position in the work area where the worker is executing the operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information. ,
A motion analysis system according to appendix 22.

[Supplementary Note 24]
The reference operation information is determined for each of a plurality of element operations,
The estimation unit (17) is configured such that the worker indicates an operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information and the element operation indicated by the extracted operation information. Estimate the position in the working area being executed,
The motion analysis system according to appendix 22 or 23.

[Supplementary Note 25]
The output unit (18) causes the worker to execute the reference operation indicated by the reference operation information and a scene in which the operator indicates the operation indicated by the operation information extracted from the sensing data. Output sensing data including the scene being
The motion analysis system according to any one of appendices 16 to 24.

[Supplementary Note 26]
The output unit (18) outputs a graph indicating the extracted operation information and a scene in which the operator is executing the operation indicated by the operation information extracted from the sensing data.
The motion analysis system according to any one of appendices 16 to 25.

[Supplementary Note 27]
A first acquisition unit (11) for acquiring operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit (12) for acquiring sensing data obtained by sensing the operation performed by the worker;
A third acquisition unit (13) for acquiring reference operation information quantitatively indicating a reference operation serving as a reference for comparison of the operation of the worker;
An extraction unit (15) for comparing the operation information and the reference operation information to extract operation information satisfying a predetermined condition;
An output unit (18) for outputting a scene in which the worker is executing the operation indicated by the extracted operation information using the sensing data;
Operation analysis device comprising:

[Supplementary Note 28]
Obtaining operation information indicating an operation of one or more workers performed in a work area;
Obtaining sensing data obtained by sensing the operation performed by the worker;
Obtaining reference operation information indicating a reference operation which is a reference of comparison for the operation of the worker;
Extracting the operation information satisfying a predetermined condition by comparing the operation information and the reference operation information;
Outputting a scene in which the operator is executing an operation indicated by the extracted operation information using the sensing data;
Operation analysis method including:

[Supplementary Note 29]
A computing device provided in the motion analysis device,
A first acquisition unit (11) for acquiring operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit (12) for acquiring sensing data obtained by sensing the operation performed by the worker;
A third acquisition unit (13) for acquiring reference operation information indicating a reference operation serving as a reference for comparison of the operation of the worker;
An extraction unit (15) which compares the operation information with the reference operation information and extracts operation information satisfying a predetermined condition; and the operation indicated by the operation information extracted using the sensing data An output unit (18) for outputting a scene that the operator is executing;
Operation analysis program to operate as.

[Supplementary Note 30]
Measuring motion information indicative of the motion of one or more workers performed in a work area;
Shooting a moving image including a scene in which the worker is performing the operation;
The moving image in which the worker is executing the operation indicated by the operation information satisfying a predetermined condition by comparing the operation information with a reference operation information indicating a reference operation serving as a reference for comparison of the operation of the worker Displaying the scene of the
Operation analysis method including:

DESCRIPTION OF SYMBOLS 10 Operation analysis apparatus 10a CPU 10b RAM 10c ROM

10d Communication unit

10e input unit

10f display unit 10g Operation analysis apparatus 11 first acquisition unit 12 second Acquisition unit, 13 third acquisition unit, 14 storage unit, 14a moving picture history, 14b operation information history, 14c reference operation information, 14d correspondence table, 14e sensing data history, 15 extraction unit, 16 Identification unit 17 Estimation unit 18 Output unit 20a First imaging unit 20b Second imaging unit 20c Third imaging unit 30 Measurement unit 40a First sensor 40b Second sensor 40c: third sensor 100: motion analysis system 100g: motion analysis system D1: motion information D2: first reference motion information D3: first correspondence table D4: second correspondence table D5: second Group Operation information, D6 ... third correspondence table, DP ... screen, DP1 ... summary, DP2 ... first video, DP3 ... second video, DP4 ... graph, R ... work area

Claims

A measurement unit that measures operation information indicating an operation of one or more workers executed in a work area;
A shooting unit for shooting a moving image including a scene in which the worker is executing the operation;
A storage unit storing reference operation information indicating a reference operation which is a reference of comparison of the operation of the worker;
An extraction unit that compares the operation information with the reference operation information to extract operation information that satisfies a predetermined condition;
A display unit that displays a scene in which the worker is executing an operation indicated by the extracted operation information using the moving image;
Analysis system with
The measurement unit measures a plurality of pieces of operation information respectively indicating the operations of the plurality of workers.
The display unit displays information for identifying the worker for which the extracted operation information has been measured.
The motion analysis system according to claim 1.
The imaging unit includes a first imaging unit that captures a first moving image that captures the work area, and a second imaging unit that captures a second moving image that captures a portion of the work area.
The display unit displays a scene of a time zone during which the worker is executing the operation indicated by the extracted operation information using the first moving image and the second moving image.
The motion analysis system according to claim 1.
The second imaging unit captures a plurality of second moving images obtained by capturing a plurality of portions of the work area, respectively.
It further comprises an estimation unit for estimating the position in the work area where the operation indicated by the extracted operation information is performed,
The display unit displays the second moving image captured at an estimated position among the plurality of second moving images.
The motion analysis system according to claim 3.
The reference operation information is determined for each of a plurality of steps,
The extraction unit compares the operation information with the reference operation information for each of a plurality of steps, and extracts the operation information satisfying the predetermined condition,
The display unit displays information identifying a process indicated by the extracted operation information.
The motion analysis system according to claim 4.
The estimation unit estimates the position in the work area where the worker is executing the operation indicated by the extracted operation information, based on the process indicated by the extracted operation information.
The motion analysis system according to claim 5.
The motion information and the reference motion information each include coordinate values of joints of the worker,
The extraction unit compares the coordinate value included in the movement information with the coordinate value included in the reference movement information to extract movement information satisfying the predetermined condition.
The motion analysis system according to claim 4.
The estimation unit estimates a position in the work area where the worker is executing the operation indicated by the extracted operation information, based on the coordinate value included in the extracted operation information.
The motion analysis system according to claim 7.
The reference operation information is determined for each of a plurality of element operations,
The operator performs an operation indicated by the extracted operation information based on the coordinate value included in the extracted operation information and the element operation indicated by the extracted operation information. Estimate the position in the work area
The motion analysis system according to claim 7 or 8.
The display unit displays a scene in which the operator is executing an operation indicated by the operation information extracted using the moving image, and a scene in which the operator is executing the reference operation indicated by the reference operation information. Display videos, including
The motion analysis system according to any one of claims 1 to 9.
The display unit displays a graph indicating the extracted operation information, and a scene of the moving image in which the operator is executing the operation indicated by the extracted operation information.
The motion analysis system according to any one of claims 1 to 10.
The display unit superimposes a plurality of frames included in a scene in which the operator is executing an operation indicated by the operation information extracted from the moving image and displays the frame as a single image.
The motion analysis system according to any one of claims 1 to 11.
A first acquisition unit that acquires operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit that acquires a moving image including a scene in which the worker is performing the operation;
A third acquisition unit configured to acquire reference operation information indicating a reference operation serving as a reference for comparison of the operation of the worker;
An extraction unit that compares the operation information with the reference operation information to extract operation information that satisfies a predetermined condition;
An identifying unit that identifies a scene in which the worker is executing the operation indicated by the extracted operation information using the moving image;
Operation analysis device comprising:
Obtaining operation information indicating an operation of one or more workers performed in a work area;
Acquiring a moving image including a scene in which the worker is performing the operation;
Obtaining reference operation information indicating a reference operation which is a reference of comparison for the operation of the worker;
Extracting the operation information satisfying a predetermined condition by comparing the operation information and the reference operation information;
Specifying the scene in which the operator is executing the operation indicated by the extracted operation information using the moving image;
Operation analysis method including:
A computing device provided in the motion analysis device,
A first acquisition unit that acquires operation information indicating an operation of one or more workers executed in a certain work area;
A second acquisition unit that acquires a moving image including a scene in which the worker is performing the operation;
A third acquisition unit configured to acquire reference operation information indicating a reference operation serving as a reference for comparison of the operation of the worker;
An extraction unit that compares the operation information with the reference operation information and extracts operation information that satisfies a predetermined condition; Identifying section that identifies the scene
Analysis program for operating as.