WO2023181805A1 - Work analyzing system and work analyzing method - Google Patents

Work analyzing system and work analyzing method Download PDF

Info

Publication number
WO2023181805A1
WO2023181805A1 PCT/JP2023/007348 JP2023007348W WO2023181805A1 WO 2023181805 A1 WO2023181805 A1 WO 2023181805A1 JP 2023007348 W JP2023007348 W JP 2023007348W WO 2023181805 A1 WO2023181805 A1 WO 2023181805A1
Authority
WO
WIPO (PCT)
Prior art keywords
work
analysis result
period
worker
analysis
Prior art date
Application number
PCT/JP2023/007348
Other languages
French (fr)
Japanese (ja)
Inventor
達矢 石橋
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2023181805A1 publication Critical patent/WO2023181805A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/04—Manufacturing

Definitions

  • the present disclosure relates to a work analysis device and a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation.
  • the present disclosure provides a work analysis device and a work analysis method that allow a user to efficiently check the analysis results over a long period of time even when the analysis results of a worker's work situation extend over a long period of time.
  • the main purpose is to
  • a work analysis device of the present disclosure is a work analysis device in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the processor , based on the video, the target object handled by the worker is recognized, and the actions that constitute the worker's work are recognized, and the analysis result information includes the recognition result of the target object and the recognition result of the action. and the first analysis result display information that visualizes the analysis result information for the first period, and the first analysis result for the second period included in the first period. It is configured to output second analysis result display information in which the analysis result information is visualized by enlarging the time axis from the display information.
  • the work analysis method of the present disclosure is a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the method comprising: Based on the video, the target object handled by the worker is recognized, and the actions that constitute the worker's work are recognized, and the analysis result information including the recognition result of the target object and the recognition result of the action is obtained. and first analysis result display information that visualizes the analysis result information for a first period, and first analysis result display information for a second period included in the first period.
  • the configuration is such that second analysis result display information that visualizes the analysis result information by further expanding the time axis is output.
  • the user can check the work status of the worker over a long period of time from a bird's-eye view. Furthermore, by viewing the second analysis result display information, the user can check the work status of the worker in detail. Thereby, even if the analysis results of the worker's work situation extend over a long period of time, the user can efficiently check the analysis results over a long period of time.
  • a first invention made to solve the above problem is a work analysis device in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation. Based on the video, the processor recognizes a target object handled by a worker, recognizes actions that constitute the worker's work, and generates a recognition result of the target object and a recognition result of the action. and first analysis result display information that visualizes the analysis result information targeting a first period, and a second period included in the first period. and second analysis result display information in which the analysis result information is visualized by enlarging the time axis from the first analysis result display information.
  • the user can check the work status of the worker over a long period of time from a bird's-eye view. Furthermore, by viewing the second analysis result display information, the user can check the work status of the worker in detail. Thereby, even if the analysis results of the worker's work situation extend over a long period of time, the user can efficiently check the analysis results over a long period of time.
  • the processor outputs the video in a state where it is associated with at least one of the first and second analysis result display information so as to be playable.
  • the user can easily understand the work status of the worker by viewing the video while comparing the first analysis result display information with the video.
  • the video included in the second period may be played back by associating the video with the second analysis result display information that presents the detailed work status of the worker.
  • a timeline cursor indicating the current playback position may be displayed on the second analysis result display information.
  • the first period is the entire target period extracted from the entire period in which the worker's work was performed, and the second period is a part of the target period. A certain configuration is assumed.
  • the users can check the worker's work status from a bird's-eye view for the entire target period, as well as check the worker's work status in detail for a part of the target period.
  • the first period and the second period may be arbitrarily designated by a user's operation.
  • the processor sets the selected specified period to the second period in response to a user's operation to select one of the plurality of specified periods included in the first period. and outputs the second analysis result display information.
  • the analysis result information for each specified period can be easily confirmed using the second analysis result display information.
  • the prescribed period is a work cycle consisting of a predetermined series of actions, and depending on the user's operation to select a work cycle, a second analysis is performed with the selected work cycle as the second period.
  • Result display information may also be output.
  • the processor controls the specified second period in response to a user's operation to arbitrarily specify the second period on the time axis in the first analysis result display information.
  • the second analysis result display information targeted at is output.
  • the user can appropriately specify the second period of the second analysis result display information on the time axis of the first analysis result display information that covers a wide first period.
  • the second period may be designated by a slider bar that visualizes the time axis in the first analysis result display information.
  • the processor outputs at least one of the first and second analysis result display information including information regarding the abnormality detection result when detecting an abnormality regarding the worker's work situation. do.
  • the user can easily check any abnormality regarding the worker's work status.
  • abnormalities regarding excess or deficiency of actions in a work cycle consisting of a predetermined series of actions, the order of actions, and the time length of the work cycle may be detected.
  • the processor outputs the second analysis result display information including information regarding the abnormality detection result, and the second analysis result display information is used as display information regarding the abnormality detection result.
  • a configuration including at least one of highlighting a display column of the target object, displaying a mark indicating an action related to the abnormality, and displaying characters representing details of the abnormality.
  • the user can more easily confirm abnormalities related to the worker's work status.
  • the abnormality detection results are displayed for each action, allowing the user to confirm the abnormality situation in detail.
  • the processor outputs the first analysis result display information including information regarding the abnormality detection result, and the first analysis result display information is used as display information regarding the abnormality detection result.
  • the configuration includes a display of statistical information aggregated over a specified period included in the first period.
  • the user can more easily confirm abnormalities related to the worker's work status.
  • the first analysis result display information having a long time axis statistical information aggregated over a specified period is displayed, so that the user can easily check the abnormal situation.
  • the number of abnormality detections may be totaled for each of a plurality of work cycles included in the first period, and the number of abnormality detections for each work cycle may be presented.
  • a ninth invention is a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of a worker's work situation, the method comprising: based on the information, recognizing the target object handled by the worker and recognizing the actions that constitute the worker's work, and acquiring the analysis result information including the recognition result of the target object and the recognition result of the action; first analysis result display information that visualizes the analysis result information targeting a first period; and time from the first analysis result display information targeting a second period included in the first period. It is configured to output second analysis result display information that visualizes the analysis result information by enlarging the axis.
  • the user can efficiently confirm the analysis results over a long period of time.
  • FIG. 1 is an overall configuration diagram of a work analysis system according to this embodiment.
  • This system analyzes the work status of workers in factories and other places, and presents the analysis results to the user.
  • This system includes a camera 1, a recorder 2, a server 3 (work analysis device), and an administrator terminal 4.
  • the camera 1 photographs the worker working at his or her post.
  • the recorder 2 records the video output from the camera 1.
  • the server 3 acquires the video from the recorder 2, performs a work analysis to analyze the work situation of the worker based on the video, and outputs the analysis result.
  • the server 3 is installed in a facility (such as a factory) and performs work analysis on that facility. Further, the server 3 may be configured as a cloud computer and perform work analysis on a plurality of facilities.
  • the administrator terminal 4 is used by a system administrator or a work administrator (user), and is composed of a PC, a tablet terminal, or the like. At the administrator terminal 4, a system administrator performs various setting operations related to processing performed by the server 3. In addition, the manager terminal 4 displays the analysis results output from the server 3 for the work manager to view.
  • FIG. 2 is a block diagram showing a schematic configuration of the server 3. As shown in FIG.
  • the server 3 includes a video input section 11, a screen output section 12, a storage section 13, and a processor 14.
  • the video recorded by the recorder 2 is input to the video input unit 11.
  • the screen output unit 12 outputs the analysis result display screen generated by the processor 14, and the analysis result display screen (see FIG. 6) is displayed on the administrator terminal 4.
  • the storage unit 13 stores programs and the like executed by the processor 14. Furthermore, the storage unit 13 stores the video for analysis acquired from the recorder 2. The storage unit 13 also stores an object behavior recognition model (object behavior estimation model) used by the processor 14, an analysis result video generated by the processor 14, and analysis result information.
  • object behavior recognition model object behavior estimation model
  • the processor 14 performs various processes by executing programs stored in the storage unit 13. In this embodiment, the processor 14 performs analysis processing P1, analysis result visualization processing P2, and the like.
  • the processor 14 analyzes the work situation of the worker based on a video of the worker performing the work, and generates analysis result information.
  • the processor 14 visualizes the analysis result information obtained in the analysis process P1 and presents it to the user.
  • an analysis result display screen (see FIG. 6) is generated.
  • FIG. 3 is a block diagram showing the contents of the analysis process P1.
  • the processor 14 performs analysis processing P1.
  • the analysis process P1 includes a video acquisition process P11, an object behavior recognition process P12, an analysis result video generation process P13, a work cycle detection process P14, an abnormality detection process P15, and an output process P16.
  • the processor 14 acquires a video to be analyzed from the storage unit 13.
  • object behavior recognition processing P12 the processor 14 performs object recognition to recognize the type of object (person, part, and work tool) from the video to be analyzed, and also performs behavior recognition (behavior estimation) to recognize the type of person's behavior. )I do.
  • action recognition a person's actions are recognized based on the person's movement and the type of object the person has come into contact with.
  • the action ID given to the action, the type of object (target object) related to the action, and the start time and end time of the action are acquired as recognition result information.
  • an object behavior recognition model constructed by machine learning such as deep learning can be used.
  • the processor 14 In analysis result video generation processing P13, the processor 14 generates an analysis result video to which recognition result information is added. Specifically, a rectangular frame surrounding the object (person, part, work tool) recognized in the object action recognition process P12, a character representing the type of the recognized object, and a character representing the type of the recognized action. is superimposed on the original video to be analyzed to generate an analysis result video.
  • the processor 14 detects a work cycle (regular period) consisting of a predetermined series of actions from among the worker actions recognized in the object action recognition process P12.
  • a work cycle regular period
  • the cycle ID assigned to the work cycle and the start time and end time of the work cycle are acquired as work cycle detection result information.
  • the processor 14 compares the worker's work procedure and the normal work procedure to detect a work error (work abnormality). Specifically, the processor 14 detects a work error by comparing the actions performed by the worker with the actions included in the regular work procedure regarding the target work cycle.
  • Work errors include, for example, excessive action, insufficient action, and a change in the order of actions.
  • the state of excessive action is a state in which an action is performed redundantly at a timing that should not have been performed.
  • a state in which an action is insufficient is a state in which an action that should originally be performed is not performed.
  • the processor 14 compares the worker's working time regarding one work cycle with the standard time to detect an abnormality in the work cycle.
  • the work cycle is determined to be abnormal if the work cycle time is longer than the standard time by more than a predetermined limit, for example, when the line is temporarily stopped due to some kind of trouble. Further, for example, when the time of the work cycle is shorter than the standard time by more than a predetermined limit, as in the case where the work cycle is canceled midway, it is determined that the work cycle is abnormal.
  • the processor 14 outputs the information related to the worker's behavior (type of behavior, start time, end time, etc.) acquired in the object behavior recognition process P12, and the information related to the work cycle (start time, etc.) acquired in the work cycle detection process P14. time, end time, etc.), and information on the detection results of the abnormality detection process P15 (work abnormalities (work mistakes), work cycle abnormalities) as analysis result information.
  • Analysis result information is stored in the storage unit 13.
  • FIG. 4 is an explanatory diagram showing the status of the work cycle detection process P14.
  • a work cycle detection process P14 is performed to detect a work cycle consisting of a predetermined series of actions. Specifically, the processor 14 detects a trigger action that defines the start timing of a work cycle from among the recognized actions, sets the timing at which the trigger action is detected as the start timing of the work cycle, and sets the next trigger action. The timing at which the action immediately before the action is detected is the end timing of the work cycle.
  • FIG. 4 is an example of actions included in one work cycle.
  • the worker takes action to grasp the main body with his/her hands.
  • the worker takes action to grasp the driver with his/her hand.
  • the operator takes action to take out the screws from the parts box.
  • the operator takes action to tighten the screw.
  • the worker takes action to carry the main body.
  • the worker takes action to grasp the main body with his/her hands.
  • the action of the worker grasping the main body with his or her hand becomes a trigger action that defines the start timing of the work cycle, and the timing at which the trigger action is started is This is the start time of the work cycle. Further, the timing at which the action immediately before the next triggered action ends is the end timing of the work cycle.
  • action information including the action ID assigned to the action, the type of object (target object) related to the action, and the start time and end time of the action is obtained as recognition result information. be done.
  • work cycle detection result information work cycle information including a cycle ID assigned to the work cycle, and the start time and end time of the work cycle is acquired.
  • FIG. 5 is a block diagram showing the contents of the analysis result visualization process P2.
  • the processor 14 performs analysis result visualization processing P2.
  • the analysis result visualization process P2 includes a video acquisition process P21, an analysis result acquisition process P22, and a screen generation process P23.
  • the processor 14 acquires the analysis result video from the storage unit 13.
  • the rectangular frame surrounding the recognized object (person, part, work tool), characters representing the type of recognized object, and characters representing the type of recognized action are the objects of analysis. This is superimposed on the original video.
  • analysis result acquisition processing P22 the processor 14 acquires analysis result information from the storage unit 13.
  • Analysis result information includes information on worker behavior (type of behavior, start time, end time, etc.), information on work cycles (start time and end time, etc.), and information on abnormality detection results (work abnormality (work error)). ), work cycle abnormalities).
  • the processor 14 In the screen generation process P23, the processor 14 generates an analysis result display screen (see FIG. 6) that visualizes the analysis result information.
  • This analysis result display screen is output from the screen output unit 12 to the administrator terminal 4, and the analysis result display screen is displayed on the administrator terminal 4.
  • analysis result information is visualized and displayed on the analysis result display screen, and an analysis result video is also displayed.
  • FIG. 6 is an explanatory diagram showing an analysis result display screen.
  • FIG. 7 is an explanatory diagram showing the process selection section 102, target period specification section 103, work video display section 104, playback operation section 105, and work cycle selection section 106 of the analysis result display screen.
  • FIG. 8 is an explanatory diagram showing the detailed analysis result display section 107 of the analysis result display screen.
  • 9 and 10 are explanatory diagrams showing the main parts of the detailed analysis result display section 107 when a work error (abnormality) occurs.
  • FIG. 11 is an explanatory diagram showing another example of the detailed analysis result display section 107.
  • FIG. 12 is an explanatory diagram showing the overall analysis result display section 108 of the analysis result display screen.
  • FIG. 13 is an explanatory diagram showing another example of the overall analysis result display section 108.
  • an analysis result display screen 101 is displayed on the administrator terminal 4.
  • the analysis result display screen 101 includes a process selection section 102, a target period specification section 103, a work video display section 104, a playback operation section 105, a work cycle selection section 106, and a detailed analysis result display section 107 (second (a display section for displaying analysis result display information) and an overall analysis result display section 108 (a display section for first analysis result display information).
  • the process selection section 102 is provided with buttons 111 for each process (picking, pressing, welding, assembly, inspection).
  • buttons 111 for each process (picking, pressing, welding, assembly, inspection).
  • the user can input a date and time (start time and end time) in characters as information regarding the target period (first period) for which the analysis results are to be displayed.
  • the target period designation unit 103 allows the user to designate any period included in the period during which video is stored in the recorder 2, that is, the period during which work analysis results based on the video can be obtained.
  • the analysis result video 121 is displayed on the work video display section 104.
  • a rectangular frame 122 surrounding a recognized object (person, part, work tool), and characters 123 representing the type of the recognized object are displayed on a video of a worker performing a task.
  • characters 124 representing the type of recognized action are displayed in a superimposed manner.
  • a rectangular frame 122 and characters 123 representing a person, a rectangular frame 122 and characters 123 representing a driver, and characters 124 representing a screw tightening action are displayed.
  • the playback operation unit 105 is provided with a button 131 for both playback and pause, and a double speed button 132.
  • the work cycle selection section 106 is provided with buttons 141 for each work cycle (1 to 6), a previous skip button 142, and a next skip button 143.
  • buttons 141 for each work cycle (1 to 6)
  • a previous skip button 142 When the user selects a work cycle by operating the button 141, analysis results regarding the selected work cycle are displayed on the work video display section 104 and the detailed analysis result display section 107.
  • the previous skip button 142 When the user operates the previous skip button 142, the process advances to the beginning of the next work cycle.
  • the skip back button 143 the process returns to the beginning of the currently displayed work cycle.
  • the detailed analysis result display section 107 is provided with a first action list display section 201.
  • the first behavior list display section 201 visualizes and presents to the user short-term analysis results regarding the behavior status related to each object.
  • action display columns 202 are arranged vertically for each object related to the worker's actions. In this example, actions are displayed for each part (main body, part #1, part #2, and part #3) and work tools (driver, tweezers, and marker) as objects related to worker actions.
  • a column 202 is provided.
  • An action bar 203 is drawn in the action display column 202.
  • the action bar 203 is an image representing the period in which the action was performed. That is, in the action display field 202, the horizontal direction is a time axis representing time, and the action bar 203 is drawn in a range corresponding to the period in which the action was performed. As a result, the implementation status of actions for each type of object is expressed in chronological order.
  • the display form of the action bar 203 differs depending on the action type, specifically, the action of grasping a part or work tool in the hand, the action of tightening a screw with a screwdriver, etc.
  • the shaded pattern (drawing pattern) drawn on the action bar 203 differs depending on the action type.
  • the filling color of the action bar 203 may vary depending on the action type.
  • the analysis results for the entire period of the selected work cycle are displayed. Further, when an arbitrary period is specified in the overall analysis result display section 108, the analysis results for the specified period are displayed on the first action list display section 201.
  • a timeline cursor 204 indicating the current playback position (playback time) of the analysis result video 121 displayed on the work video display section 104 is displayed.
  • the timeline cursor 204 moves in the horizontal direction (time axis direction) as the playback time of the analysis result video 121 on the work video display section 104 advances.
  • the detailed analysis result display section 107 is provided with a time display section 205.
  • the time display section 205 displays the start time, end time, and time length of the range (display period) of the analysis results displayed on the first action list display section 201 .
  • the displayed time length is Represents the working time of the selected work cycle.
  • an alert mark 206 is displayed when a work error (abnormality) is detected.
  • the alert mark 206 is displayed at a position corresponding to the implementation timing of the action related to the work error in the action display field 202 regarding the object related to the detected work error. For example, if an action is insufficient, an alert mark 206 is displayed at the position where the action should be performed. Furthermore, if the action is redundant, an alert mark 206 is displayed at the position where the redundant action was performed.
  • the action bar 203 related to the work error may be highlighted by changing its display form.
  • the frame line of the action bar 203 may be drawn as a dotted line, or the transparency of the action bar 203 may be changed.
  • FIGS. 9(A-1) and (A-2) are examples of actions related to component #3.
  • the part handling work consists of the action of grasping the part by hand.
  • the action of grasping the part with the hand is performed one more time; This is detected as a work error (abnormality) and an alert mark 206 is displayed.
  • FIGS. 9(B-1) and (B-2) are examples of actions related to the driver.
  • the screw tightening work consists of the action of grasping the screwdriver by hand, the action of taking out the screw from the parts box, and the action of tightening the screw.
  • the action of taking out the screw from the parts box and the action of tightening the screw are Repeated 3 times.
  • the worker's work procedure shown in Figure 9 (B-2) compared to the regular work procedure shown in Figure 9 (B-1), there is one less action to take out the screw from the parts box, which is a work error. (Abnormality) is detected, and an alert mark 206 is displayed.
  • Figures 10 (A-1) and (A-2) are examples of actions related to markers.
  • the marking work consists of the action of grasping the marker with the hand, the action of opening the cap, the action of placing a mark, and the action of closing the cap.
  • the worker's work procedure shown in Figure 10 (A-2) there is a lack of action to close the cap compared to the normal work procedure shown in Figure 10 (A-1), and this is due to a work error (abnormality). ) is detected and an alert mark 206 is displayed.
  • an abnormality content display section 211 may be displayed.
  • the abnormality content display section 211 displays characters representing the content of the detected work error.
  • there is one excess action related to #3, one less screw tightening action related to the driver, and forgetting to close the cap as an action related to the marker. is displayed.
  • the overall analysis result display section 108 is provided with a second action list display section 301.
  • the second action list display section 301 visualizes and presents to the user long-term analysis results regarding the state of actions related to each object.
  • action display columns 302 are arranged vertically for each object related to the worker's actions.
  • an action bar 303 is drawn, similar to the first action list display section 201.
  • the analysis results for the period (target period) specified by the target period designation section 103 are displayed.
  • the overall analysis result display section 108 is provided with a slider bar 304.
  • the user can specify an arbitrary period for the analysis results to be displayed on the detailed analysis result display section 107 by operating the left and right sliders 305.
  • the slider 305 on the left side defines the start time
  • the slider 305 on the right side defines the end time.
  • the analysis results for the specified period are displayed in the detailed analysis result display section 107 in an enlarged state in the time axis direction.
  • the left and right sliders 305 move to correspond to the period (start time and end time) of the selected work cycle, and
  • the analysis result of the selected work cycle is displayed in the detailed analysis result display section 107.
  • a number mark 306 is displayed when a work error (abnormality) is detected.
  • the number of times mark 306 represents the number of times a work error (abnormality) is detected in one work cycle.
  • the number of times mark 306 is displayed in the display area of the work cycle in which the work error was detected. In this example, one work error is detected in the #1 work cycle, and three work mistakes are detected in the #4 work cycle.
  • an abnormality in a work cycle when an abnormality in a work cycle is detected, the work cycle in which the abnormality was detected is highlighted. Specifically, rectangular image 307 is displayed during the corresponding work cycle. The rectangular image 307 is highlighted with, for example, a filled color or a hatched pattern. In this example, an abnormality in work cycle #4 has been detected.
  • an abnormality in the work cycle specifically refers to a case where the time of the work cycle is longer than the standard time by more than a predetermined limit, or a case where the time of the work cycle is shorter than the standard time by more than a predetermined limit.
  • the period specified by the slider bar 304 that is, the period in which the analysis results were displayed in the detailed analysis result display section 107, is highlighted.
  • Ru Specifically, rectangular image 311 is displayed during the corresponding period.
  • the rectangular image 311 is highlighted with, for example, a filled color or a hatched pattern.
  • the rectangular image 311 is displayed in a different display form from the rectangular image 307 so that it can be distinguished from the rectangular image 307 that highlights the work cycle in which an abnormality has been detected. In this example, the #5 work cycle is selected, and the rectangular image 311 is displayed in the #5 work cycle.
  • the user can arbitrarily specify the period (target period, first period) of the analysis results to be displayed on the overall analysis result display section 108 using the target period specifying section 103. Further, the user can use the slider bar 304 to arbitrarily specify the period (a part of the target period, the second period) of the analysis results to be displayed on the detailed analysis result display section 107. Further, the user can use the work cycle selection unit 106 to display analysis results regarding one work cycle on the detailed analysis result display unit 107.
  • the first period targeted by the overall analysis result display section 108 is the entire target period extracted from the entire period in which the worker's work was performed, and the second period targeted by the detailed analysis result display section 107 is The period is part of the period of interest.
  • the detailed analysis result display section 107 and the overall analysis result display section 108 have different time axis scales, and some of the analysis results displayed on the overall analysis result display section 108 are enlarged in the time axis direction. It is displayed on the detailed analysis result display section 107.
  • the user first uses the work cycle selection unit 106 to display the analysis results for one work cycle on the detailed analysis result display unit 107 to view them, and if necessary, selects the slider bar 304.
  • analysis results in various ranges can be viewed.
  • the user can use the slider bar 304 to display only the analysis results of an arbitrary range of interest within one work cycle on the detailed analysis result display section 107.
  • the user can use the slider bar 304 to display analysis results in a range spanning two or more work cycles on the detailed analysis result display section 107.
  • the work analysis device and work analysis method according to the present disclosure have the effect that even when the analysis results of the work situation of a worker extend over a long period of time, the user can efficiently check the analysis results over a long period of time.
  • the present invention is useful as a work analysis device and a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation.

Abstract

[Problem] To enable a user to check, even when a result of analyzing a work situation of a worker extends over a long time, this analysis result extending over the long time efficiently. [Solution] A target object handled by a worker is recognized and an action that constitutes the work by the worker is recognized on the basis of a video acquired by capturing a scene of the work by the worker, analysis result information including the recognition result regarding the target object and the recognition result regarding the action is obtained, and a display portion (overall analysis result display portion 108) of first analysis result display information in which the analysis result information on a first period is visualized and a display portion (detailed analysis result display portion 107) of second analysis result display information in which the analysis result information on a second period included in the first period is visualized while the time axis is extended more than that of the first analysis result display information are outputted.

Description

作業分析装置および作業分析方法Work analysis device and work analysis method
　本開示は、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析装置および作業分析方法に関するものである。 The present disclosure relates to a work analysis device and a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation.
　工場では、作業者の作業状況を分析して、その分析結果に基づいて、作業の手順などを見直すことで、作業効率の向上を図ることができる。このため、作業者の作業状況に関する分析を効率的に行える技術が望まれる。さらに、作業者の作業状況に関する分析結果情報をわかりやすくユーザに提示できる技術が望まれる。 In factories, work efficiency can be improved by analyzing the work status of workers and reviewing work procedures based on the analysis results. Therefore, a technology that can efficiently analyze the work status of workers is desired. Furthermore, a technology is desired that can present analysis result information regarding the worker's work status to the user in an easy-to-understand manner.
　このような要望に関係する技術として、従来、作業者が作業している様子をカメラで撮影して、その映像から作業者の作業状況を分析して、工程ごとの作業の実施状況を可視化したガントチャートや、作業者の様子が撮影された映像を、ユーザに提示する技術が知られている（特許文献１参照）。 Conventionally, technology related to such requests is to record workers working with a camera, analyze the worker's work status from the video, and visualize the work implementation status for each process. 2. Description of the Related Art There is a known technique for presenting a Gantt chart or a video of a worker to a user (see Patent Document 1).
特開２０１９－１６２２６号公報JP2019-16226A
　従来の技術では、ユーザが、ガントチャートと映像とを比較しながら閲覧することで、各工程における作業者の作業状況を比較的容易に把握することができる。一方、作業者の様子が撮影された映像が長時間に及ぶ場合、作業者の作業状況を分析した分析結果も長時間に及ぶ。このため、長時間に及ぶ分析結果を効率よく確認できることが望まれる。しかしながら、従来の技術は、このような要望に対して何ら配慮されないものであった。 In the conventional technology, a user can comparatively easily understand the work status of a worker in each process by viewing the Gantt chart and the video while comparing them. On the other hand, if the video of the worker's state is taken over a long period of time, the analysis result of the worker's work situation will also be taken over a long period of time. For this reason, it is desirable to be able to efficiently confirm analysis results over a long period of time. However, conventional techniques have not taken any consideration to such demands.
　そこで、本開示は、作業者の作業状況を分析した分析結果が長時間に及ぶ場合でも、その長時間に及ぶ分析結果をユーザが効率よく確認できるようになる作業分析装置および作業分析方法を提供することを主な目的とする。 Therefore, the present disclosure provides a work analysis device and a work analysis method that allow a user to efficiently check the analysis results over a long period of time even when the analysis results of a worker's work situation extend over a long period of time. The main purpose is to
　本開示の作業分析装置は、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析装置であって、前記プロセッサは、前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力する構成とする。 A work analysis device of the present disclosure is a work analysis device in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the processor , based on the video, the target object handled by the worker is recognized, and the actions that constitute the worker's work are recognized, and the analysis result information includes the recognition result of the target object and the recognition result of the action. and the first analysis result display information that visualizes the analysis result information for the first period, and the first analysis result for the second period included in the first period. It is configured to output second analysis result display information in which the analysis result information is visualized by enlarging the time axis from the display information.
　また、本開示の作業分析方法は、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析方法であって、前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力する構成とする。 Further, the work analysis method of the present disclosure is a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the method comprising: Based on the video, the target object handled by the worker is recognized, and the actions that constitute the worker's work are recognized, and the analysis result information including the recognition result of the target object and the recognition result of the action is obtained. and first analysis result display information that visualizes the analysis result information for a first period, and first analysis result display information for a second period included in the first period. The configuration is such that second analysis result display information that visualizes the analysis result information by further expanding the time axis is output.
　本開示によれば、ユーザが、第１の分析結果表示情報を閲覧することで、長い期間を対象にした作業者の作業状況を俯瞰的に確認することができる。また、ユーザが、第２の分析結果表示情報を閲覧することで、作業者の作業状況を詳細に確認することができる。これにより、作業者の作業状況を分析した分析結果が長時間に及ぶ場合でも、その長時間に及ぶ分析結果をユーザが効率よく確認できるようになる。 According to the present disclosure, by viewing the first analysis result display information, the user can check the work status of the worker over a long period of time from a bird's-eye view. Furthermore, by viewing the second analysis result display information, the user can check the work status of the worker in detail. Thereby, even if the analysis results of the worker's work situation extend over a long period of time, the user can efficiently check the analysis results over a long period of time.
本実施形態に係る作業分析システムの全体構成図Overall configuration diagram of the work analysis system according to this embodiment サーバの概略構成を示すブロック図Block diagram showing the schematic configuration of the server サーバで行われる分析処理の内容を示すブロック図Block diagram showing the content of analysis processing performed on the server サーバで行われる作業サイクル検出処理の状況を示す説明図Explanatory diagram showing the status of work cycle detection processing performed on the server サーバで行われる分析結果可視化処理の内容を示すブロック図Block diagram showing the content of analysis result visualization processing performed on the server 管理者端末に表示される分析結果表示画面を示す説明図Explanatory diagram showing the analysis results display screen displayed on the administrator terminal 分析結果表示画面の工程選択部、対象期間指定部、作業映像表示部、再生操作部、および作業サイクル選択部を示す説明図Explanatory diagram showing the process selection section, target period specification section, work video display section, playback operation section, and work cycle selection section of the analysis result display screen 分析結果表示画面の詳細分析結果表示部を示す説明図Explanatory diagram showing the detailed analysis result display section of the analysis result display screen 作業ミス（異常）が発生した場合における詳細分析結果表示部の要部を示す説明図Explanatory diagram showing the main parts of the detailed analysis result display section when a work error (abnormality) occurs 作業ミス（異常）が発生した場合における詳細分析結果表示部の要部を示す説明図Explanatory diagram showing the main parts of the detailed analysis result display section when a work error (abnormality) occurs 詳細分析結果表示部の別例を示す説明図Explanatory diagram showing another example of the detailed analysis result display section 分析結果表示画面の全体分析結果表示部を示す説明図Explanatory diagram showing the overall analysis result display section of the analysis result display screen 全体分析結果表示部の別例を示す説明図Explanatory diagram showing another example of the overall analysis result display section
　前記課題を解決するためになされた第１の発明は、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析装置であって、前記プロセッサは、前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力する構成とする。 A first invention made to solve the above problem is a work analysis device in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation. Based on the video, the processor recognizes a target object handled by a worker, recognizes actions that constitute the worker's work, and generates a recognition result of the target object and a recognition result of the action. and first analysis result display information that visualizes the analysis result information targeting a first period, and a second period included in the first period. and second analysis result display information in which the analysis result information is visualized by enlarging the time axis from the first analysis result display information.
　これによると、ユーザが、第１の分析結果表示情報を閲覧することで、長い期間を対象にした作業者の作業状況を俯瞰的に確認することができる。また、ユーザが、第２の分析結果表示情報を閲覧することで、作業者の作業状況を詳細に確認することができる。これにより、作業者の作業状況を分析した分析結果が長時間に及ぶ場合でも、その長時間に及ぶ分析結果をユーザが効率よく確認できるようになる。 According to this, by viewing the first analysis result display information, the user can check the work status of the worker over a long period of time from a bird's-eye view. Furthermore, by viewing the second analysis result display information, the user can check the work status of the worker in detail. Thereby, even if the analysis results of the worker's work situation extend over a long period of time, the user can efficiently check the analysis results over a long period of time.
　また、第２の発明は、前記プロセッサは、前記映像を、前記第１および第２の分析結果表示情報の少なくとも一方に対応付けられた状態で再生可能に出力する構成とする。 Further, in a second aspect of the present invention, the processor outputs the video in a state where it is associated with at least one of the first and second analysis result display information so as to be playable.
　これによると、ユーザが、第１の分析結果表示情報と映像とを比較しながら閲覧することで、作業者の作業状況を容易に把握することができる。この場合、作業者の詳細な作業状況を提示する第２の分析結果表示情報に映像が対応付けられることで、第２の期間に含まれる映像が再生されるようにしてもよい。また、例えば、第２の分析結果表示情報上に、現在の再生位置を示すタイムラインカーソルが表示されるようにしてもよい。 According to this, the user can easily understand the work status of the worker by viewing the video while comparing the first analysis result display information with the video. In this case, the video included in the second period may be played back by associating the video with the second analysis result display information that presents the detailed work status of the worker. Further, for example, a timeline cursor indicating the current playback position may be displayed on the second analysis result display information.
　また、第３の発明は、前記第１の期間は、作業者の作業が実施された全期間から抽出された対象期間の全体であり、前記第２の期間は、前記対象期間の一部である構成とする。 Further, in a third invention, the first period is the entire target period extracted from the entire period in which the worker's work was performed, and the second period is a part of the target period. A certain configuration is assumed.
　これによると、ユーザが、対象期間の全体を対象にして作業者の作業状況を俯瞰的に確認すると共に、対象期間の一部を対象にして作業者の作業状況を詳細に確認することができる。この場合、第１の期間および第２の期間は、ユーザの操作により任意に指定されるものとしてもよい。 According to this, users can check the worker's work status from a bird's-eye view for the entire target period, as well as check the worker's work status in detail for a part of the target period. . In this case, the first period and the second period may be arbitrarily designated by a user's operation.
　また、第４の発明は、前記プロセッサは、前記第１の期間に含まれる複数の規定期間のいずれかを選択するユーザの操作に応じて、選択された規定期間を前記第２の期間に設定して前記第２の分析結果表示情報を出力する構成とする。 Further, in the fourth invention, the processor sets the selected specified period to the second period in response to a user's operation to select one of the plurality of specified periods included in the first period. and outputs the second analysis result display information.
　これによると、規定期間ごとの分析結果情報を第２の分析結果表示情報により容易に確認することができる。この場合、例えば、規定期間は、所定の一連の行動で構成される作業サイクルであり、作業サイクルを選択するユーザの操作に応じて、選択された作業サイクルを第２の期間として第２の分析結果表示情報が出力されるものとしてもよい。 According to this, the analysis result information for each specified period can be easily confirmed using the second analysis result display information. In this case, for example, the prescribed period is a work cycle consisting of a predetermined series of actions, and depending on the user's operation to select a work cycle, a second analysis is performed with the selected work cycle as the second period. Result display information may also be output.
　また、第５の発明は、前記プロセッサは、前記第１の分析結果表示情報における時間軸上で前記第２の期間を任意に指定するユーザの操作に応じて、指定された前記第２の期間を対象にした前記第２の分析結果表示情報を出力する構成とする。 Further, in a fifth aspect of the invention, the processor controls the specified second period in response to a user's operation to arbitrarily specify the second period on the time axis in the first analysis result display information. The second analysis result display information targeted at is output.
　これによると、ユーザが、広い第１の期間を対象にした第１の分析結果表示情報における時間軸上で、第２の分析結果表示情報の第２の期間を適切に指定することができる。この場合、例えば、第１の分析結果表示情報における時間軸を可視化したスライダーバーで第２の期間が指定されるものとしてもよい。 According to this, the user can appropriately specify the second period of the second analysis result display information on the time axis of the first analysis result display information that covers a wide first period. In this case, for example, the second period may be designated by a slider bar that visualizes the time axis in the first analysis result display information.
　また、第６の発明は、前記プロセッサは、作業者の作業状況に関する異常を検知すると、その異常検知結果に関する情報を含む前記第１および第２の分析結果表示情報の少なくとも一方を出力する構成とする。 In a sixth aspect of the invention, the processor outputs at least one of the first and second analysis result display information including information regarding the abnormality detection result when detecting an abnormality regarding the worker's work situation. do.
　これによると、ユーザが、作業者の作業状況に関する異常を容易に確認することができる。この場合、例えば、所定の一連の行動で構成される作業サイクルにおける行動の過不足、行動の順番、および作業サイクルの時間長に関する異常が検知されるものとしてもよい。 According to this, the user can easily check any abnormality regarding the worker's work status. In this case, for example, abnormalities regarding excess or deficiency of actions in a work cycle consisting of a predetermined series of actions, the order of actions, and the time length of the work cycle may be detected.
　また、第７の発明は、前記プロセッサが、前記異常検知結果に関する情報を含む前記第２の分析結果表示情報を出力し、前記第２の分析結果表示情報は、前記異常検知結果に関する表示情報として、前記対象物体の表示欄の強調表示と、前記異常に関係する行動を示すマークの表示と、前記異常の内容を表す文字の表示との少なくともいずれかを含む構成とする。 Further, in a seventh invention, the processor outputs the second analysis result display information including information regarding the abnormality detection result, and the second analysis result display information is used as display information regarding the abnormality detection result. , a configuration including at least one of highlighting a display column of the target object, displaying a mark indicating an action related to the abnormality, and displaying characters representing details of the abnormality.
　これによると、ユーザが、作業者の作業状況に関する異常をより一層容易に確認することができる。特に、時間軸が短時間になる第２の分析結果表示情報では、行動単位で異常検知結果が表示されることで、ユーザが、異常の状況を詳細に確認することができる。 According to this, the user can more easily confirm abnormalities related to the worker's work status. In particular, in the second analysis result display information where the time axis is short, the abnormality detection results are displayed for each action, allowing the user to confirm the abnormality situation in detail.
　また、第８の発明は、前記プロセッサが、前記異常検知結果に関する情報を含む前記第１の分析結果表示情報を出力し、前記第１の分析結果表示情報は、前記異常検知結果に関する表示情報として、前記第１の期間に含まれる規定期間で集計された統計情報の表示を含む構成とする。 Further, in an eighth aspect, the processor outputs the first analysis result display information including information regarding the abnormality detection result, and the first analysis result display information is used as display information regarding the abnormality detection result. , the configuration includes a display of statistical information aggregated over a specified period included in the first period.
　これによると、ユーザが、作業者の作業状況に関する異常をより一層容易に確認することができる。特に、時間軸が長時間になる第１の分析結果表示情報では、規定期間で集計された統計情報が表示されることで、ユーザが、異常の状況を容易に確認することができる。例えば、第１の期間に含まれる複数の作業サイクルごとに異常の検知回数が集計されて、作業サイクルごとの異常の検知回数が提示されるものとしてもよい。 According to this, the user can more easily confirm abnormalities related to the worker's work status. In particular, in the first analysis result display information having a long time axis, statistical information aggregated over a specified period is displayed, so that the user can easily check the abnormal situation. For example, the number of abnormality detections may be totaled for each of a plurality of work cycles included in the first period, and the number of abnormality detections for each work cycle may be presented.
　また、第９の発明は、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析方法であって、前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力する構成とする。 Further, a ninth invention is a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of a worker's work situation, the method comprising: based on the information, recognizing the target object handled by the worker and recognizing the actions that constitute the worker's work, and acquiring the analysis result information including the recognition result of the target object and the recognition result of the action; first analysis result display information that visualizes the analysis result information targeting a first period; and time from the first analysis result display information targeting a second period included in the first period. It is configured to output second analysis result display information that visualizes the analysis result information by enlarging the axis.
　これによると、第１の発明と同様に、作業者の作業状況を分析した分析結果が長時間に及ぶ場合でも、その長時間に及ぶ分析結果をユーザが効率よく確認できるようになる。 According to this, as in the first invention, even if the analysis results of the worker's work situation extend over a long period of time, the user can efficiently confirm the analysis results over a long period of time.
　以下、本開示の実施の形態を、図面を参照しながら説明する。 Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
　図１は、本実施形態に係る作業分析システムの全体構成図である。 FIG. 1 is an overall configuration diagram of a work analysis system according to this embodiment.
　本システムは、工場などにおける作業者の作業状況を分析して、その分析結果をユーザに提示するものである。本システムは、カメラ１と、レコーダ２と、サーバ３（作業分析装置）と、管理者端末４とを備えている。 This system analyzes the work status of workers in factories and other places, and presents the analysis results to the user. This system includes a camera 1, a recorder 2, a server 3 (work analysis device), and an administrator terminal 4.
　カメラ１は、作業者が持ち場で作業を行う様子を撮影する。 The camera 1 photographs the worker working at his or her post.
　レコーダ２は、カメラ１から出力される映像を録画する。 The recorder 2 records the video output from the camera 1.
　サーバ３は、レコーダ２から映像を取得して、その映像に基づいて、作業者の作業状況を分析する作業分析を行い、その分析結果を出力する。サーバ３は、施設（工場など）に設置され、その施設を対象にして作業分析を行う。また、サーバ３は、クラウドコンピュータとして構成され、複数の施設を対象にして作業分析を行うものでもよい。 The server 3 acquires the video from the recorder 2, performs a work analysis to analyze the work situation of the worker based on the video, and outputs the analysis result. The server 3 is installed in a facility (such as a factory) and performs work analysis on that facility. Further, the server 3 may be configured as a cloud computer and perform work analysis on a plurality of facilities.
　管理者端末４は、システム管理者や作業管理者（ユーザ）が使用するものであり、ＰＣやタブレット端末などで構成される。管理者端末４では、サーバ３で行われる処理に関する各種の設定の操作をシステム管理者が行う。また、管理者端末４では、サーバ３から出力される分析結果を表示して作業管理者が閲覧する。 The administrator terminal 4 is used by a system administrator or a work administrator (user), and is composed of a PC, a tablet terminal, or the like. At the administrator terminal 4, a system administrator performs various setting operations related to processing performed by the server 3. In addition, the manager terminal 4 displays the analysis results output from the server 3 for the work manager to view.
　次に、サーバ３の概略構成について説明する。図２は、サーバ３の概略構成を示すブロック図である。 Next, the schematic configuration of the server 3 will be explained. FIG. 2 is a block diagram showing a schematic configuration of the server 3. As shown in FIG.
　サーバ３は、映像入力部１１と、画面出力部１２と、記憶部１３と、プロセッサ１４と、を備えている。 The server 3 includes a video input section 11, a screen output section 12, a storage section 13, and a processor 14.
　映像入力部１１では、レコーダ２で録画された映像が入力される。 The video recorded by the recorder 2 is input to the video input unit 11.
　画面出力部１２では、プロセッサ１４で生成した分析結果表示画面が出力され、管理者端末４に分析結果表示画面（図６参照）が表示される。 The screen output unit 12 outputs the analysis result display screen generated by the processor 14, and the analysis result display screen (see FIG. 6) is displayed on the administrator terminal 4.
　記憶部１３は、プロセッサ１４で実行されるプログラムなどを記憶する。また、記憶部１３は、レコーダ２から取得した分析用の映像を記憶する。また、記憶部１３は、プロセッサ１４で用いられる物体行動認識モデル（物体行動推定モデル）、プロセッサ１４で生成した分析結果映像、および分析結果情報を記憶する。 The storage unit 13 stores programs and the like executed by the processor 14. Furthermore, the storage unit 13 stores the video for analysis acquired from the recorder 2. The storage unit 13 also stores an object behavior recognition model (object behavior estimation model) used by the processor 14, an analysis result video generated by the processor 14, and analysis result information.
　プロセッサ１４は、記憶部１３に記憶されたプログラムを実行することで各種の処理を行う。本実施形態では、プロセッサ１４が、分析処理Ｐ１、および分析結果可視化処理Ｐ２などを行う。 The processor 14 performs various processes by executing programs stored in the storage unit 13. In this embodiment, the processor 14 performs analysis processing P1, analysis result visualization processing P2, and the like.
　分析処理Ｐ１では、プロセッサ１４が、作業者が作業を行う様子を撮影した映像に基づいて作業者の作業状況を分析して分析結果情報を生成する。 In the analysis process P1, the processor 14 analyzes the work situation of the worker based on a video of the worker performing the work, and generates analysis result information.
　分析結果可視化処理Ｐ２では、プロセッサ１４が、分析処理Ｐ１で取得した分析結果情報を可視化してユーザに提示する。本実施形態では、分析結果表示画面（図６参照）を生成する。 In the analysis result visualization process P2, the processor 14 visualizes the analysis result information obtained in the analysis process P1 and presents it to the user. In this embodiment, an analysis result display screen (see FIG. 6) is generated.
　次に、サーバ３で行われる分析処理Ｐ１について説明する。図３は、分析処理Ｐ１の内容を示すブロック図である。 Next, analysis processing P1 performed by the server 3 will be explained. FIG. 3 is a block diagram showing the contents of the analysis process P1.
　サーバ３では、プロセッサ１４が、分析処理Ｐ１を行う。分析処理Ｐ１には、映像取得処理Ｐ１１と、物体行動認識処理Ｐ１２と、分析結果映像生成処理Ｐ１３と、作業サイクル検出処理Ｐ１４と、異常検知処理Ｐ１５と、出力処理Ｐ１６とが含まれる。 In the server 3, the processor 14 performs analysis processing P1. The analysis process P1 includes a video acquisition process P11, an object behavior recognition process P12, an analysis result video generation process P13, a work cycle detection process P14, an abnormality detection process P15, and an output process P16.
　映像取得処理Ｐ１１では、プロセッサ１４が、分析対象とする映像を記憶部１３から取得する。 In the video acquisition process P11, the processor 14 acquires a video to be analyzed from the storage unit 13.
　物体行動認識処理Ｐ１２では、プロセッサ１４が、分析対象とする映像から物体（人物、部品および作業ツール）の種別を認識する物体認識を行うと共に、人物の行動の種別を認識する行動認識（行動推定）を行う。行動認識では、人物の動きと人物が接触した物体の種別とから人物の行動を認識する。物体行動認識処理Ｐ１２では、認識結果情報として、行動に付与された行動ｉｄと、行動に関連する物体（対象物体）の種別と、行動の開始時刻および終了時刻とを取得する。物体行動認識処理Ｐ１２では、ディープラーニングなどの機械学習により構築された物体行動認識モデル（機械学習モデル）を用いることができる。 In object behavior recognition processing P12, the processor 14 performs object recognition to recognize the type of object (person, part, and work tool) from the video to be analyzed, and also performs behavior recognition (behavior estimation) to recognize the type of person's behavior. )I do. In action recognition, a person's actions are recognized based on the person's movement and the type of object the person has come into contact with. In the object action recognition process P12, the action ID given to the action, the type of object (target object) related to the action, and the start time and end time of the action are acquired as recognition result information. In the object behavior recognition process P12, an object behavior recognition model (machine learning model) constructed by machine learning such as deep learning can be used.
　分析結果映像生成処理Ｐ１３では、プロセッサ１４が、認識結果情報が付加された分析結果映像を生成する。具体的には、物体行動認識処理Ｐ１２で認識された物体（人物、部品、作業ツール）を取り囲む矩形枠と、認識された物体の種別を表す文字と、認識された行動の種別を表す文字とを、分析対象となる元の映像上に重畳して、分析結果映像を生成する。 In analysis result video generation processing P13, the processor 14 generates an analysis result video to which recognition result information is added. Specifically, a rectangular frame surrounding the object (person, part, work tool) recognized in the object action recognition process P12, a character representing the type of the recognized object, and a character representing the type of the recognized action. is superimposed on the original video to be analyzed to generate an analysis result video.
　作業サイクル検出処理Ｐ１４では、プロセッサ１４が、物体行動認識処理Ｐ１２で認識された作業者の行動の中から、所定の一連の行動で構成される作業サイクル（規定期間）を検出する。作業サイクル検出処理Ｐ１４では、作業サイクルの検出結果情報として、作業サイクルに付与されたサイクルｉｄと、作業サイクルの開始時刻および終了時刻とを取得する。 In the work cycle detection process P14, the processor 14 detects a work cycle (regular period) consisting of a predetermined series of actions from among the worker actions recognized in the object action recognition process P12. In the work cycle detection process P14, the cycle ID assigned to the work cycle and the start time and end time of the work cycle are acquired as work cycle detection result information.
　異常検知処理Ｐ１５では、プロセッサ１４が、作業者の作業手順と正規の作業手順とを比較して、作業ミス（作業の異常）を検知する。具体的には、プロセッサ１４が、対象とする作業サイクルに関して、作業者が実施した行動と、正規の作業手順に含まれる行動とを比較して、作業ミスを検知する。作業ミスには、例えば、行動が過剰な状態、行動が不足する状態、行動の順番の入れ替わりがある状態などがある。なお、行動が過剰な状態とは、本来実施されるべきでないタイミングで行動が余分に実施された状態である。行動が不足する状態とは、本来実施されるべき行動が実施されていない状態である。 In the abnormality detection process P15, the processor 14 compares the worker's work procedure and the normal work procedure to detect a work error (work abnormality). Specifically, the processor 14 detects a work error by comparing the actions performed by the worker with the actions included in the regular work procedure regarding the target work cycle. Work errors include, for example, excessive action, insufficient action, and a change in the order of actions. Note that the state of excessive action is a state in which an action is performed redundantly at a timing that should not have been performed. A state in which an action is insufficient is a state in which an action that should originally be performed is not performed.
　また、異常検知処理Ｐ１５では、プロセッサ１４が、１つの作業サイクルに関する作業者の作業時間を標準時間と比較して、作業サイクルの異常を検知する。具体的には、例えば、何らかのトラブルによりラインが一時的に停止した場合のように、作業サイクルの時間が標準時間より所定限度以上に長い場合に、作業サイクルの異常と判定される。また、例えば、作業サイクルが途中で取りやめになった場合のように、作業サイクルの時間が標準時間より所定限度以上に短い場合に、作業サイクルの異常と判定される。 In addition, in the abnormality detection process P15, the processor 14 compares the worker's working time regarding one work cycle with the standard time to detect an abnormality in the work cycle. Specifically, the work cycle is determined to be abnormal if the work cycle time is longer than the standard time by more than a predetermined limit, for example, when the line is temporarily stopped due to some kind of trouble. Further, for example, when the time of the work cycle is shorter than the standard time by more than a predetermined limit, as in the case where the work cycle is canceled midway, it is determined that the work cycle is abnormal.
　出力処理Ｐ１６では、プロセッサ１４が、物体行動認識処理Ｐ１２で取得した作業者の行動に関する情報（行動の種別、開始時刻および終了時刻など）、作業サイクル検出処理Ｐ１４で取得した作業サイクルに関する情報（開始時刻および終了時刻など）、および異常検知処理Ｐ１５の検知結果の情報（作業の異常（作業ミス）、作業サイクルの異常）を、分析結果情報として出力する。分析結果情報は記憶部１３に保存される。 In the output process P16, the processor 14 outputs the information related to the worker's behavior (type of behavior, start time, end time, etc.) acquired in the object behavior recognition process P12, and the information related to the work cycle (start time, etc.) acquired in the work cycle detection process P14. time, end time, etc.), and information on the detection results of the abnormality detection process P15 (work abnormalities (work mistakes), work cycle abnormalities) as analysis result information. Analysis result information is stored in the storage unit 13.
　次に、サーバ３で行われる作業サイクル検出処理Ｐ１４について説明する。図４は、作業サイクル検出処理Ｐ１４の状況を示す説明図である。 Next, the work cycle detection process P14 performed by the server 3 will be explained. FIG. 4 is an explanatory diagram showing the status of the work cycle detection process P14.
　サーバ３では、所定の一連の行動で構成される作業サイクルを検出する作業サイクル検出処理Ｐ１４が行われる。具体的には、プロセッサ１４が、認識された行動の中から、作業サイクルの開始タイミングを規定するトリガ行動を検出し、そのトリガ行動が検出されたタイミングを作業サイクルの開始タイミングとし、次のトリガ行動の直前の行動が検出されたタイミングを作業サイクルの終了タイミングとする。 In the server 3, a work cycle detection process P14 is performed to detect a work cycle consisting of a predetermined series of actions. Specifically, the processor 14 detects a trigger action that defines the start timing of a work cycle from among the recognized actions, sets the timing at which the trigger action is detected as the start timing of the work cycle, and sets the next trigger action. The timing at which the action immediately before the action is detected is the end timing of the work cycle.
　図４は、１つの作業サイクルに含まれる行動の一例である。この場合、まず、図４（Ａ）に示すように、作業者が本体を手で把持する行動をとる。次に、図４（Ｂ）に示すように、作業者がドライバを手で把持する行動をとる。次に、図４（Ｃ）に示すように、作業者がネジを部品箱から取り出す行動をとる。次に、図４（Ｄ）に示すように、作業者がネジを締める行動をとる。次に、図４（Ｅ）に示すように、作業者が本体を運ぶ行動をとる。次に、図４（Ｆ）に示すように、作業者が本体を手で把持する行動をとる。 FIG. 4 is an example of actions included in one work cycle. In this case, first, as shown in FIG. 4(A), the worker takes action to grasp the main body with his/her hands. Next, as shown in FIG. 4(B), the worker takes action to grasp the driver with his/her hand. Next, as shown in FIG. 4(C), the operator takes action to take out the screws from the parts box. Next, as shown in FIG. 4(D), the operator takes action to tighten the screw. Next, as shown in FIG. 4(E), the worker takes action to carry the main body. Next, as shown in FIG. 4(F), the worker takes action to grasp the main body with his/her hands.
　ここで、図４（Ａ），（Ｆ）に示すように、作業者が本体を手で把持する行動が、作業サイクルの開始タイミングを規定するトリガ行動となり、そのトリガ行動が開始されたタイミングが作業サイクルの開始タイミングとなる。また、次のトリガ行動の直前の行動が終了したタイミングが作業サイクルの終了タイミングとなる。 Here, as shown in FIGS. 4(A) and 4(F), the action of the worker grasping the main body with his or her hand becomes a trigger action that defines the start timing of the work cycle, and the timing at which the trigger action is started is This is the start time of the work cycle. Further, the timing at which the action immediately before the next triggered action ends is the end timing of the work cycle.
　また、認識された各行動に関して、認識結果情報として、行動に付与された行動ｉｄと、行動に関連する物体（対象物体）の種別と、行動の開始時刻および終了時刻とを含む行動情報が取得される。また、作業サイクルの検出結果情報として、作業サイクルに付与されたサイクルｉｄと、作業サイクルの開始時刻および終了時刻とを含む作業サイクル情報が取得される。 In addition, for each recognized action, action information including the action ID assigned to the action, the type of object (target object) related to the action, and the start time and end time of the action is obtained as recognition result information. be done. Furthermore, as work cycle detection result information, work cycle information including a cycle ID assigned to the work cycle, and the start time and end time of the work cycle is acquired.
　次に、サーバ３で行われる分析結果可視化処理Ｐ２について説明する。図５は、分析結果可視化処理Ｐ２の内容を示すブロック図である。 Next, the analysis result visualization process P2 performed by the server 3 will be explained. FIG. 5 is a block diagram showing the contents of the analysis result visualization process P2.
　サーバ３では、プロセッサ１４が、分析結果可視化処理Ｐ２を行う。分析結果可視化処理Ｐ２には、映像取得処理Ｐ２１と、分析結果取得処理Ｐ２２と、画面生成処理Ｐ２３とが含まれる。 In the server 3, the processor 14 performs analysis result visualization processing P2. The analysis result visualization process P2 includes a video acquisition process P21, an analysis result acquisition process P22, and a screen generation process P23.
　映像取得処理Ｐ２１では、プロセッサ１４が、分析結果映像を記憶部１３から取得する。分析結果映像は、認識された物体（人物、部品、作業ツール）を取り囲む矩形枠と、認識された物体の種別を表す文字と、認識された行動の種別を表す文字とが、分析対象となる元の映像上に重畳されたものである。 In the video acquisition process P21, the processor 14 acquires the analysis result video from the storage unit 13. In the analysis result video, the rectangular frame surrounding the recognized object (person, part, work tool), characters representing the type of recognized object, and characters representing the type of recognized action are the objects of analysis. This is superimposed on the original video.
　分析結果取得処理Ｐ２２では、プロセッサ１４が、分析結果情報を記憶部１３から取得する。分析結果情報は、作業者の行動に関する情報（行動の種別、開始時刻および終了時刻など）、作業サイクルに関する情報（開始時刻および終了時刻など）、および異常検知結果の情報（作業の異常（作業ミス）、作業サイクルの異常）である。 In analysis result acquisition processing P22, the processor 14 acquires analysis result information from the storage unit 13. Analysis result information includes information on worker behavior (type of behavior, start time, end time, etc.), information on work cycles (start time and end time, etc.), and information on abnormality detection results (work abnormality (work error)). ), work cycle abnormalities).
　画面生成処理Ｐ２３では、プロセッサ１４が、分析結果情報を可視化した分析結果表示画面（図６参照）を生成する。この分析結果表示画面は、画面出力部１２から管理者端末４に出力され、管理者端末４に分析結果表示画面が表示される。本実施形態では、分析結果表示画面に、分析結果情報を可視化して表示すると共に分析結果映像を表示する。 In the screen generation process P23, the processor 14 generates an analysis result display screen (see FIG. 6) that visualizes the analysis result information. This analysis result display screen is output from the screen output unit 12 to the administrator terminal 4, and the analysis result display screen is displayed on the administrator terminal 4. In this embodiment, analysis result information is visualized and displayed on the analysis result display screen, and an analysis result video is also displayed.
　次に、管理者端末４に表示される分析結果表示画面について説明する。図６は、分析結果表示画面を示す説明図である。図７は、分析結果表示画面の工程選択部１０２、対象期間指定部１０３、作業映像表示部１０４、再生操作部１０５、および作業サイクル選択部１０６を示す説明図である。図８は、分析結果表示画面の詳細分析結果表示部１０７を示す説明図である。図９，図１０は、作業ミス（異常）が発生した場合における詳細分析結果表示部１０７の要部を示す説明図である。図１１は、詳細分析結果表示部１０７の別例を示す説明図である。図１２は、分析結果表示画面の全体分析結果表示部１０８を示す説明図である。図１３は、全体分析結果表示部１０８の別例を示す説明図である。 Next, the analysis result display screen displayed on the administrator terminal 4 will be explained. FIG. 6 is an explanatory diagram showing an analysis result display screen. FIG. 7 is an explanatory diagram showing the process selection section 102, target period specification section 103, work video display section 104, playback operation section 105, and work cycle selection section 106 of the analysis result display screen. FIG. 8 is an explanatory diagram showing the detailed analysis result display section 107 of the analysis result display screen. 9 and 10 are explanatory diagrams showing the main parts of the detailed analysis result display section 107 when a work error (abnormality) occurs. FIG. 11 is an explanatory diagram showing another example of the detailed analysis result display section 107. FIG. 12 is an explanatory diagram showing the overall analysis result display section 108 of the analysis result display screen. FIG. 13 is an explanatory diagram showing another example of the overall analysis result display section 108.
　図６に示すように、管理者端末４では、分析結果表示画面１０１が表示される。分析結果表示画面１０１には、工程選択部１０２と、対象期間指定部１０３と、作業映像表示部１０４と、再生操作部１０５と、作業サイクル選択部１０６と、詳細分析結果表示部１０７（第２の分析結果表示情報の表示部）と、全体分析結果表示部１０８（第１の分析結果表示情報の表示部）とが設けられている。 As shown in FIG. 6, an analysis result display screen 101 is displayed on the administrator terminal 4. The analysis result display screen 101 includes a process selection section 102, a target period specification section 103, a work video display section 104, a playback operation section 105, a work cycle selection section 106, and a detailed analysis result display section 107 (second (a display section for displaying analysis result display information) and an overall analysis result display section 108 (a display section for first analysis result display information).
　図７に示すように、工程選択部１０２には、工程（ピッキング、プレス、溶接、組立、検査）ごとのボタン１１１が設けられている。ユーザが、ボタン１１１を操作していずれかの工程を選択すると、選択された工程に関する分析結果が分析結果表示画面１０１に表示される。 As shown in FIG. 7, the process selection section 102 is provided with buttons 111 for each process (picking, pressing, welding, assembly, inspection). When the user selects one of the processes by operating the button 111, the analysis results regarding the selected process are displayed on the analysis result display screen 101.
　対象期間指定部１０３では、ユーザが、分析結果を表示させる対象期間（第１の期間）に関する情報として、日付および時間（開始時刻および終了時刻）を文字で入力することができる。なお、対象期間指定部１０３では、レコーダ２に映像が蓄積された期間、すなわち、映像に基づく作業分析結果を得ることができる期間に含まれる任意の期間をユーザが指定することができる。 In the target period designation section 103, the user can input a date and time (start time and end time) in characters as information regarding the target period (first period) for which the analysis results are to be displayed. Note that the target period designation unit 103 allows the user to designate any period included in the period during which video is stored in the recorder 2, that is, the period during which work analysis results based on the video can be obtained.
　作業映像表示部１０４には、分析結果映像１２１が表示される。分析結果映像１２１では、作業者が作業を行う様子を撮影した映像上に、認識された物体（人物、部品、作業ツール）を取り囲む矩形枠１２２と、認識された物体の種別を表す文字１２３と、認識された行動の種別を表す文字１２４とが重畳表示される。本例では、人物を表す矩形枠１２２および文字１２３と、ドライバを表す矩形枠１２２および文字１２３と、ネジ締めの行動を表す文字１２４とが表示される。 The analysis result video 121 is displayed on the work video display section 104. In the analysis result video 121, a rectangular frame 122 surrounding a recognized object (person, part, work tool), and characters 123 representing the type of the recognized object are displayed on a video of a worker performing a task. , and characters 124 representing the type of recognized action are displayed in a superimposed manner. In this example, a rectangular frame 122 and characters 123 representing a person, a rectangular frame 122 and characters 123 representing a driver, and characters 124 representing a screw tightening action are displayed.
　再生操作部１０５には、再生と一時停止とを兼用するボタン１３１と、倍速のボタン１３２とが設けられている。 The playback operation unit 105 is provided with a button 131 for both playback and pause, and a double speed button 132.
　作業サイクル選択部１０６には、各作業サイクル（１～６）のボタン１４１と、前スキップのボタン１４２と、後スキップのボタン１４３とが設けられている。ユーザが、ボタン１４１を操作して作業サイクルを選択すると、選択された作業サイクルに関する分析結果が、作業映像表示部１０４、および詳細分析結果表示部１０７に表示される。ユーザが、前スキップのボタン１４２を操作すると、次の作業サイクルの最初に進む。ユーザが、後スキップのボタン１４３を操作すると、現在表示中の作業サイクルの最初に戻る。 The work cycle selection section 106 is provided with buttons 141 for each work cycle (1 to 6), a previous skip button 142, and a next skip button 143. When the user selects a work cycle by operating the button 141, analysis results regarding the selected work cycle are displayed on the work video display section 104 and the detailed analysis result display section 107. When the user operates the previous skip button 142, the process advances to the beginning of the next work cycle. When the user operates the skip back button 143, the process returns to the beginning of the currently displayed work cycle.
　図８に示すように、詳細分析結果表示部１０７には、第１の行動一覧表示部２０１が設けられている。第１の行動一覧表示部２０１は、各物体に関連する行動の状況に関する短期的な分析結果を可視化してユーザに提示するものである。第１の行動一覧表示部２０１では、作業者の行動に関連する物体ごとに行動表示欄２０２が縦方向に並べて設けられている。本例では、作業者の行動に関連する物体として、部品（本体、＃１の部品、＃２の部品、および＃３の部品）、および作業ツール（ドライバ、ピンセット、およびマーカ）ごとに行動表示欄２０２が設けられている。 As shown in FIG. 8, the detailed analysis result display section 107 is provided with a first action list display section 201. The first behavior list display section 201 visualizes and presents to the user short-term analysis results regarding the behavior status related to each object. In the first action list display section 201, action display columns 202 are arranged vertically for each object related to the worker's actions. In this example, actions are displayed for each part (main body, part #1, part #2, and part #3) and work tools (driver, tweezers, and marker) as objects related to worker actions. A column 202 is provided.
　行動表示欄２０２には、行動バー２０３が描画されている。行動バー２０３は、行動が行われた期間を表す画像である。すなわち、行動表示欄２０２では、横方向が時刻を表す時間軸となり、行動バー２０３は、行動が実施された期間に対応する範囲に描画される。これにより、物体の種別ごとに行動の実施状況が時系列で表現される。 An action bar 203 is drawn in the action display column 202. The action bar 203 is an image representing the period in which the action was performed. That is, in the action display field 202, the horizontal direction is a time axis representing time, and the action bar 203 is drawn in a range corresponding to the period in which the action was performed. As a result, the implementation status of actions for each type of object is expressed in chronological order.
　行動バー２０３は、行動種別、具体的には、部品や作業ツールを手に把持する行動、ドライバでネジを締める行動などに応じて、表示形態が異なる。本例では、行動バー２０３に描画される網掛けの模様（描画パターン）が行動種別に応じて異なる。なお、行動バー２０３の塗り潰しの色が行動種別に応じて異なるようにしてもよい。 The display form of the action bar 203 differs depending on the action type, specifically, the action of grasping a part or work tool in the hand, the action of tightening a screw with a screwdriver, etc. In this example, the shaded pattern (drawing pattern) drawn on the action bar 203 differs depending on the action type. Note that the filling color of the action bar 203 may vary depending on the action type.
　また、第１の行動一覧表示部２０１では、作業サイクル選択部１０６で作業サイクルが選択された場合には、その選択された１つの作業サイクルの全期間の分析結果が表示される。また、全体分析結果表示部１０８で任意の期間が指定された場合には、その指定された期間の分析結果が第１の行動一覧表示部２０１に表示される。 Furthermore, in the first action list display section 201, when a work cycle is selected in the work cycle selection section 106, the analysis results for the entire period of the selected work cycle are displayed. Further, when an arbitrary period is specified in the overall analysis result display section 108, the analysis results for the specified period are displayed on the first action list display section 201.
　また、第１の行動一覧表示部２０１では、作業映像表示部１０４に表示された分析結果映像１２１の現在の再生位置（再生時刻）を示すタイムラインカーソル２０４が表示される。タイムラインカーソル２０４は、作業映像表示部１０４の分析結果映像１２１の再生時刻が進むのに応じて横方向（時間軸方向）に移動する。 Furthermore, in the first action list display section 201, a timeline cursor 204 indicating the current playback position (playback time) of the analysis result video 121 displayed on the work video display section 104 is displayed. The timeline cursor 204 moves in the horizontal direction (time axis direction) as the playback time of the analysis result video 121 on the work video display section 104 advances.
　また、詳細分析結果表示部１０７には、時間表示部２０５が設けられている。時間表示部２０５には、第１の行動一覧表示部２０１に表示された分析結果の範囲（表示期間）の開始時刻、終了時刻、時間長さが表示される。ここで、作業サイクル選択部１０６で作業サイクルが選択されることで、１つの作業サイクルの全期間が第１の行動一覧表示部２０１に表示されている状態では、表示された時間長さが、選択された作業サイクルの作業時間を表す。 Further, the detailed analysis result display section 107 is provided with a time display section 205. The time display section 205 displays the start time, end time, and time length of the range (display period) of the analysis results displayed on the first action list display section 201 . Here, when a work cycle is selected in the work cycle selection unit 106 and the entire period of one work cycle is displayed in the first action list display unit 201, the displayed time length is Represents the working time of the selected work cycle.
　また、詳細分析結果表示部１０７では、作業ミス（異常）が検知された場合に、検知された作業ミスに関係する物体（工具、部品）に関する行動表示欄２０２が強調表示（ハイライト）される。具体的には、該当する行動表示欄２０２の背景の表示形態（例えば、塗り潰しの色、網掛けの模様など）が変化する。なお、作業ミス（異常）には、行動が過剰な状態、行動が不足する状態、行動の順番の入れ替わりがある状態などがある。 Further, in the detailed analysis result display section 107, when a work error (abnormality) is detected, an action display column 202 regarding an object (tool, part) related to the detected work error is highlighted. . Specifically, the display form of the background of the corresponding action display column 202 (for example, the color of the fill, the pattern of the shading, etc.) changes. Note that work errors (abnormalities) include excessive action, insufficient action, and a change in the order of actions.
　また、詳細分析結果表示部１０７では、作業ミス（異常）が検知された場合に、アラートマーク２０６が表示される。アラートマーク２０６は、検知された作業ミスに関係する物体に関する行動表示欄２０２において、作業ミスに関係する行動の実施タイミングに対応する位置に表示される。例えば、行動が不足する場合には、その行動が実施されるべき位置にアラートマーク２０６が表示される。また、行動が余分である場合には、その余分な行動が実施された位置にアラートマーク２０６が表示される。 Further, in the detailed analysis result display section 107, an alert mark 206 is displayed when a work error (abnormality) is detected. The alert mark 206 is displayed at a position corresponding to the implementation timing of the action related to the work error in the action display field 202 regarding the object related to the detected work error. For example, if an action is insufficient, an alert mark 206 is displayed at the position where the action should be performed. Furthermore, if the action is redundant, an alert mark 206 is displayed at the position where the redundant action was performed.
　なお、アラートマーク２０６の代わりに、あるいはアラートマーク２０６に加えて、作業ミスに関係する行動バー２０３の表示形態が変化することで強調表示されてもよい。例えば、行動バー２０３の枠線が点線で描画されたり、行動バー２０３の透明度が変更されたりしてもよい。 Note that instead of or in addition to the alert mark 206, the action bar 203 related to the work error may be highlighted by changing its display form. For example, the frame line of the action bar 203 may be drawn as a dotted line, or the transparency of the action bar 203 may be changed.
　ここで、図９（Ａ－１），（Ａ－２）は、＃３の部品に関連する行動の例である。この場合、部品ハンドリング作業が、部品を手で把持する行動で構成される。図９（Ａ－２）に示す作業者の作業手順では、図９（Ａ－１）に示す正規の作業手順と比較して、部品を手で把持する行動が１回過剰であり、ここが作業ミス（異常）として検知されて、アラートマーク２０６が表示される。 Here, FIGS. 9(A-1) and (A-2) are examples of actions related to component #3. In this case, the part handling work consists of the action of grasping the part by hand. In the worker's work procedure shown in Figure 9 (A-2), compared to the regular work procedure shown in Figure 9 (A-1), the action of grasping the part with the hand is performed one more time; This is detected as a work error (abnormality) and an alert mark 206 is displayed.
　図９（Ｂ－１），（Ｂ－２）は、ドライバに関連する行動の例である。この場合、ネジ締め作業が、ドライバを手で把持する行動と、ネジを部品箱から取り出す行動と、ネジを締める行動とで構成され、ネジを部品箱から取り出す行動、およびネジを締める行動は、３回繰り返される。図９（Ｂ－２）に示す作業者の作業手順では、図９（Ｂ－１）に示す正規の作業手順と比較して、ネジを部品箱から取り出す行動が１回少なく、ここが作業ミス（異常）として検知されて、アラートマーク２０６が表示される。 FIGS. 9(B-1) and (B-2) are examples of actions related to the driver. In this case, the screw tightening work consists of the action of grasping the screwdriver by hand, the action of taking out the screw from the parts box, and the action of tightening the screw.The action of taking out the screw from the parts box and the action of tightening the screw are Repeated 3 times. In the worker's work procedure shown in Figure 9 (B-2), compared to the regular work procedure shown in Figure 9 (B-1), there is one less action to take out the screw from the parts box, which is a work error. (Abnormality) is detected, and an alert mark 206 is displayed.
　図１０（Ａ－１），（Ａ－２）は、マーカに関連する行動の例である。この場合、マーキング作業が、マーカを手で把持する行動と、キャップを開ける行動と、マークを付ける行動と、キャップを閉める行動とで構成される。図１０（Ａ－２）に示す作業者の作業手順では、図１０（Ａ－１）に示す正規の作業手順と比較して、キャップを閉める行動が不足しており、ここが作業ミス（異常）として検知されて、アラートマーク２０６が表示される。 Figures 10 (A-1) and (A-2) are examples of actions related to markers. In this case, the marking work consists of the action of grasping the marker with the hand, the action of opening the cap, the action of placing a mark, and the action of closing the cap. In the worker's work procedure shown in Figure 10 (A-2), there is a lack of action to close the cap compared to the normal work procedure shown in Figure 10 (A-1), and this is due to a work error (abnormality). ) is detected and an alert mark 206 is displayed.
　また、図１１に示すように、詳細分析結果表示部１０７では、作業ミス（異常）が検知された場合に、異常内容表示部２１１が表示されようにしてもよい。異常内容表示部２１１には、検知された作業ミスの内容を表す文字が表示される。ここでは、＃３の部品に関連する行動が１回過剰であることと、ドライバに関連する行動としてのねじ締めが１回少ないことと、マーカに関連する行動としてのキャップの閉め忘れが発生していることとが表示される。 Further, as shown in FIG. 11, in the detailed analysis result display section 107, when a work error (abnormality) is detected, an abnormality content display section 211 may be displayed. The abnormality content display section 211 displays characters representing the content of the detected work error. Here, there is one excess action related to #3, one less screw tightening action related to the driver, and forgetting to close the cap as an action related to the marker. is displayed.
　図１２に示すように、全体分析結果表示部１０８には、第２の行動一覧表示部３０１が設けられている。第２の行動一覧表示部３０１は、各物体に関連する行動の状況に関する長期的な分析結果を可視化してユーザに提示するものである。第２の行動一覧表示部３０１では、第１の行動一覧表示部２０１と同様に、作業者の行動に関連する物体ごとに行動表示欄３０２が縦方向に並べて設けられている。行動表示欄３０２には、第１の行動一覧表示部２０１と同様に、行動バー３０３が描画されている。 As shown in FIG. 12, the overall analysis result display section 108 is provided with a second action list display section 301. The second action list display section 301 visualizes and presents to the user long-term analysis results regarding the state of actions related to each object. In the second action list display section 301, similarly to the first action list display section 201, action display columns 302 are arranged vertically for each object related to the worker's actions. In the action display field 302, an action bar 303 is drawn, similar to the first action list display section 201.
　第２の行動一覧表示部３０１では、対象期間指定部１０３で指定された期間（対象期間）の分析結果が表示される。 In the second action list display section 301, the analysis results for the period (target period) specified by the target period designation section 103 are displayed.
　また、全体分析結果表示部１０８には、スライダーバー３０４が設けられている。スライダーバー３０４では、ユーザが、左右のスライダー３０５を操作することで、詳細分析結果表示部１０７に表示させる分析結果に関して任意の期間を指定することができる。ここで、左側のスライダー３０５は開始時刻を規定し、右側のスライダー３０５は終了時刻を規定する。これにより、全体分析結果表示部１０８において期間が指定されることで、その指定された期間の分析結果が、時間軸方向に拡大された状態で詳細分析結果表示部１０７に表示される。 Further, the overall analysis result display section 108 is provided with a slider bar 304. In the slider bar 304, the user can specify an arbitrary period for the analysis results to be displayed on the detailed analysis result display section 107 by operating the left and right sliders 305. Here, the slider 305 on the left side defines the start time, and the slider 305 on the right side defines the end time. As a result, by specifying a period in the overall analysis result display section 108, the analysis results for the specified period are displayed in the detailed analysis result display section 107 in an enlarged state in the time axis direction.
　なお、作業サイクル選択部１０６において、ユーザが、作業サイクルを選択する操作を行うと、選択された作業サイクルの期間（開始時刻および終了時刻）に対応するように左右のスライダー３０５が移動すると共に、選択された作業サイクルの分析結果が詳細分析結果表示部１０７に表示される。 Note that when the user performs an operation to select a work cycle in the work cycle selection unit 106, the left and right sliders 305 move to correspond to the period (start time and end time) of the selected work cycle, and The analysis result of the selected work cycle is displayed in the detailed analysis result display section 107.
　また、全体分析結果表示部１０８では、作業ミス（異常）が検知された場合に、回数マーク３０６が表示される。回数マーク３０６は、１つの作業サイクルにおける作業ミス（異常）の検知回数を表す。回数マーク３０６は、作業ミスが検知された作業サイクルの表示領域に表示される。本例では、＃１の作業サイクルにおいて作業ミスが１回検出され、＃４の作業サイクルにおいて作業ミスが３回検出されている。 Furthermore, in the overall analysis result display section 108, a number mark 306 is displayed when a work error (abnormality) is detected. The number of times mark 306 represents the number of times a work error (abnormality) is detected in one work cycle. The number of times mark 306 is displayed in the display area of the work cycle in which the work error was detected. In this example, one work error is detected in the #1 work cycle, and three work mistakes are detected in the #4 work cycle.
　また、全体分析結果表示部１０８では、作業サイクルの異常が検知された場合に、その異常が検知された作業サイクルが強調表示（ハイライト）される。具体的には、該当する作業サイクルの期間に矩形画像３０７が表示される。矩形画像３０７は、例えば、塗り潰しの色や網掛けの模様などで強調表示される。本例では、＃４の作業サイクルの異常が検知されている。なお、作業サイクルの異常とは、具体的には、作業サイクルの時間が標準時間より所定限度以上に長い場合や、作業サイクルの時間が標準時間より所定限度以上に短い場合である。 Furthermore, in the overall analysis result display section 108, when an abnormality in a work cycle is detected, the work cycle in which the abnormality was detected is highlighted. Specifically, rectangular image 307 is displayed during the corresponding work cycle. The rectangular image 307 is highlighted with, for example, a filled color or a hatched pattern. In this example, an abnormality in work cycle #4 has been detected. Note that an abnormality in the work cycle specifically refers to a case where the time of the work cycle is longer than the standard time by more than a predetermined limit, or a case where the time of the work cycle is shorter than the standard time by more than a predetermined limit.
　また、図１３に示すように、全体分析結果表示部１０８では、スライダーバー３０４で指定された期間、すなわち、詳細分析結果表示部１０７に分析結果が表示された期間が強調表示（ハイライト）される。具体的には、該当する期間に矩形画像３１１が表示される。矩形画像３１１は、例えば、塗り潰しの色や網掛けの模様などで強調表示される。矩形画像３１１は、異常が検知された作業サイクルを強調表示する矩形画像３０７と識別できるように、矩形画像３０７と異なる表示形態で表示される。本例では、＃５の作業サイクルが選択されて、＃５の作業サイクルに矩形画像３１１が表示されている。 Further, as shown in FIG. 13, in the overall analysis result display section 108, the period specified by the slider bar 304, that is, the period in which the analysis results were displayed in the detailed analysis result display section 107, is highlighted. Ru. Specifically, rectangular image 311 is displayed during the corresponding period. The rectangular image 311 is highlighted with, for example, a filled color or a hatched pattern. The rectangular image 311 is displayed in a different display form from the rectangular image 307 so that it can be distinguished from the rectangular image 307 that highlights the work cycle in which an abnormality has been detected. In this example, the #5 work cycle is selected, and the rectangular image 311 is displayed in the #5 work cycle.
　このように本実施形態では、ユーザは、対象期間指定部１０３により、全体分析結果表示部１０８に表示させる分析結果の期間（対象期間、第１の期間）を任意に指定することができる。また、ユーザは、スライダーバー３０４により、詳細分析結果表示部１０７に表示させる分析結果の期間（対象期間の一部、第２の期間）を任意に指定することができる。また、ユーザは、作業サイクル選択部１０６により、１つの作業サイクルに関する分析結果を詳細分析結果表示部１０７に表示させることができる。 As described above, in the present embodiment, the user can arbitrarily specify the period (target period, first period) of the analysis results to be displayed on the overall analysis result display section 108 using the target period specifying section 103. Further, the user can use the slider bar 304 to arbitrarily specify the period (a part of the target period, the second period) of the analysis results to be displayed on the detailed analysis result display section 107. Further, the user can use the work cycle selection unit 106 to display analysis results regarding one work cycle on the detailed analysis result display unit 107.
　全体分析結果表示部１０８の対象となる第１の期間は、作業者の作業が実施された全期間から抽出された対象期間の全体であり、詳細分析結果表示部１０７の対象となる第２の期間は、対象期間の一部である。すなわち、詳細分析結果表示部１０７と全体分析結果表示部１０８とでは時間軸のスケールが異なり、全体分析結果表示部１０８に表示された分析結果の一部が、時間軸方向に拡大された状態で詳細分析結果表示部１０７に表示される。これにより、ユーザが、第１の分析結果表示情報を閲覧することで、広い期間を対象にした作業者の作業状況を俯瞰的に確認することができる。また、ユーザが、第２の分析結果表示情報を閲覧することで、作業者の作業状況を詳細に確認することができる。 The first period targeted by the overall analysis result display section 108 is the entire target period extracted from the entire period in which the worker's work was performed, and the second period targeted by the detailed analysis result display section 107 is The period is part of the period of interest. In other words, the detailed analysis result display section 107 and the overall analysis result display section 108 have different time axis scales, and some of the analysis results displayed on the overall analysis result display section 108 are enlarged in the time axis direction. It is displayed on the detailed analysis result display section 107. Thereby, by viewing the first analysis result display information, the user can check the work status of the worker from a bird's-eye view over a wide period of time. Furthermore, by viewing the second analysis result display information, the user can check the work status of the worker in detail.
　具体的な手順としては、例えば、ユーザが、まず、作業サイクル選択部１０６により、１つの作業サイクルに関する分析結果を詳細分析結果表示部１０７に表示させて閲覧し、必要であれば、スライダーバー３０４により、詳細分析結果表示部１０７に表示させる分析結果の期間を調整することで、様々な範囲の分析結果を閲覧することができる。例えば、ユーザは、スライダーバー３０４により、１つの作業サイクル内の注目する任意の範囲の分析結果だけを詳細分析結果表示部１０７に表示させることができる。また、ユーザは、スライダーバー３０４により、２つ以上の作業サイクルに跨がる範囲の分析結果を詳細分析結果表示部１０７に表示させることができる。なお、作業状況の分析として、工場での各工程での作業を対象に説明したが、工場以外での作業、すなわち、各種施設内で定期的に発生する作業（例えば、商品の搬入・搬出作業、清掃作業、巡回警備作業）にも適用することが可能である。 As a specific procedure, for example, the user first uses the work cycle selection unit 106 to display the analysis results for one work cycle on the detailed analysis result display unit 107 to view them, and if necessary, selects the slider bar 304. By adjusting the period of analysis results displayed on the detailed analysis result display section 107, analysis results in various ranges can be viewed. For example, the user can use the slider bar 304 to display only the analysis results of an arbitrary range of interest within one work cycle on the detailed analysis result display section 107. Furthermore, the user can use the slider bar 304 to display analysis results in a range spanning two or more work cycles on the detailed analysis result display section 107. Although the analysis of the work situation was explained based on work in each process at the factory, work outside the factory, that is, work that occurs periodically within various facilities (for example, loading and unloading of products) , cleaning work, patrolling security work).
　以上のように、本出願において開示する技術の例示として、実施形態を説明した。しかしながら、本開示における技術は、これに限定されず、変更、置き換え、付加、省略などを行った実施形態にも適用できる。また、上記の実施形態で説明した各構成要素を組み合わせて、新たな実施形態とすることも可能である。 As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments in which changes, replacements, additions, omissions, etc. are made. Furthermore, it is also possible to create a new embodiment by combining the components described in the above embodiments.
　本開示に係る作業分析装置および作業分析方法は、作業者の作業状況を分析した分析結果が長時間に及ぶ場合でも、その長時間に及ぶ分析結果をユーザが効率よく確認できるようになる効果を有し、作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析装置および作業分析方法などとして有用である。 The work analysis device and work analysis method according to the present disclosure have the effect that even when the analysis results of the work situation of a worker extend over a long period of time, the user can efficiently check the analysis results over a long period of time. The present invention is useful as a work analysis device and a work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation.
１　カメラ
２　レコーダ
３　サーバ（作業分析装置）
４　管理者端末
１１　映像入力部
１２　画面出力部
１３　メモリ
１４　プロセッサ
１０１　分析結果表示画面
１０２　工程選択部
１０３　対象期間指定部
１０４　作業映像表示部
１０５　再生操作部
１０６　作業サイクル選択部
１０７　詳細分析結果表示部
１０８　全体分析結果表示部
１４１　作業サイクルを選択するボタン
２０１　第１の行動一覧表示部
２０２　行動表示欄
２０３　行動バー
２０４　タイムラインカーソル
２０６　アラートマーク
２１１　異常内容表示部
３０１　第２の行動一覧表示部
３０２　行動表示欄
３０３　行動バー
３０４　スライダーバー
３０５　スライダー
３０６　回数マーク 1 Camera 2 Recorder 3 Server (work analysis device)
4 Administrator terminal 11 Video input section 12 Screen output section 13 Memory 14 Processor 101 Analysis result display screen 102 Process selection section 103 Target period specification section 104 Work video display section 105 Playback operation section 106 Work cycle selection section 107 Detailed analysis result display section 108 Overall analysis result display section 141 Button to select work cycle 201 First action list display section 202 Action display field 203 Action bar 204 Timeline cursor 206 Alert mark 211 Abnormal content display section 301 Second action list display section 302 Action Display field 303 Action bar 304 Slider bar 305 Slider 306 Number of times mark

Claims (9)

  1. 　作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析装置であって、
    　前記プロセッサは、
    　前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、
    　第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力することを特徴とする作業分析装置。     A work analysis device in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the work analysis device comprising:
    The processor includes:
    Based on the video, the target object handled by the worker is recognized, and the actions constituting the worker's work are recognized, and the analysis result information including the recognition result of the target object and the recognition result of the action is generated. Acquired,
    first analysis result display information that visualizes the analysis result information targeting a first period; and time from the first analysis result display information targeting a second period included in the first period. A work analysis device characterized by outputting second analysis result display information that visualizes the analysis result information by enlarging an axis.
  2. 　前記プロセッサは、
    　前記映像を、前記第１および第２の分析結果表示情報の少なくとも一方に対応付けられた状態で再生可能に出力することを特徴とする請求項１に記載の作業分析装置。     The processor includes:
    The work analysis device according to claim 1, wherein the video is output in a reproducible manner in a state where the video is associated with at least one of the first and second analysis result display information.
  3. 　前記第１の期間は、作業者の作業が実施された全期間から抽出された対象期間の全体であり、
    　前記第２の期間は、前記対象期間の一部であることを特徴とする請求項１に記載の作業分析装置。     The first period is the entire target period extracted from the entire period in which the worker's work was performed,
    The work analysis device according to claim 1, wherein the second period is a part of the target period.
  4. 　前記プロセッサは、
    　前記第１の期間に含まれる複数の規定期間のいずれかを選択するユーザの操作に応じて、選択された規定期間を前記第２の期間に設定して前記第２の分析結果表示情報を出力することを特徴とする請求項１に記載の作業分析装置。     The processor includes:
    In response to a user's operation to select one of the plurality of prescribed periods included in the first period, the selected prescribed period is set as the second period and the second analysis result display information is output. The work analysis device according to claim 1, characterized in that:
  5. 　前記プロセッサは、
    　前記第１の分析結果表示情報における時間軸上で前記第２の期間を任意に指定するユーザの操作に応じて、指定された前記第２の期間を対象にした前記第２の分析結果表示情報を出力することを特徴とする請求項１に記載の作業分析装置。     The processor includes:
    In response to a user's operation to arbitrarily designate the second period on the time axis in the first analysis result display information, the second analysis result display information targets the specified second period. The work analysis device according to claim 1, wherein the work analysis device outputs the following.
  6. 　前記プロセッサは、
    　作業者の作業状況に関する異常を検知すると、その異常検知結果に関する情報を含む前記第１および第２の分析結果表示情報の少なくとも一方を出力することを特徴とする請求項１に記載の作業分析装置。     The processor includes:
    2. The work analysis device according to claim 1, wherein when an abnormality related to the worker's work situation is detected, at least one of the first and second analysis result display information including information regarding the abnormality detection result is outputted. .
  7. 　前記プロセッサが、前記異常検知結果に関する情報を含む前記第２の分析結果表示情報を出力し、
    　前記第２の分析結果表示情報は、前記異常検知結果に関する表示情報として、
    　前記対象物体の表示欄の強調表示と、
    　前記異常に関係する行動を示すマークの表示と、
    　前記異常の内容を表す文字の表示との少なくともいずれかを含むことを特徴とする請求項６に記載の作業分析装置。     the processor outputs the second analysis result display information including information regarding the abnormality detection result;
    The second analysis result display information includes, as display information regarding the abnormality detection result,
    highlighting the display field of the target object;
    displaying a mark indicating an action related to the abnormality;
    7. The work analysis device according to claim 6, further comprising at least one of displaying characters representing the content of the abnormality.
  8. 　前記プロセッサが、前記異常検知結果に関する情報を含む前記第１の分析結果表示情報を出力し、
    　前記第１の分析結果表示情報は、前記異常検知結果に関する表示情報として、
    　前記第１の期間に含まれる規定期間で集計された統計情報の表示を含むことを特徴とする請求項６に記載の作業分析装置。     the processor outputs the first analysis result display information including information regarding the abnormality detection result;
    The first analysis result display information includes, as display information regarding the abnormality detection result,
    7. The work analysis device according to claim 6, further comprising a display of statistical information aggregated over a specified period included in the first period.
  9. 　作業者の作業の様子を撮影した映像に基づいて、作業者の作業状況に関する分析結果情報を出力する処理をプロセッサにより実行する作業分析方法であって、
    　前記映像に基づいて、作業者が取り扱う対象物体を認識すると共に、作業者の作業を構成する行動を認識して、前記対象物体の認識結果と前記行動の認識結果とを含む前記分析結果情報を取得し、
    　第１の期間を対象にした前記分析結果情報を可視化した第１の分析結果表示情報と、前記第１の期間に含まれる第２の期間を対象にして前記第１の分析結果表示情報より時間軸を拡大して前記分析結果情報を可視化した第２の分析結果表示情報とを出力することを特徴とする作業分析方法。     A work analysis method in which a processor executes a process of outputting analysis result information regarding a worker's work situation based on a video shot of the worker's work situation, the method comprising:
    Based on the video, the target object handled by the worker is recognized, and the actions constituting the worker's work are recognized, and the analysis result information including the recognition result of the target object and the recognition result of the action is generated. Acquired,
    first analysis result display information that visualizes the analysis result information targeting a first period; and time from the first analysis result display information targeting a second period included in the first period. A work analysis method characterized by outputting second analysis result display information that visualizes the analysis result information by enlarging an axis.
PCT/JP2023/007348 2022-03-22 2023-02-28 Work analyzing system and work analyzing method WO2023181805A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-045888 2022-03-22
JP2022045888A JP2023140047A (en) 2022-03-22 2022-03-22 Operation analysis device and operation analysis method

Publications (1)

Publication Number Publication Date
WO2023181805A1 true WO2023181805A1 (en) 2023-09-28

Family

ID=88100575

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/007348 WO2023181805A1 (en) 2022-03-22 2023-02-28 Work analyzing system and work analyzing method

Country Status (2)

Country Link
JP (1) JP2023140047A (en)
WO (1) WO2023181805A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021071778A (en) * 2019-10-29 2021-05-06 オムロン株式会社 Skill evaluation device, skill evaluation method, and skill evaluation program
JP2021077070A (en) * 2019-11-08 2021-05-20 オムロン株式会社 Motion analysis device, motion analysis method, and motion analysis program
JP2021076920A (en) * 2019-11-05 2021-05-20 オムロン株式会社 Element work dividing device, element work dividing method, element work dividing program, and element work dividing system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021071778A (en) * 2019-10-29 2021-05-06 オムロン株式会社 Skill evaluation device, skill evaluation method, and skill evaluation program
JP2021076920A (en) * 2019-11-05 2021-05-20 オムロン株式会社 Element work dividing device, element work dividing method, element work dividing program, and element work dividing system
JP2021077070A (en) * 2019-11-08 2021-05-20 オムロン株式会社 Motion analysis device, motion analysis method, and motion analysis program

Also Published As

Publication number Publication date
JP2023140047A (en) 2023-10-04

Similar Documents

Publication Publication Date Title
CN107924173B (en) Monitoring device, monitoring system, and computer-readable storage medium
US20040138783A1 (en) Operation state analyzer for robot
US10019346B2 (en) Generating software test script from video
US20230044842A1 (en) Work analyzing device and work analyzing method
US20180107192A1 (en) Information processing device
WO2019058775A1 (en) Abnormality detection device and abnormality detection method
US11138805B2 (en) Quantitative quality assurance for mixed reality
JP2017010277A (en) Work analysis system and work analysis method
JP7393720B2 (en) Skill evaluation device, skill evaluation method, and skill evaluation program
JP6928880B2 (en) Motion analysis device, motion analysis method, motion analysis program and motion analysis system
WO2023181805A1 (en) Work analyzing system and work analyzing method
US20160110611A1 (en) Numerical control device
WO2023181804A1 (en) Operation analysis device and operation analysis method
JP7456111B2 (en) Production visualization system
WO2022044637A1 (en) Image processing device, image processing method, and program
JP2022164831A (en) Work operation rate measurement device and work operation rate measurement method
JP6948294B2 (en) Work abnormality detection support device, work abnormality detection support method, and work abnormality detection support program
EP3301529A1 (en) Method and apparatus for displaying monitoring information
JP2009110239A (en) System and method for analysis of working operation
KR20190097116A (en) Analytical Supporting Equipment of Steel Plant
JPWO2023067642A5 (en)
JP2007164800A (en) Data display apparatus
WO2020157790A1 (en) Device state reproduction device, device state reproduction method, and device state reproduction program
JP2001005522A (en) Data displaying device
WO2024029411A1 (en) Work feature amount display device, work feature amount display method, and work feature amount display program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23774404

Country of ref document: EP

Kind code of ref document: A1