JP2024033364A - Worker support system - Google Patents

Abstract

[Problem] To provide a worker support system that can be used to train workers with inexperienced skills. [Solution] The worker support system includes a camera that takes videos of a worker wearing a suit, goggles, and a control device, and the control device has a storage unit that stores standard motion data and past motion data, an acquisition unit that uses the captured data taken by the camera to acquire target motion data indicating the motions of the worker to be supported, an extraction unit that compares the standard motion data with the target motion data to extract motions that cause delays in task time compared to the standard motion data, a prediction unit that uses the past motion data to predict the time required to complete all steps of the task to be supported, and a display control unit that displays the extracted causative motion on the goggles when the predicted time exceeds the standard time required to complete all steps of the task in the standard motion data. [Selected Figure] Figure 2

Description

The present disclosure relates to a worker support system.

Conventionally, in parts assembly work support systems, mixed reality technology is used to create a composite image in which a virtual image showing the actual shape of the second part is superimposed on the real image of the first part, which is attached to the head of the worker. A technique for displaying information on a head mounted display is known (Patent Document 1).

Japanese Patent Application Publication No. 2014-229057

With conventional techniques, problems in the worker's movements are not displayed on the head mounted display. Therefore, it is not possible to notify the operator of operational problems during the work, and there are cases where the system cannot be used to instruct workers with less skill.

The present disclosure can be realized as the following forms.

According to one form of the present disclosure, a worker support system is provided. The worker support system includes a camera that takes video of the worker wearing a suit that allows the worker's movements to be captured throughout the entire process of the work to be supported, and a camera worn by the worker. goggles and a control device, the control device includes standard movement data indicating the movement of an expert who performs the work in a standard manner, and information about the work performed by the worker when the worker performed the work in the past. past motion data indicating the motion of the worker who is the target of the support; , an extraction unit that compares the standard motion data with the target motion data and extracts the cause of a delay in work time compared to the standard motion data; and a target of the support using the past motion data. a prediction unit that predicts the time required to complete the all steps of the work; and the predicted time exceeds the standard time required to complete the all steps of the work according to the standard operation data. and a display control unit that causes the goggles to display the extracted causative action. According to this embodiment, when the worker's movements are behind the standard movements of an expert, the cause of the delay can be notified to the worker during the work. This allows the worker to understand and improve problems in his or her own movements during work. Therefore, it can be used to instruct workers with unskilled skills.
The present disclosure can be realized in various forms other than the worker support system described above. For example, it can be realized in the form of a method of manufacturing a worker support system, a method of controlling a worker support system, a computer program for realizing the control method, a non-temporary recording medium on which the computer program is recorded, or the like. Further, for example, the acquisition unit may acquire the target motion data by inputting the imaging data into a machine learning model stored in the storage unit in advance.

FIG. 1 is a diagram showing a schematic configuration of a worker support system. Flowchart showing a worker support method.

A. Embodiment:
FIG. 1 is a diagram showing a schematic configuration of a worker support system 1. As shown in FIG. The worker support system 1 is a system for instructing unskilled workers in place of skilled workers who carry out tasks in a standard manner. Specifically, the worker support system 1 acquires the movement (whole body movement) of the worker P using motion capture technology, and then uses augmented reality (AR) technology to identify problems in the worker P's movement. Notify worker P during work. The worker support system 1 includes a suit 8 worn by the worker P, cameras 61 to 64 that take pictures of the worker P wearing the suit 8, goggles 7 worn on the head of the worker P, and worker support systems. A control device 10 that controls each component of the system 1 is provided. In this embodiment, a worker support system 1 including an optical motion capture system will be described as an example.

The suit 8 is a motion capture suit that allows the movement of the worker P to be captured when the worker P is photographed by the cameras 61 to 64. The suit 8 includes, for example, a plurality of marker members 80 for detecting the three-dimensional position of each body part (shoulders, elbows, knees, wrists, ankles, etc.) of the worker P wearing the suit 8. In this embodiment, the marker member 80 is a retroreflector that reflects the irradiated infrared rays I.

The cameras 61 to 64 take videos of the worker P wearing the suit 8 throughout the entire work process to be supported. In this embodiment, the cameras 61 to 64 emit infrared rays I toward the worker P during photographing. Then, the cameras 61 to 64 capture the infrared light reflected by the marker member 80 and obtain an image using the infrared light (hereinafter, photographed data 40). Note that the number of cameras 61 to 64 is not limited to this, and it is preferable that a plurality of cameras be provided surrounding the worker P in order to improve the detection accuracy of each part of the worker P's body. In addition, in order to prevent the worker P's hands from entering the worker P's own blind spot, the plurality of cameras 61 to 64 are positioned so that they can photograph the worker P from a plurality of different directions and angles. It is preferable that the

The goggles 7 are head-mounted display devices (HMDs) that allow the worker P to visually see information indicating problems in the worker P's movements (hereinafter referred to as worker support information). . The goggles 7 have a display section 70 that displays augmented reality in which worker support information is superimposed on the real world.

The control device 10 controls each component of the worker support system 1. The control device 10 includes a communication section 3, a storage section 4, and a CPU 2. The control device 10 is communicably connected to the cameras 61 to 64 and goggles 7 via the communication unit 3. The storage unit 4 stores various data including various programs that control the operation of the worker support system 1, photographic data 40, standard operation data 41, target operation data 45, and past operation data 43. The standard operation data 41 is data indicating the operation when a skilled person who performs the work in a standard manner executes each work process according to the standard work time. Hereinafter, the standard working time in each work process will be referred to as "standard working time." Furthermore, when all the work steps are executed using the standard work time, the time required to complete all the work steps is referred to as the "standard required time." The standard required time is, for example, an ideal takt time. The target motion data 45 is data indicating the motion of the worker P who is the target of support from the time when the worker P starts work to the present time. In the following, the work time for each work process included in the target action data 45 will be referred to as "current work time." The past motion data 43 is data indicating motions when the same worker P as the worker P in the target motion data 45 performed a task in the past. In the following, the working time for each working process included in the past working data 43 will be referred to as "past working time." Furthermore, when all the work steps are executed using the past work time, the time required to complete all the work steps is referred to as the "past required time."

The CPU 2 functions as an acquisition section 21, an extraction section 23, a prediction section 25, and a display control section 27 by developing various programs stored in the storage section 4. Details of each function will be described later.

FIG. 2 is a flowchart showing the worker support method. In the worker support method, an acquisition step (step S1), an extraction step (step S3), a prediction step (step S5), and a display control step (step S7) are executed in this order.

The acquisition step (step S1) is a step of acquiring target motion data 45. First, in step S11, the worker support system 1 uses the cameras 61 to 64 to photograph the worker P wearing the suit 8 while working, and obtains photographic data 40. The photographic data 40 is stored in the storage unit 4 as appropriate. Next, in step S13, the acquisition unit 21 uses the photographic data 40 to acquire three-dimensional position information indicating the three-dimensional position of each marker member 80. Specifically, the acquisition unit 21 detects the marker member 80 included in the photographed data 40 of each camera 61 to 64 through binarization processing or the like, and obtains a two-dimensional position indicating the two-dimensional position of the marker member 80. Get information. Further, the acquisition unit 21 can obtain each of the markers by combining, for example, two-dimensional position information for the same marker member 80 acquired by each camera 61 to 64 and camera position information indicating the positional relationship of each camera 61 to 64. Three-dimensional position information of each marker member 80 is acquired. Next, in step S15, the acquisition unit 21 acquires the target motion data 45, for example, by arranging the three-dimensional position information of the plurality of marker members 80 in chronological order for each of the same marker members 80. Note that the standard operation data 41 and past operation data 43 may also be acquired by a similar method and stored in the storage unit 4 in advance.

The extraction step (step S3) is a step of extracting, from among the actions of the worker P included in the past action data 43, actions that cause a delay in work time compared to the standard action data 41 (hereinafter referred to as the "cause action"). be. First, in step S31, the extraction unit 23 compares the standard motion data 41 and past motion data 43. Specifically, the extraction unit 23 compares, for example, the standard work time derived from the standard operation data 41 and the past work time derived from the past operation data 43 for each work process. Next, in step S33, the extraction unit 23 calculates the difference (delay time) between the standard work time and the past work time for each work process. Next, in step S35, the extraction unit 23 extracts a work process whose past work time is longer than the standard work time, and sets the operation included in the extracted work process as the cause operation to the display control unit 27. Output to.

The prediction process (step S5) is a process of predicting the time required to complete all work processes (hereinafter referred to as the predicted required time) when the worker P continues the work at the current progress (current work time). be. First, in step S51, the prediction unit 25 compares the target motion data 45 and the past motion data 43. Next, in step S53, the prediction unit 25 acquires preliminary data for calculating the predicted required time. The preliminary data is, for example, data regarding the time difference for each work process between the past work time and the current work time, and the progress of work up to the present time for all work processes. Next, in step S55, the prediction unit 25 predicts the estimated required time based on the preliminary data and outputs it to the display control unit 27.

The display control step (step S7) is a step in which worker support information is displayed on the display section 70 of the goggles 7 when the predicted required time exceeds the standard required time. First, in step S71, the display control unit 27 determines whether the predicted required time exceeds the standard required time. If the predicted required time exceeds the standard required time (step S71: Yes), the display control unit 27 causes the display unit 70 of the goggles 7 to display the worker support information in step S73. The worker support information is, for example, the causative action extracted in the extraction process (step S3), the difference between the predicted required time and the standard required time, and the delay time with respect to the current standard working time. The display control unit 27 causes the display unit 70 of the goggles 7 to display, for example, augmented reality in which a still image or a moving image is superimposed on the real world when a work process including the causative action is executed by an expert. Thereby, the display control unit 27 notifies the worker P who is working in a state that emphasizes the difference between the worker P's motion and the expert's motion. On the other hand, if the predicted required time is within the standard required time (step S71: No), the display control unit 27 does not display the worker support information on the display unit 70 of the goggles 7 in step S75.

After executing either step S73 or step S75, the display control unit 27 determines in step S77 whether all the work steps targeted for support have been completed. If it is determined that all the work steps have not been completed (step S77: No), the display control unit 27 returns to step S11 and repeatedly executes each step from step S11 to step S77. On the other hand, if it is determined that all the work steps have been completed (step S77: Yes), the display control unit 27 ends the worker support method in this embodiment.

According to the embodiment described above, when the worker P's motion is delayed compared to the standard motion of an expert, the worker support system 1 provides the worker P with the cause motion, etc. during the work. can be notified of worker support information. In other words, the worker support system 1 can notify the worker P of worker support information such as the causative action in a timely manner without interrupting the work of the worker P who is working. Therefore, the worker P can grasp problems in his/her own movement while working. Thereby, the worker P can timely improve the causative action.

Further, according to the embodiment described above, the skilled worker can check the moving image taken of the worker P during the work after the work is completed, or directly observe the worker P during the work, and It is possible to instruct the worker P without extracting problems in the operation and providing feedback to the worker P. Thereby, the burden on the expert who instructs the worker P can be reduced.

B. Other embodiments:
B-1. Other embodiment 1:
In the above embodiment, as shown in FIG. 1, the worker support system 1 acquires the motion of the worker P using an optical motion capture system, but the present invention is not limited to this. The worker support system 1, for example, inputs the photographic data 40 into a machine learning model stored in advance in the storage unit 4, and specifies each part and posture of the worker P's body by machine learning. The user P's actions may also be acquired.

B-2. Other embodiment 2:
In the above embodiment, as shown in FIG. 2, the worker support method ends when the worker P completes all the work steps, but the present invention is not limited to this. For example, when worker P repeats the same work process after completing all the work processes, even after completing all the work processes (step S77: Yes), the process returns to step S11 and steps S11 to S77 are executed. You may perform each process repeatedly. At this time, it is preferable not to store the photographic data 40 in the storage unit 4, for example, when an abnormality treatment is performed due to the stoppage of the production line during work. Thereby, it is possible to suppress a decrease in the accuracy of the worker support information.

The present disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in each form described in the summary column of the invention may be used to solve some or all of the above problems, or to achieve some of the above effects. Alternatively, in order to achieve all of the above, it is possible to perform appropriate replacements or combinations. Further, unless the technical feature is described as essential in this specification, it can be deleted as appropriate.

DESCRIPTION OF SYMBOLS 1... Worker support system, 2... CPU, 3... Communication unit, 4... Storage unit, 7... Goggles, 8... Suit, 10... Control device, 21... Acquisition unit, 23... Extraction unit, 25... Prediction unit, 27 ...Display control unit, 40...Photography data, 41...Standard operation data, 43...Past operation data, 45...Target operation data, 61-64...Camera, 70...Display section, 80...Marker member, I...Infrared rays, P... worker

Claims (1)

A worker support system,
a camera that takes video of the worker wearing a suit capable of capturing the worker's movements throughout the entire process of the work to be supported;
Goggles worn by the worker;
comprising a control device;
The control device includes:
a storage unit that stores standard motion data indicating the motions of an expert who performs the work in a standard manner; and past motion data indicating the motions of the worker when the worker executed the work in the past; ,
an acquisition unit that uses photographic data taken by the camera to acquire target motion data indicating the motion of the worker who is the target of the support;
an extraction unit that compares the standard motion data and the target motion data and extracts a motion that causes a delay in work time compared to the standard motion data;
a prediction unit that uses the past operation data to predict the time required to complete all the steps of the work targeted for support;
a display control unit that displays the extracted causative action on the goggles when the predicted required time exceeds the standard required time required to complete all steps of the work according to the standard action data; A worker support system with

Images