JP2011164694A - Device and method for support of standard operation execution - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a standard operation execution support device for supporting an operator under operation to execute an operation following a standard operation procedure. <P>SOLUTION: A standard operation information memory 101 stores position information indicating the position of the hand of an operator following a standard operation procedure and time information indicating a timing in which the hand of the operator is present at the position indicated by the position information. An image analysis unit 102 generates position information indicating the position of the hand of the other operator. A difference calculation unit 103 calculates a difference between the position information stored in the standard operation information memory 101 and the position information generated by the image analysis unit 102. Also, the difference calculation unit 103 calculates a difference between the timing indicated by the time information stored in the standard operation information memory 101 and the timing in which the position information is generated by the image analysis unit 102. When the calculation result of the difference calculation unit 103 is larger than a predetermined value, a superimposition part 104 superimposes an image with a moving image, and makes a display unit display it. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a standard work execution support apparatus and a standard work execution support method for supporting the execution of work according to a standard work procedure by an operator.

  Companies such as the manufacturing industry and the service industry are provided with procedure manuals, work manuals, texts, photographs, etc. that describe the standard work procedures in an easy-to-understand manner so that workers can learn the standard work procedures that are correct work procedures. ing.

  Further, there is a case where the work performed by the worker is photographed to assist the execution of the work according to the standard work procedure.

  Patent Document 1 describes a system in which an administrator determines that the work is in an abnormal state when the work is photographed and a difference occurs between the work time and the standard work time.

  Patent Document 2 describes a system that compares an input pattern indicating a change in position of a predetermined part of an operator based on a moving image obtained by photographing an operator's work with a standard pattern.

JP 2008-59322 A (paragraphs 0014 to 0038, FIG. 1) JP 2008-108008 (paragraphs 0041 to 0113, FIG. 1)

  In the system described in Patent Document 1, since the manager confirms the work status of the worker who is actually working based on the stored video, the worker should improve during the work. It is difficult to convey the content.

  Further, in the system described in Patent Document 2, it is not considered to point out a difference from the standard work procedure during the work, and thus there is a possibility that an erroneous work is performed.

  Therefore, an object of the present invention is to provide a standard work execution support device and a standard work execution support method for supporting a worker to perform a work according to a standard work procedure when the worker is working. .

  The standard work execution support device according to the present invention includes position information indicating the position of the operator's hand in an image frame constituting a moving image of work according to the standard work procedure, and time information indicating timing at which the hand is at the position indicated by the position information. And a position information indicating the position of the hand of the other worker in the image frame constituting the moving image of the work performed by the other worker taken by the moving image photographing unit. The image analysis unit generates the image analysis unit, the position indicated by the position information corresponding to the time information stored in the standard work information storage unit, and the timing indicated by the time information in the moving image shot by the moving image shooting unit. The time information corresponding to the position information stored in the standard work information storage unit and the difference from the position indicated by the position information The difference calculation unit calculates a difference between the timing indicated by the image analysis unit and the timing at which the image analysis unit generates position information indicating a position matching the position indicated by the position information based on the moving image shot by the moving image shooting unit When the calculation result of the difference calculation unit is larger than a predetermined value, the image corresponding to the position information or time information stored in the standard work information storage unit is superimposed on the moving image shot by the moving image shooting unit. And a superimposing unit to be displayed on the display means.

  The standard work execution support method according to the present invention generates position information indicating the position of the hand of another worker in an image frame constituting a moving image of the work performed by another worker taken by the moving image photographing unit, and performs a standard work procedure. The standard work information storage unit that stores in advance the position information indicating the position of the operator's hand in the image frame constituting the moving image of the work according to the time information indicating the timing at which the hand is at the position indicated by the position information The difference between the position indicated by the position information corresponding to the time information stored in the position information and the position indicated by the position information generated at the timing indicated by the time information in the moving image captured by the moving image capturing unit, and the standard work information The timing indicated by the time information corresponding to the position information stored in the storage unit and the video shooting unit Based on the recorded moving image, a difference from the timing at which position information indicating a position matching the position indicated by the position information is generated is calculated, and when the calculation result is larger than a predetermined value, the moving image shooting unit captures the difference. An image corresponding to the position information or time information stored in the standard work information storage unit is superimposed on the moving image and displayed on the display means.

  ADVANTAGE OF THE INVENTION According to this invention, it can assist performing an operation | work according to a standard operation | work procedure, when an operator is working.

It is a block diagram which shows the structural example of the standard work execution assistance apparatus of embodiment of this invention. It is a flowchart which shows the operation | movement which produces | generates the moving image data based on the standard work procedure by the worker A who is an expert of work, the positional information, and the time information by the standard work execution assistance apparatus of embodiment of this invention. It is explanatory drawing which shows the example in which an image analysis part detects the position of the hand of the worker A. It is explanatory drawing which shows the example of the image superimposed on a moving image and displayed on a moving image display part. It is explanatory drawing which shows the example of the image superimposed and displayed on a moving image display part according to progress of the work according to the standard work procedure by the worker A. It is a flowchart which shows the operation | movement which the worker B instruct | indicates by the standard work execution assistance apparatus by this invention. It is explanatory drawing which shows the operation | work according to the standard operation procedure by the worker A, and the operation | work of the worker B. An example of an image displayed on the moving image display unit during the work process 1 according to the standard work procedure by the worker A and an example of an image displayed on the moving image display unit during the work process 1 by the worker B It is explanatory drawing shown. An example of an image displayed on the moving image display unit during the work step 2 according to the standard work procedure by the worker A and an example of an image displayed on the moving image display unit during the work step 2 by the worker B It is explanatory drawing shown. It is a block diagram which shows the outline | summary of this invention.

  An embodiment of a standard work execution support apparatus according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of a standard work execution support apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, the standard work execution support apparatus of the embodiment of the present invention includes an image analysis unit 11, a hand type recognition unit 12, a drawing reading unit 15, a moving image reading unit 16, an execution management unit 18, and a voice recognition unit 20. , And a character reading unit 22.

  The image analysis unit 11 has a function of analyzing the time of each image frame and the position of a predetermined location in the image of each image frame constituting the moving image in which the image input unit 10, which is a video camera having a shooting function, has photographed the work. Have. And it has the function which produces | generates and outputs the positional information which shows the position of an analysis result, and the time information which shows time. Note that the position of the predetermined portion in the image of each image frame is, for example, the position of the center of the operator's hand in the image of each image frame (hereinafter simply referred to as the hand position). The time of each image frame is the time elapsed from the start of work until the image of each image frame is taken.

  The hand-type recognizing unit 12 recognizes a gesture that is a predetermined hand movement by using a method of performing gesture recognition by measuring the movement of the limbs, information indicating the recognized gesture, and an image frame constituting the moving image. Among these, there is a function of outputting time information indicating the time of the image frame in which the gesture is photographed and position information indicating the position of the hand on which the gesture is performed based on a preset coordinate axis. Note that the moving image captured by the image input unit 10 is input to the hand recognition unit 12 via the image analysis unit 11, and the hand recognition unit 12 captures a gesture over a plurality of image frames in the moving image. For example, time information indicating the time of the image frame in which the gesture is first photographed is output.

Gesture recognition is described in the following references 1 and 2.
Reference 1: Japanese Patent No. 3144400 Reference 2: Japanese Patent No. 3863809

  The drawing storage unit 13 stores still image data corresponding to the gesture in advance. The still image data stored in the drawing storage unit 13 is read by the drawing reading unit 15. Note that the still image data stored in the drawing storage unit 13 is, for example, graphic still image data or text still image data.

  The moving image storage unit 14 temporarily stores moving image data of a moving image taken by the image input unit 10. The moving image storage unit 14 stores moving image data for displaying an image based on the still image data superimposed on the moving image captured by the image input unit 10 as a part of the moving image.

  The drawing reading unit 15 has a function of reading still image data corresponding to the gesture recognized by the hand recognition unit 12 from the drawing storage unit 13. The drawing reading unit 15 has a function of reading still image data from the drawing storage unit 13 in response to a request from the execution management unit 18.

  The moving image reading unit 16 reads the moving image data stored in the moving image storage unit 14 and causes the moving image display unit 17, which is a display for displaying moving images, to display the moving image based on the moving image data and the image based on the still image data. A function, that is, a function of superimposing and displaying an image based on still image data on a moving image based on moving image data is provided.

  The moving image display unit 17 is preferably a head-mounted display that is a display mounted near the head of the worker in order to confirm the standard work or the work status of the worker during the work. It may be a display fixed at a position that can be confirmed by a person during work.

  The voice recognition unit 20 has a function of recognizing the voice input to the voice input unit 19 that is a microphone and converting it into character information. That is, it has a function of generating character data based on the voice input to the voice input unit 19. In the character storage unit 21, time information indicating the timing when the voice is input to the voice input unit 19 and character information obtained by converting the voice input by the voice recognition unit 20 are stored in association with each other. The timing at which the voice is input to the voice input unit 19 is, for example, the time elapsed from when the work is started until the voice is input. The character storage unit 21 may store the position information generated by the image analysis unit 11 at the timing when the voice is input to the voice input unit 19 in association with the character information and the time information.

  The character reading unit 22 has a function of reading the character information stored in the character storage unit 21 by the voice recognition unit 20 and inputting it to the moving image reading unit 16. In addition, the character reading unit 22 has the time information stored in the character storage unit 21 associated with the character information and the image analysis unit 11 at the timing when the sound corresponding to the character information is input to the voice input unit 19. It has a function of inputting the generated position information indicating the position of the operator's hand to the execution management unit 18. The character reading unit 22 has a function of reading character information from the character storage unit 21 and inputting the character information to the moving image reading unit 16 in response to a request from the execution management unit 18.

  The execution management unit 18 has a function of storing position information and time information in each work according to the standard work procedure output by the hand type recognition unit 12, and a function of storing position information and time information output by the character reading unit 22. Have. Then, the position indicated by the position information according to the stored standard work procedure and the time indicated by the time information, and the position indicated by the position information generated by the image analysis unit 11 based on the moving image photographed by the image input unit 10 And the time indicated by the time information, and has a function of inputting the difference information indicating the comparison result to the drawing reading unit 15 and the character reading unit 22. The execution management unit 18 may store the position information and time information output from the hand recognition unit 12 and the position information and time information output from the character reading unit 22 separately.

  Next, the operation of the standard work execution support apparatus according to the present invention will be described with reference to the drawings. FIG. 2 is a flowchart showing an operation in which the standard work execution support apparatus according to the embodiment of the present invention generates moving image data, position information, and time information based on a work in accordance with a standard work procedure by a worker A who is a skilled worker. is there.

  An image input unit 10, which is a video camera, is mounted near the head of the worker A and photographs the situation in which the worker is working. The moving image data of the moving image captured by the image input unit 10 is stored in the moving image storage unit 14. The moving image reading unit 16 inputs the moving image data stored in the moving image storage unit 14 to the moving image display unit 17. The moving image display unit 17 displays a moving image based on the input moving image data.

  The image analysis unit 11 generates the position information and time information of the hand of the worker A based on the moving image captured by the image input unit 10 (step S10). FIG. 3 is an explanatory diagram illustrating an example in which the image analysis unit 11 detects the position of the hand of the worker A. In the example shown in FIG. 3, the upper left corner of the image is the base point (0, 0), the horizontal axis is the X coordinate (0 ≦ x ≦ 320), and the vertical axis is the Y coordinate (0 ≦ y ≦ 240). In the example illustrated in FIG. 3, the image analysis unit 11 detects that the hand position is (30, 200), and the position information indicating (30, 200) and the hand position are (30, 200). 200), time information indicating the time when it is detected is generated.

  For example, when it is necessary to give an instruction to the worker with a figure during work, when the worker A performs a predetermined gesture with his / her hand at the timing of drawing input while looking at the screen of the moving image display unit 17, The image analysis unit 11 inputs time information indicating the time of the image frame in which the gesture was photographed in the work and position information indicating the position coordinates of the hand where the gesture was performed in the image of the frame to the hand type recognition unit 12. The

  The hand type recognition unit 12 performs gesture recognition by measuring the three-dimensional coordinates of the hand (Y in step S11). The hand recognition unit 12 inputs the result of gesture recognition, position information, and time information to the drawing reading unit 15.

  The drawing reading unit 15 reads still image data corresponding to the result of the gesture recognition input by the hand recognition unit 12 from the drawing storage unit 13 (step S13). The drawing reading unit 15 may cause the drawing storage unit 13 to store the position information and time information input by the hand recognition unit 12 in association with the still image data stored in the drawing storage unit 13. Good. The moving image reading unit 16 superimposes an image based on the still image data read from the drawing storage unit 13 by the drawing reading unit 15 on the moving image based on the moving image data stored in the moving image storage unit 14 and displays it on the moving image display unit 17. (Step S14).

  FIG. 4 is an explanatory diagram illustrating an example of an image displayed on the moving image display unit 17 so as to be superimposed on the moving image. In the example shown in FIG. 4A, a gesture for raising the index finger by the worker A is performed. In the example shown in FIG. 4B, the drawing reading unit 15 reads still image data indicated by an arrow, which is still image data associated with the gesture of raising the index finger, from the drawing storage unit 13, and the moving image reading unit 16 The position (position information generated by the image analysis unit 11) corresponding to the position of the hand (specifically, for example, the position indicated by the position information generated by the image analysis unit 11) in the image of each image frame constituting the moving image. 4 is superimposed on the image based on the still image data at a position away from the position indicated by a predetermined distance in the predetermined direction and corresponding to the position of the fingertip in the example shown in FIG. 4B. This is displayed on the unit 17. The hand recognition unit 12 generates position information indicating the position of the fingertip when a gesture is performed, and the moving image reading unit 16 is based on the position information indicating the position of the fingertip generated by the hand recognition unit 12. Thus, the position at which the image based on the still image data is displayed on the moving image display unit 17 may be determined.

  The hand type recognition unit 12 inputs time information of an image frame in which the gesture is photographed and position information indicating the position of the hand where the gesture is performed to the execution management unit 18. The execution management unit 18 stores the input time information and position information in association with each other (step S17). The execution management unit 18 may store the time information and position information input by the hand type recognition unit 12 and the still image data read from the drawing storage unit 13 by the drawing reading unit 15 in association with each other.

  In addition, for example, when it is necessary to give an instruction by characters during work, the worker A looks at the screen of the moving image display unit 17 and at the timing of character input related to the work procedure or work precautions. A voice is input to the voice input unit 19 attached to the head of A.

  When speech is input to the speech input unit 19 (Y in step S12), the speech recognition unit 20 recognizes the speech input to the speech input unit 19 and converts it into character information (step S15). Is stored in association with the character information, the time information indicating the time corresponding to the timing when the voice is input to the voice input unit 19, and the position information indicating the position when the voice is input to the voice input unit 19. Store in the unit 22.

  Note that the time information indicating the time corresponding to the timing when the sound is input to the sound input unit 19 is, for example, the time of the image frame displayed on the moving image display unit 17 when the sound is input to the sound input unit 19. Information. The position information indicating the position when the voice is input to the voice input unit 19 is, for example, the position information generated by the image analysis unit 11 when the voice is input to the voice input unit 19.

  Then, the moving image reading unit 16 superimposes characters based on the character information on the moving image captured by the image input unit 10 and causes the moving image display unit 17 to display the superimposed characters (step S16). In the moving image displayed on the moving image display unit 17 by the moving image reading unit 16, the position where the character is superimposed and displayed is a position corresponding to the position indicated by the position information read from the character storage unit 21 by the character reading unit 22 (character A position that is a predetermined distance away from the position indicated by the position information read from the storage unit 21 in a predetermined direction, for example, a position corresponding to the position of the fingertip of the worker A). The hand recognition unit 12 generates position information indicating the position of the fingertip when a gesture is performed when a voice is input to the voice input unit 19, and the moving image reading unit 16 On the basis of the position information indicating the position of the fingertip generated by 12, the position where the character based on the character information is superimposed and displayed on the moving image display unit 17 may be determined.

  The character reading unit 22 inputs the time information and the position information stored in the character storage unit 21 to the execution management unit 18. The execution management unit 18 stores the input time information and position information in association with each other (step S17). When the character reading unit 22 transmits time information and position information to the execution management unit 18, the character reading unit 22 stores character information stored in the character storage unit 21 in association with the time information and position information. The execution management unit 18 may store the time information, the position information, and the character information input by the character reading unit 22 in association with each other.

  In the example shown in FIGS. 4C and 4D, since the voice “Inject three places of grease” is input to the voice input unit 19, the “three places of grease” converted into the character information by the voice recognition unit 20. "Inject" is displayed at a position corresponding to the position of the hand when the voice is input.

  FIG. 5 is an explanatory diagram illustrating an example of an image that is superimposed and displayed on the moving image display unit 17 in accordance with the progress of work according to the standard work procedure by the worker A. In the example illustrated in FIG. 5, the work process 1 starts at time “00:00” and ends at “0: 1: 30”. In addition, work process 2 starts at time “0:01:45” and ends at “0:03:00”. Then, at the start and end of the work process 1 and the work process 2, a gesture of raising the index finger is performed by the worker A, and an arrow that is an image corresponding to the gesture is displayed on the moving image display unit 17. ing.

  In the example shown in FIG. 5, the position of the hand at the start of work process 1 (time “00:00”) is (20, 50), and at time “00:00:30” At the position (30, 200), the work of injecting the grease is started, and the voice “injecting three places of grease” is inputted to the voice input section 19, so that “three places of grease is injected into the moving picture display section 17”. "Is superimposed and displayed. Then, the position of the hand at the end of the work process 1 (time “0:01:30”) is (120, 100).

  In the example shown in FIG. 5, the position of the hand at the start of work process 2 (time “0:01:45”) is (50, 50), and at time “0:02:15” Since the cutting operation is started at the position (200, 200) and the voice “cutting the right tip” is input to the voice input unit 19, the “right tip cutting” is displayed on the moving image display unit 17. Characters are displayed. The position of the hand at the end of the work process 2 (time “0:03:00”) is (300, 200).

  Next, an operation in which the standard work execution support apparatus according to the present invention instructs the worker based on the generated position information and time information will be described with reference to the drawings. FIG. 6 is a flowchart showing an operation instructed to the worker B by the standard work execution support apparatus according to the present invention.

  An image input unit 10, which is a video camera, is mounted near the head of the worker B and photographs the situation in which the worker is working. The moving image data of the moving image captured by the image input unit 10 is stored in the moving image storage unit 14. The moving image reading unit 16 inputs the moving image captured by the image input unit 10 to the moving image display unit 17. The moving image display unit 17 displays the input moving image.

  The image analysis unit 11 generates position information and time information of the hand of the worker B based on the moving image captured by the image input unit 10 (step S20).

  The hand type recognition unit 12 inputs the position information and time information generated by the image analysis unit 11 to the execution management unit 18. The execution management unit 18 compares the position information and time information input by the hand recognition unit 12 with the stored position information and time information (step S21), and calculates a difference.

  Specifically, for example, when position information indicating a position matching the position indicated by the stored position information is input, the execution management unit 18 stores the position information in association with the stored position information. The difference between the time indicated by the current time information and the time indicated by the input time information is calculated. Further, for example, when time information indicating a time that matches a time indicated by the stored time information is input, the execution management unit 18 stores the position in association with the stored time information. The difference between the position indicated by the information and the position indicated by the input position information is calculated.

  Then, when the calculated difference exceeds a predetermined value, the execution management unit 18 inputs the difference information indicating the difference to the drawing reading unit 15 and the character reading unit 22 and displays the superimposed information on the moving image. It requests to read data according to The execution management unit 18 may input the position information and time information stored for comparison in the process of step S21 together with the difference information to the drawing reading unit 15 and the character reading unit 22.

  Further, the execution management unit 18 may determine the information input destination according to the output source when the position information and the time information are distinguished and stored according to the output source. Specifically, for example, when the position information and time information compared with the position information and time information input by the hand type recognition unit 12 in the process of step S21 are input and stored by the hand type recognition unit 12 If the information is input to the drawing reading unit 15 and is input and stored by the character reading unit 22, the information may be determined to be input to the character reading unit 22.

  The drawing reading unit 15 reads still image data from the drawing storage unit 13. The character reading unit 22 reads character information from the character storage unit 21 (step S23). Note that the still image data read by the drawing reading unit 15 from the drawing storage unit 13 is, for example, still image data associated with position information and time information input together with the difference information. Moreover, the character information which the character reading part 22 reads from the character memory | storage part 21 is character information matched with the positional information and time information input with difference information, for example.

  The moving image reading unit 16 superimposes the still image data read from the drawing storage unit 13 by the drawing reading unit 15 or the character information based on the character information read from the character storage unit 21 by the character reading unit 22 on the moving image display unit 17. It is displayed (step S24).

  When the work is completed (Y in step S25), the execution management unit 18 determines that the standard work has been executed when the difference is within the standard range based on the result of the comparison in step S21. Is outside the standard range, it is determined that the standard work has not been executed (step S26).

  FIG. 7 is an explanatory diagram showing work according to the standard work procedure by the worker A and work by the worker B. FIG. 8 illustrates an example of an image displayed on the moving image display unit 17 during the work process 1 according to the standard work procedure performed by the worker A, and is displayed on the moving image display unit 17 during the work process 1 performed by the worker B. It is explanatory drawing which shows the example of the image to be. FIG. 9 shows an example of an image displayed on the moving image display unit during the work process 2 according to the standard work procedure by the worker A, and an image displayed on the moving image display unit 17 during the work process 2 by the worker B. It is explanatory drawing which shows the example of an image.

  In the example shown in FIG. 7 and FIG. 8A, it is shown that the worker A starts the grease injection at the position (30,200) at the time “0:00:30”. The worker B starts the grease injection at the position (30,200) at time “0:00:35”.

  The execution management unit 18 calculates a difference between the position and time indicated by the position information and time information stored based on the work of the worker A and the position and time of the work content of the worker B.

  Specifically, the execution management unit 18 indicates the position (30, 200) indicated by the position information stored based on the work of the worker A and the position information input based on the work of the worker B. The difference (0, 0) from the position (30, 200) to be calculated is calculated. In addition, the execution management unit 18 is indicated by the time “0:00:30” indicated by the time information stored based on the work of the worker A and the time information input based on the work of the worker B. The difference “0:00:05” from the time “0:00:35” is calculated.

  Since the execution management unit 18 exceeds 3 seconds, which is 10% of 30 seconds which is the work start time of the grease injection indicated by the time information stored based on the work of the worker A, the time difference is as follows. The stored position information and time information are input to the drawing reading unit 15 and the character reading unit 22 together with the difference information indicating the calculated position difference (0, 0) and the time difference “0:00:05”. .

  Since the character reading unit 22 indicates that the time difference is generated by the difference information, the character reading unit 22 reads the character information stored in the character storage unit 21 in association with the input time information. The moving image reading unit 16 causes the moving image display unit 17 to superimpose the character “three places of grease” on the moving image based on the character information read from the character storage unit 21 by the character reading unit 22, and pays attention to the operator B. Arouse. That is, the characters “Inject three places of grease” are displayed on the moving image display unit 17 at time “0:00:35” (see FIG. 8B).

  In the example shown in FIG. 7 and FIG. 8C, it is indicated that the worker A has finished the work of the work process 1 at the position (120, 100) at time “0:01:30”. On the other hand, the worker B finishes the work of the work process 1 at the position (110, 100) at time “0:01:30”.

  The execution management unit 18 calculates a difference between the position and time indicated by the position information and time information stored based on the work of the worker A and the position and time of the work content of the worker B.

  Specifically, the execution management unit 18 indicates the position (120, 100) indicated by the position information stored based on the work of the worker A and the position information input based on the work of the worker B. The difference (10, 0) from the position (110, 100) to be calculated is calculated. The execution management unit 18 is indicated by the time “0: 0: 30” indicated by the time information stored based on the work of the worker A and the time information input based on the work of the worker B. The difference “0:00: 00” from the time “0: 01:30” is calculated.

  The execution management unit 18 indicates that the Y-axis position difference is 0, the X-axis position difference “10” does not exceed 10% of the X-axis maximum value “320”, and the time difference is “0”. : 00: 00 ", it is determined that the work according to the standard work procedure has been executed. Then, the execution management unit 18 superimposes and displays an image (“◯” in the example illustrated in FIG. 7) indicating that the work according to the standard work procedure has been performed on the moving image displayed on the moving image display unit 17.

  In the example shown in FIG. 7 and FIG. 9A, it is shown that the worker A starts work at the position “50, 50” at time “0:01:45”. Worker B starts work at time (0:01:45) at position (50, 100).

  The execution management unit 18 calculates a difference between the position and time indicated by the position information and time information stored based on the work of the worker A and the position and time of the work content of the worker B.

  Specifically, the execution management unit 18 indicates the position (50, 50) indicated by the position information stored based on the work of the worker A and the position information input based on the work of the worker B. The difference (0, 50) from the position (50, 100) to be measured is calculated. Further, the execution management unit 18 is indicated by the time “0:01:45” indicated by the time information stored based on the work of the worker A and the time information input based on the work of the worker B. A difference “0:00: 00” from the time “0: 01: 45” is calculated.

  The execution management unit 18 calculates the difference in the X-axis position because the difference in the X-axis position is 0, but the Y-axis position difference “50” exceeds 10% of the maximum Y-axis value “240”. The stored position information and time information are input to the drawing reading unit 15 and the character reading unit 22 together with the difference information indicating (0, 50) and the time difference “0:00:00”.

  The drawing reading unit 15 reads the still image data stored in the drawing storage unit 13 in association with the input position information because the difference information indicates that a difference in position has occurred. The moving image reading unit 16 converts an arrow, which is a figure based on the still image data read from the drawing storage unit 13 by the drawing reading unit 15, into a moving image at a position indicated by the position information stored in association with the still image data. The superimposed images are displayed on the moving image display unit 17 to alert the worker B. That is, an arrow is displayed on the moving image display unit 17 at time “0:01:45” (see FIG. 9A).

  In the example shown in FIG. 7 and FIG. 9B, it is shown that the worker A starts cutting at the position (200, 200) at the time “0:02:15”. Worker B starts cutting at time (0:02:25) at position (100, 100).

  The execution management unit 18 calculates a difference between the position and time indicated by the position information and time information stored based on the work of the worker A and the position and time of the work content of the worker B.

  Specifically, the execution management unit 18 indicates the position (200, 200) indicated by the position information stored based on the work of the worker A and the position information input based on the work of the worker B. The difference (100, 100) from the position (100, 100) to be calculated is calculated. In addition, the execution management unit 18 is indicated by the time “0:02:15” indicated by the time information stored based on the work of the worker A and the time information input based on the work of the worker B. The difference “0:00:10” from the time “0:02:25” is calculated.

  The execution management unit 18 determines that the X-axis position difference “100” exceeds 10% of the X-axis maximum value “320”, and the Y-axis position difference “100” is the maximum of the Y-axis maximum value “240”. 10% of the value, which is 10% of 30 seconds, which is the time from the start of the work process 2 calculated based on the time information in which the time difference is stored until the cutting work is started 3 Since it exceeds the second, the position information and time information stored together with the difference information indicating the calculated position difference (100, 100) and the time difference “0:00:10” are displayed in the drawing and reading unit 15 and Input to the character reading unit 22.

  The character reading unit 22 indicates that a difference in position and a difference in time are generated by the difference information, and is stored in the character storage unit 21 in association with the input position information and time information. Read character information. The moving image reading unit 16 superimposes the character “cut right end portion” based on the character information read out from the character storage unit 21 by the character reading unit 22 on the moving image and displays it on the moving image display unit 17 to pay attention to the operator B. Arouse. That is, at the time “0:02:25”, the characters “cut the right end” are displayed on the moving image display unit 17 (see FIG. 9B).

  In the example shown in FIG. 7 and FIG. 9C, it is indicated that the worker A has finished the work of the work process 2 at the position (300, 200) at the time “0:03:00”. On the other hand, the worker finishes the work of work process 2 at time “0:03:10” at the position (300, 200).

  The execution management unit 18 calculates a difference between the position and time indicated by the position information and time information stored based on the work of the worker A and the position and time of the work content of the worker B.

  Specifically, the execution management unit 18 indicates the position (300, 200) indicated by the position information stored based on the work of the worker A and the position information input based on the work of the worker B. The difference (0, 0) from the position (300, 200) to be measured is calculated. In addition, the execution management unit 18 is indicated by the time “0:03:00” indicated by the time information stored based on the work of the worker A and the time information input based on the work of the worker B. The difference “0:00:10” from the time “0:03:10” is calculated.

  The execution management unit 18 has a position difference of (0, 0), and the time difference is the time until the work end of the work process 2 indicated by the time information stored based on the work of the worker A. Since 7.5 seconds, which is 10% of 75 seconds, is exceeded, it is determined that the operation according to the standard operation procedure has not been executed. Then, the execution management unit 18 superimposes and displays an image (“Δ” in the example illustrated in FIG. 7) indicating that the operation according to the standard operation procedure has not been performed on the moving image displayed on the moving image display unit 17.

  According to this embodiment, since the figure and the character to be displayed on the moving image display unit 17 are input during the work in accordance with the standard work procedure by the worker A who is an expert, it is easy without taking time and effort. Movie data can be created.

  Further, when it is detected that the difference between the work content and time in accordance with the standard work procedure and the work content and time by the worker B exceeds a predetermined value during the work by the worker B, a figure or a character is displayed. Since it is displayed on the moving image display unit 17, it is possible to alert the operator B to prompt confirmation of the work contents and prevent work mistakes.

  Further, by recording the moving image data obtained by photographing the work by the worker A who is an expert in the moving image storage unit 14, the moving image data can be used for work progress management and work analysis.

  In a general work management system, when checking the progress of work, the worker must input information into the system at the start of work or at the end of work, but information is input by gestures. Therefore, it is possible to prevent work leakage due to forgetting to input information and improve work accuracy.

  Next, the outline of the present invention will be described. FIG. 10 is a block diagram showing an outline of the present invention. As shown in FIG. 10, a standard work execution support apparatus 100 according to the present invention includes a standard work information storage unit 101 (corresponding to the execution management unit 18 shown in FIG. 1) and an image analysis unit 102 (image analysis unit 11 shown in FIG. 1). ), A difference calculating unit 103 (corresponding to the execution managing unit 18 shown in FIG. 1), and a superimposing unit 104 (corresponding to the drawing reading unit 15, the character reading unit 22, and the moving image reading unit 16 shown in FIG. 1). .

  The standard work information storage unit 101 includes position information indicating the position of the worker's hand in the image frame constituting the moving image of the work following the standard work procedure, and time information indicating the timing at which the hand is at the position indicated by the position information. Pre-correlate and store.

  The image analysis unit 102 is a position indicating the position of the other worker's hand in an image frame constituting a moving image of the work performed by the other worker, which is shot by the moving image shooting unit (corresponding to the image input unit 10 shown in FIG. 1). Generate information.

  The difference calculation unit 103 is an image analysis unit 102 at a position indicated by the position information corresponding to the time information stored in the standard work information storage unit 101 and the timing indicated by the time information in the moving image shot by the moving image shooting unit. The difference from the position indicated by the position information generated by is calculated.

  Further, the difference calculation unit 103 is configured to display the position indicated by the position information based on the timing indicated by the time information corresponding to the position information stored in the standard work information storage unit 101 and the moving image shot by the moving image shooting unit. The difference from the timing at which the image analysis unit 102 generated the position information indicating the position that matches is calculated.

  When the calculation result of the difference calculation unit 103 is larger than a predetermined value, the superimposing unit 104 corresponds to the position information or time information stored in the standard work information storage unit 101 on the moving image shot by the moving image shooting unit. The image to be displayed is superimposed and displayed on the display means (corresponding to the moving image display unit 17 shown in FIG. 1).

  According to such a configuration, during the work by the worker, when it is detected that the difference between the work content and time according to the standard work procedure and the work content and time by the worker exceeds a predetermined value, Since the image is superimposed and displayed on the display means, the operator can be alerted to confirm the work contents, and work mistakes can be prevented.

  Further, in the above embodiment, standard work execution support devices as shown in the following (1) to (4) are also disclosed.

(1) The superimposition unit 104 stores still image data corresponding to the position information and time information stored in the standard work information storage unit 101 when the calculation result of the difference calculation unit 103 is larger than a predetermined value. Still image data corresponding to position information or time information is read from a still image data storage unit (corresponding to the drawing storage unit 13 shown in FIG. 1), and a still image based on the still image data is captured by the moving image shooting unit. A standard work execution support device that superimposes and displays on a display means.

(2) A gesture recognition unit (corresponding to the hand type recognition unit 12 shown in FIG. 1) for recognizing a predetermined gesture of the worker in a moving image of work according to the standard work procedure, position information generated by the image analysis unit 102, A gesture information registration unit (corresponding to the hand type recognition unit 12 shown in FIG. 1) that stores time information indicating the timing at which a predetermined gesture is photographed in the standard work information storage unit 101, and a still image data storage unit In FIG. 3, still image data corresponding to a predetermined gesture by the worker is stored in advance, and the image analysis unit 102 indicates a position where the predetermined gesture is performed in an image frame constituting a moving image of the work according to the standard work procedure. The position information is generated, and the gesture information registration unit associates the position information and the time information with the still image data associated with the predetermined gesture, and Standard work execution support device to be stored in the data storage unit.

(3) The superimposing unit 104 stores character data corresponding to position information and time information stored in the standard work information storage unit 101 when the calculation result of the difference calculation unit 103 is larger than a predetermined value. Character data corresponding to position information or time information is read from a data storage unit (corresponding to the character storage unit 21 shown in FIG. 1), and characters based on the character data are superimposed and displayed on the moving image shot by the moving image shooting unit. Standard work execution support device displayed on the means.

(4) A voice recognition unit (speech shown in FIG. 1) that recognizes input voice, generates character data based on the voice, and time information indicating the timing when the voice is input and stores it in the character data storage unit. A voice information registration unit (character reading unit 22 shown in FIG. 1) that stores the position information generated by the image analysis unit 102 and the time information generated by the voice recognition unit in the standard work information storage unit 101. The image analysis unit 102 generates position information indicating the position of the operator's hand at the timing in the image frame constituting the moving image of the work according to the standard work procedure, and the audio information registration unit A standard work execution support apparatus that stores information in a character data storage unit in association with character data and time information.

  The present invention can be applied to a system for supporting execution of work according to a standard work procedure by an operator during the work.

DESCRIPTION OF SYMBOLS 10 Image input part 11,102 Image analysis part 12 Hand type recognition part 13 Drawing memory | storage part 14 Movie storage part 15 Drawing reading part 16 Movie reading part 17 Movie display part 18 Execution management part 19 Voice input part 20 Voice recognition part 21 Character storage Unit 22 character reading unit 100 standard work execution support device 101 standard work information storage unit 103 difference calculation unit 104 superposition unit

Claims (10)

The position information indicating the position of the worker's hand in the image frame constituting the moving image of the work following the standard work procedure and the time information indicating the timing at which the hand is at the position indicated by the position information are stored in association with each other. A standard work information storage unit;
An image analysis unit that generates position information indicating a position of the hand of the other worker in an image frame constituting a moving image of the work performed by the other worker taken by the moving image shooting unit;
The image analysis unit generates the position indicated by the position information corresponding to the time information stored in the standard work information storage unit and the timing indicated by the time information in the moving image captured by the moving image capturing unit. Based on the difference between the position indicated by the position information, the timing indicated by the time information corresponding to the position information stored in the standard work information storage unit, and the moving image shot by the moving image shooting unit, A difference calculation unit that calculates a difference from the timing at which the image analysis unit generates position information indicating a position that matches the position indicated by the position information;
When the calculation result of the difference calculation unit is larger than a predetermined value, an image corresponding to the position information or time information stored in the standard work information storage unit is added to the moving image shot by the moving image shooting unit. A standard work execution support apparatus, comprising: a superimposing unit that superimposes and displays on a display means. When the calculation result of the difference calculation unit is larger than a predetermined value, the superimposition unit is configured to store a still image data corresponding to the position information and time information stored in the standard work information storage unit. The standard according to claim 1, wherein the still image data corresponding to the position information or the time information is read, and a still image based on the still image data is superimposed on the moving image captured by the moving image capturing unit and displayed on the display means. Work execution support device. A gesture recognizing unit for recognizing a predetermined gesture of the worker in the video of the work according to the standard work procedure;
A gesture information registration unit that stores position information generated by the image analysis unit and time information indicating timing at which the predetermined gesture is captured in the moving image in a standard work information storage unit;
In the still image data storage unit, still image data corresponding to the predetermined gesture by the operator is stored in advance,
The image analysis unit generates position information indicating a position where the predetermined gesture is performed in an image frame constituting a moving image of work according to a standard work procedure,
The standard work execution according to claim 2, wherein the gesture information registration unit stores the position information and the time information in association with the still image data associated with the predetermined gesture in the still image data storage unit. Support device. When the calculation result of the difference calculation unit is larger than a predetermined value, the superimposition unit stores the position data from the character data storage unit that stores character data corresponding to the position information and time information stored in the standard work information storage unit. The character data corresponding to the information or the time information is read out, and the character based on the character data is superimposed on the moving image photographed by the moving image photographing unit and displayed on the display means. The standard work execution support apparatus according to Item 1. A voice recognition unit that recognizes an input voice, generates character data based on the voice, and time information indicating a timing at which the voice is input, and stores the generated time information in a character data storage unit;
A voice information registration unit that stores position information generated by the image analysis unit and time information generated by the voice recognition unit in a standard work information storage unit;
The image analysis unit generates position information indicating the position of the operator's hand at the timing in an image frame constituting a moving image of work according to a standard work procedure,
The standard work execution support apparatus according to claim 4, wherein the voice information registration unit stores the position information in the character data storage unit in association with the character data and the time information. Generating position information indicating a position of the hand of the other worker in an image frame constituting a moving image of work by another worker photographed by the moving image photographing unit;
The position information indicating the position of the worker's hand in the image frame constituting the moving image of the work following the standard work procedure and the time information indicating the timing at which the hand is at the position indicated by the position information are stored in association with each other. The position indicated by the position information corresponding to the time information stored in the standard work information storage unit, and the position indicated by the position information generated at the timing indicated by the time information in the moving image captured by the moving image capturing unit And the position indicated by the position information based on the timing indicated by the time information corresponding to the position information stored in the standard work information storage unit and the moving image shot by the moving image shooting unit Calculate the difference from the timing that generated the position information indicating the position that matches
When the calculation result is larger than a predetermined value, a display unit that superimposes an image corresponding to position information or time information stored in the standard work information storage unit on the moving image captured by the moving image capturing unit A standard work execution support method characterized by being displayed on the screen. When the difference calculation result is larger than a predetermined value, the position information or the time from the still image data storage unit that stores the still image data corresponding to the position information and time information stored in the standard work information storage unit The standard work execution support method according to claim 6, wherein still image data corresponding to the information is read, and a still image based on the still image data is superimposed on a moving image captured by the moving image capturing unit and displayed on a display unit. In the still image data storage unit, still image data corresponding to a predetermined gesture by the operator is stored in advance,
Recognizing the predetermined gesture of the worker in the video of the work following the standard work procedure;
Generating position information indicating a position where the predetermined gesture is performed;
Storing the position information and time information indicating a timing at which the predetermined gesture is captured in the moving image in a standard work information storage unit;
The standard work execution support method according to claim 7, wherein the position information and the time information are associated with each other and stored in the still image data storage unit in the still image data associated with the predetermined gesture. When the difference calculation result is larger than a predetermined value, the position information or the time information is stored in the position information or the time information from the character data storage unit that stores character data corresponding to the position information and time information stored in the standard work information storage unit. The standard work according to any one of claims 6 to 8, wherein the corresponding character data is read out, and the character based on the character data is superimposed on the moving image photographed by the moving image photographing unit and displayed on the display means. Execution support method. Recognizing the input voice, generating character data based on the voice and time information indicating the timing at which the voice is input and storing it in the character data storage unit,
Store the position information indicating the position of the hand of the worker who performs the work according to the standard work procedure at the timing and the time information in the standard work information storage unit,
The standard work execution support method according to claim 9, wherein the position information is stored in the character data storage unit in association with the character data and the time information.

Images