JP2015114676A - Operation confirmation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation confirmation system that allows an operator to easily confirm the details of an operation.SOLUTION: There is provided an operation confirmation system for an operator 13 to confirm the details of an operation to be performed by the operator, the system including a glove 11 that is worn by the operator 13 on his/her hand and has marks 21 and 22 having directivity from the back of the hand to the fingertips, an imaging unit 15 that photographs the marks 21 and 22 on the glove 11, and a control apparatus 18 that compares image data of the marks photographed by the imaging unit 15 with the details of the operation to perform quality determination.

Description

  The present invention relates to a work confirmation system capable of easily confirming work contents in, for example, a production line.

  The conventional work confirmation system confirms the work contents by attaching a mark to a tool, detecting the mark by image processing, and specifying the position direction of the tool (for example, refer to Patent Document 1). In another conventional work confirmation system, a light emitter is attached to a fastener, the light emitter is detected by image processing, and the work content is confirmed by means for specifying the position of the tightening work ( For example, see Patent Document 2).

Japanese Patent No. 5022045 Japanese Patent No. 4724211

  There are cases where tools of various shapes are used in assembling products on the production line. Conventional work confirmation systems may use nippers, pliers, scissors, brushes, pens, etc. in addition to drivers and wrench. There is also work that does not use tools. For example, such operations as connector connection, member fitting, and switch switching. In a method of attaching a mark or a light emitter to a tool as in a conventional work confirmation system, a preparation work is required to devise a different attachment method depending on the shape of the tool and verify that it can be detected by image processing. In addition, there are problems that it is difficult to attach a mark or a light emitter when using a small tool, and that it is not applicable when a tool is not used.

  The present invention has been made to solve the above-described problems, and is a work confirmation system capable of confirming work contents even for work using tools of various shapes in work and work without using tools. The purpose is to provide.

The work confirmation system of this invention is
In the work confirmation system that confirms the work content performed by the worker,
A glove having a mark attached to the hand by the operator and having directionality from the back of the hand to a fingertip;
An imaging device for photographing the mark of the globe;
And a control device that performs pass / fail judgment by comparing the image data of the mark photographed by the imaging device and the work content.

Since the work confirmation system of the present invention is configured as described above,
By photographing the glove mark, the work content can be easily confirmed.

It is the figure which showed the structure of the work confirmation system of Embodiment 1 of this invention. It is the figure which showed the structure of the glove of the work confirmation system shown in FIG. It is the figure which showed the operation | movement at the time of mounting | wearing with the glove shown in FIG. It is the figure which showed the operation | movement at the time of mounting | wearing with the glove shown in FIG. It is a block diagram which shows the structure of the control apparatus and indicator of the work confirmation system shown in FIG. It is a flowchart which shows operation | movement of the work confirmation system shown in FIG. It is a flowchart which shows operation | movement of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is a flowchart which shows operation | movement of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is the figure which showed the example of the image data of the work confirmation system shown in FIG. It is the figure which showed the example of the screen state of the work confirmation system shown in FIG. It is the figure which showed the other structure of the glove of the work confirmation system shown in FIG. It is the figure which showed the other structure of the glove of the work confirmation system shown in FIG. It is the flowchart which showed operation | movement of the work confirmation system of Embodiment 2 of this invention. It is the figure which showed the data of the standard deviation in the work confirmation system of Embodiment 3 of this invention. It is the flowchart which showed operation | movement of the work confirmation system of Embodiment 4 of this invention. It is the figure which showed the mark for individual difference correction | amendment of the work confirmation system of Embodiment 5 of this invention. It is the flowchart which showed operation | movement of the work confirmation system of Embodiment 5 of this invention. It is the figure which showed the structure of the work confirmation system of Embodiment 6 of this invention. It is the figure which showed the structure of the imaging device of the work confirmation system shown in FIG. It is the figure which showed the structure of the position reference mark of the work confirmation system shown in FIG. It is the figure which showed the structure of the work confirmation system of Embodiment 7 of this invention.

Embodiment 1 FIG.
Embodiments of the present invention will be described below. 1 is a perspective view showing a configuration of a work confirmation system according to Embodiment 1 of the present invention, FIG. 2 is a diagram showing a configuration of a glove of the work confirmation system shown in FIG. 1, and FIGS. 3 and 4 are FIG. FIG. 5 is a block diagram showing the configuration of the control device and the display of the work confirmation system shown in FIG. 1, and FIG. 6, FIG. 7 and FIG. FIG. 8 to FIG. 11, FIG. 13 and FIG. 14 show examples of image data of the work confirmation system shown in FIG. 1, and FIG. 15 shows the work shown in FIG. The figure which showed the example of the screen state of a confirmation system, FIG.16 and FIG.17 are the figures which showed the other structure of the glove of the work confirmation system shown in FIG.

  In the figure, this work confirmation system confirms the work contents performed by the worker 13. An image of the globe 11 worn by the operator 13 and having the mark 2 having the direction 2 from the back of the hand to the fingertip, the imaging device 15 for photographing the mark 2 of the globe 11, and the mark 2 photographed by the imaging device 15 A control device 18 that performs pass / fail determination by comparing data and work content, and a display 17 that displays the work content and pass / fail determination are provided.

  In the first embodiment, the work content performed by the worker 13 is, for example, a work performed by the worker 13 using his / her hand while the work object 12 is placed on the work desk 14. Is. It should be noted that this work content can correspond to either the case where the work is performed only by the hand of the worker 13 or the case where the worker 13 carries a tool or the like in his hand. In FIG. 1, as an example, a situation is shown in which the worker 13 is working on the work target 12 according to the work content displayed on the display unit 17 with the glove 11 attached to the hand. Is. The work object 12 is, for example, a device or an electronic circuit board.

  The imaging device 15 and the control device 18 are connected by an image signal input line 19, and the display device 17 and the control device 18 are connected by a display signal output line 20. The imaging device 15 is installed on the work desk 14 via the support 16 and uses, for example, a general imaging device such as a CCD camera or a CMOS camera. The installation position of the imaging device 15 has been shown as an example in which the imaging device 15 is installed above the work target 12. However, the imaging device 15 may be installed at any position where the mark 2 of the globe 11 can be photographed during the work. May be.

  The image signal input line 19 is configured by a cable line corresponding to an input / output interface between the imaging device 15 and the control device 18. The display signal output line 20 is configured by a cable line corresponding to an input / output interface between the display device 17 and the control device 18. The image signal input line 19 and the display signal output line 20 are configured by, for example, a USB cable, a VGA cable, an HDMI (registered trademark) cable, a coaxial cable, an Ethernet (registered trademark) cable, or the like.

  The display device 17 is installed on the work desk 14 and is a general display device such as a liquid crystal monitor, for example. In the first embodiment, the display device 17 has a touch panel input function and also serves as an operation input device. In the case where the display device 17 does not have an input function, it is possible to connect a keyboard, mouse, switch, or the like as an operation input device to the control device 18. Moreover, although the example which fixed the display device 17 to the support stand 16 was shown, it is not restricted to this, What is necessary is just to be able to confirm the display content from the operator 13, As another example, the display device 17 is work. It can be considered to be installed on the upper surface of the desk 14.

  The mark 2 of the globe 11 only needs to have directionality from the back of the hand of the worker 13 to the fingertip, and has this directionality using at least one of color, brightness, shape, and size. It is comprised as follows. Specifically, as shown in FIGS. 2, 3, and 4, when the operator 13 wears the glove 11 on the right hand 3, it corresponds to the pair of the thumb 23 and the index finger 24 on the back side of the right hand 3. The mark 22 corresponding to the pair of the mark 21, the middle finger 25, and the ring finger 26 is configured as a different color or pattern. Although the glove 11 for the right hand 3 is shown here, the left hand glove can also be dealt with by configuring each mark in the same manner, and therefore description thereof will be omitted as appropriate.

  The globe 11 may be made of any material. Here, the globe 11 is made of a dust-proof cloth, and is gray except for the mark 2 portion, the pair mark 21 of the thumb 23 and index finger 24 is yellow, and the mark 22 of the middle finger 25 and ring finger 26 is painted green. Suppose that As another example of the mark 2 of the glove 11, as shown in FIG. 16, a thumb mark 50 having a different color or pattern corresponding to each finger, an index finger mark 51, a middle finger mark 52, and a ring finger Mark 53, a little finger mark 54, and a mark 55 on the back side of the hand connecting these marks 50-54. In another example, as shown in FIG. 17, as the mark 2, marks 56, 57, 58, 59 for the thumb, index finger, middle finger, and ring finger having the same color and the same pattern corresponding to each finger. And there may be a form such as one having two marks 60 and 61 having different colors and patterns on the back side of the hand. According to the mark 2 as shown in FIG. 16 and FIG. 17, the mark 2 having directionality from the back of the hand to the fingertip is formed, and the processing can be performed in the same manner as the globe 11 shown in FIG. 2.

  Here, a glove 11 having an open fingertip is shown, but it may be a glove 11 covering the fingertip. Moreover, the shape of the mark 2 is mainly shown as a band shape, but a shape having directionality by connecting a shape having directionality such as a thin line or an ellipse or a shape having no directionality such as a plurality of circles or rectangles may be used. Good. Further, although the right hand glove 11 is shown, the left hand glove 11 may be used, and the right hand glove and the left hand glove may be used simultaneously.

  The control device 18 includes, for example, a personal computer, an industrial computer, a microcomputer board, a programmable logic controller, and the like, and includes the following parts. As shown in FIG. 5, the control device 18 includes a work definition unit 31, a work instruction unit 32, an image input unit 33, an image processing unit 34, a work confirmation unit 35, and a result recording unit 36. ing. The display unit 17 is configured to display a setting screen 41, a work instruction screen 42, and a work result screen 43.

  The work definition unit 31 stores work instruction data corresponding to each work content, and work position and work direction judgment condition data for judging each work content. Furthermore, the data input by the operation of the operator 13 from the setting screen 41 displayed on the display unit 17, for example, the work interruption button 98 for setting the work interruption shown in FIG. A setting function for setting the object change button 97 and the like, and an output function for transmitting work instruction data to the work instruction unit 32 are provided. The work instruction data includes work instruction diagrams, work instruction messages, timeout times, and detection range determination conditions. For example, JPEG format image file name in work instruction diagram, Unicode character string in work instruction message, floating-point number in seconds for timeout time, upper left and upper right of reference rectangle for detection range judgment condition A coordinate value indicating an allowable value and an allowable error value are shown.

  The work position determination condition data includes the coordinate values in the horizontal axis X and vertical axis Y directions and the allowable radius R of the allowable error circle in the image acquired by the imaging device 15. For example, X and Y are defined as absolute coordinate values in pixel units with the lower left corner of the image as the origin, and R is defined as a relative coordinate value in pixel units. The work direction determination condition data includes the angle θ on the plane of the image acquired by the imaging device 15 and the absolute value φ of the allowable error angle. For example, θ is defined as a value up to 360 degrees with the right direction of the horizontal axis of the image being 0 degrees, and φ is defined as a relative angle based on θ.

  The work instruction unit 32 acquires work instruction data from the work definition unit 31 and displays the work instruction data on the work instruction screen 42 of the display unit 17. The image input unit 33 inputs image data from the imaging device 15. For example, by using an API of driver software for processing an image, the image is input as a bitmap image. The image processing unit 34 acquires image data from the image input unit 33, detects the mark 2 of the globe 11 that appears in the image data, and detects the estimated position and estimated value of the mark 2.

  The work confirmation unit 35 acquires the estimated value of the estimated position and the estimated direction from the image processing unit 34, acquires the determination condition data of the work position and the working direction from the work definition unit 31, and estimates the estimated position and the estimated direction. Is determined to be within the allowable error of the determination condition data, and the determination result is displayed on the work result screen 43 of the display unit 17. The result recording unit 36 records work result data.

  Next, the operation of the work confirmation system of the first embodiment configured as described above will be described. In the present invention, the work confirmation system acquires an image of the appearance of a work situation with the imaging device 15, performs image processing with the control device 18, detects the mark 2 of the globe 11, and estimates the estimated position and the estimated direction. After the estimation, the pass / fail judgment is performed and the judgment result is output to the display unit 17. The detailed operation will be described below.

  In the first embodiment, a work instruction and work confirmation are repeated for a plurality of work contents to be performed on a certain work object 12, and a work result is determined. First, work instruction data is obtained from the work definition unit 31 and work instruction preparation is performed (step S1 in FIG. 6). Next, the work instruction data is transferred to the work instruction unit 32, and the work instruction unit 32 outputs it to the work instruction screen 42 to display the work instruction data (step S2 in FIG. 6).

  Next, the worker 13 confirms the work instruction data displayed on the work instruction screen 42 and performs work on the work object 12 with the glove 11 attached. Next, during the work of the operator 13, the imaging device 15 captures the globe 11, and inputs the captured image data to the image input unit 33 via the image signal input line 19 to acquire the image data. The image is input (step S3 in FIG. 6). Next, the image data is transferred to the image processing unit 34, and image processing for acquiring the estimated position and estimated direction value of the mark 2 of the globe 11 is performed (step S4 in FIG. 6).

  Next, the estimated values of the estimated position and the estimated direction are transferred to the work confirmation unit 35, and the work confirmation unit 35 compares the work determination data of the work position and the work direction to make a pass / fail decision to confirm the work (FIG. 6). Step S5). If the pass / fail determination is NG, it is determined whether it is time-out or interruption (step S6 in FIG. 6). And if it is time-out or interruption, a work interruption message will be displayed on the work result screen 43 of the display 17 (step S9 of FIG. 6).

  If the pass / fail determination is OK in step S5, it is determined whether or not all work has been completed in order to check whether or not there is any remaining work on the work object 12 (step S7 in FIG. 6). If there are remaining tasks and all the tasks have not been completed, the process returns to step S1 and the above-described processing is repeated. If all the work has been completed, a work completion message is displayed on the work result screen 43 of the display 17 (step S8 in FIG. 6). Next, the work result of this work is output and stored in the result recording unit 36 (step S10 in FIG. 6). If it is determined in step S6 that no timeout or interruption has occurred, the processing from step S3 is repeated as a retry.

  Specifically, the output of the work result in step S8 includes the work start date and time and the work end date and time of each work on the work object 12, work position data, work direction data, and pass / fail judgment results of the work position and work direction. The data is stored in the result recording unit 36 and a part thereof is displayed on the work result screen 43.

  The time-out is a time-out when the pass / fail judgment value remains NG during this repetitive processing and the time-out time included in the work instruction data is exceeded, otherwise there is no time-out. Also, the interruption means that the operator 13 presses the work interruption button 98 displayed on the work instruction screen 42 during the repetition process, and there is no interruption otherwise. The work interruption button 98 may use other input means such as a keyboard or a switch.

  Next, the contents of the image processing of the image processing unit 34 will be described based on the flowchart of FIG. First, according to the control of the image input (step S3 in FIG. 6) and the image processing (step S4 in FIG. 6) of the control device 18, the image data is acquired from the image input unit 33, and the noise included in the image data 70 is removed. Pre-processing is performed (step S11 in FIG. 7). This pre-processing is, for example, pre-processing in a general image processing method such as image data reduction / enlargement processing, blurring processing using a blur filter, processing using a median filter, and the like.

  Next, only the vicinity of the globe 11 is set as the detection range 74 (step S12 in FIG. 7). This is to prevent erroneous detection using the image data 70 at a place other than the globe 11 when the mark 2 is detected. Specifically, a case will be described in which image data 70 is acquired as shown in FIG. 8 when an image of 1600 pixels in the x-axis direction and 1200 pixels in the y-axis direction is acquired using a digital camera with 2 million pixels.

  FIG. 8 shows image data 70 of a work situation in which the connector 71 is inserted into the connector receiving portion 72 on the board 73 as the work object 12. The color of the part other than the mark 2 of the globe 11 is extracted from the image data 70 to obtain the outer shape of the globe 11, and considering the case where the mark 2 is present at the boundary of the outer shape, about 50 pixels above and below the outer shape. A rectangular area widened to the left and right is set as the detection range 74.

  Although the setting is made using the color of the globe 11 here, other features that can extract the outer shape of the globe 11 from the image data 70, such as brightness and pattern, may be used. If no detection error occurs even if the detection range 74 is not set, the process for setting the detection range 74 may be omitted and the entire image data 70 may be set as the detection range. Further, the detection range 74 is set by determining whether or not the outer dimension of the globe 11 is within the allowable reference, and thus an appropriate distance is not taken between the globe 11 and the imaging device 15, and the image data 70 is set. This also prevents the detection of the mark 2 from being continued in a situation where the globe 11 appears large or conversely small.

  Next, from the detection range 74, the mark 21 of the pair of the thumb 23 and the index finger 24 of the mark 2 and the mark 22 of the pair of the middle finger 25 and the ring finger 26 from the characteristics of the mark 2 such as each color or each pattern, 22 is extracted (step S13 in FIG. 7). FIG. 9 shows a situation in which this step S13 in FIG. 8 is performed and the pair mark 21 of the thumb 23 and the index finger 24 is extracted.

  For example, when extracting the mark 2 by color, a range of pixel values corresponding to the color of the mark 2 is set in advance as a determination condition, and the detection condition 74 within the detection range 74 is extracted. Can be implemented in a method. For example, when extracting the mark 2 by a pattern, an image piece corresponding to the pattern of the mark 2 is set in advance, and the image piece rotated by a predetermined angle at each coordinate in the detection range 74 is used. It can be implemented by a method of performing extraction by obtaining a matching part by performing a comparison process.

  Next, the shape of the mark 21 is extracted (step S14 in FIG. 7). Specifically, as in the example shown in FIG. 10, the binarization process and the thinning process, which are general image processing methods, are performed on the previously extracted mark 21 shown in FIG. A line segment 80 connecting the centers in the width direction of the band is obtained. Next, the estimated position and estimated value of the mark 21 are detected (step S15 in FIG. 7). Specifically, the line segment 80 of the mark 21 extracted in FIG. 10 is scanned and divided into four line segments 81, 82, 83, and 84 of equal length as shown in FIG. By obtaining a straight line approximation of the line segments 81 and 84 corresponding to both ends, the directed line segment 85 facing the fingertip having the shape of the mark 21 with reference to the midpoints 86 and 88 of the line segments 81 and 84, Each of 87 is obtained.

  If the mark 2 has the shape of the mark 2 shown in FIGS. 17 and 18 other than that shown in FIG. 2 and the estimated value of the estimated position and direction is to be obtained, it can be obtained by extracting the mark 2 shape. This can be done in the same manner by dividing the line segment into two line segments at each midpoint and obtaining the line segment far from the mark 2 on the back side of the hand, that is, the line segment on the fingertip side as a directed line segment.

  The midpoints 86 and 88 of the directed line segments 85 and 87 are estimated values of the estimated position, and the directions of the directed line segments 85 and 87 are estimated values of the estimated direction. Here, the estimated position is represented by a set of (x-axis pixel position, y-axis pixel position). The estimated direction is expressed as an angle with the x-axis direction set to 0 and counterclockwise as positive. The estimated values of the estimated position and the estimated direction thus obtained are transmitted to the work confirmation unit 35.

  Next, the pass / fail determination of the work confirmation unit 35 will be described based on the flowchart of FIG. First, the work confirmation unit 35 makes a determination by comparing the estimated value of the estimated position and the determination data of the work position acquired from the work definition unit 31 (step S21 in FIG. 12). Specifically, as shown in FIG. 13, the reference directed line segments 201 and 203 as the work position determination condition data and the allowable radius R of the work position are acquired from the work definition unit 31, and the respective reference directed directions are obtained. A circle having an allowable radius R at the working position centered on the midpoints 202 and 204 of the line segments 201 and 203 is defined as a determination condition 205 for the midpoint 202 of the reference directed line segment 201 and the midpoint 204 of the reference directed line segment 203. The determination condition 206 is as follows. If the midpoints 86 and 88 of each directed line segment 85 and 87 acquired as the estimated value of the estimated position are within the range of the respective determination conditions 205 and 206, the determination of the work position is OK and the next is performed. If not, the result is NG as NG (step S24 in FIG. 12).

More specifically, the estimated value of the estimated position is (312 and 513), that is, x = 312 and y = 513, the coordinate values X = 310 and Y = 510 of the work position determination condition data, and the allowable radius of allowable error If R = 10,
√ ((x−X) × (x−X) + (y−Y) × (y−Y)) ≈3.6 ≦ R
The determination is OK.

  Next, the work confirmation unit 35 compares the estimated value of the estimated direction with the determination condition data of the work direction acquired from the work definition unit 31 and performs determination (step S22 in FIG. 12). Specifically, as shown in FIG. 14, an allowable angle in the work direction is acquired from the work definition unit 31, and the allowable angle in the work direction is applied based on the respective reference directed line segments 207 and 208. The determination conditions 209 and 210 for the directions of the reference directed line segments 207 and 208 are set. If the direction of each of the directed line segments 85 and 87 acquired as the estimated value of the estimated direction is within the range of the determination conditions 209 and 210 of the direction of each reference directed line segment 207 and 208, the work direction If the determination is OK, it is OK / NG determination OK (step S23 in FIG. 12), otherwise NG is OK / NG determination NG (step S24 in FIG. 12).

More specifically, for example, if the reference value θ of the allowable angle in the work direction, the relative angle φ of the allowable error, and the estimated value of the estimated direction are p,
| P-θ | ≦ φ
The determination is OK.

  Then, the result of pass / fail determination of the work position and work direction is output to the work result screen 43 of the display 17. The work instruction screen 42 and the work result screen 43 are displayed on the display device 17 and confirmed by the operator 13. Each screen may be displayed as a separate screen or may be displayed as a single integrated screen. FIG. 15 shows an example in which the work instruction screen 42 and the work result screen 43 are integrated and displayed as one screen on the display unit 17. By displaying a work name 91, a work instruction diagram 92, and a work instruction message 93, the work content is transmitted to the worker 13.

  The work instruction diagram 92 illustrates the positional relationship between the member to be used and the work object 12 and the globe 11. Here, instead of the entire globe 11, only the thumb index finger pair mark 21 is displayed. Further, the work status and work results are communicated to the operator 13 by means of the assembly number target / result 94, image data 95, and work confirmation status display 96. In the image data 95, the image data input by the image input unit 33 is displayed. The work confirmation status display 96 displays a result indicating whether or not the work position and the work direction are output from the work confirmation unit 35 and a message indicating the status of the work.

  The work target change button 97 notifies the control device 18 that the type of the work target 12 is to be changed, transitions to a work target selection screen, and selects the type of the work target 12 to select the work target. Can be changed. This is because work instruction data corresponding to the type of work object 12 is stored in the work definition unit 31 in advance, and a work instruction required from the work definition unit 31 when the work object change button 97 is selected. Data is acquired and the display contents of the work name 91 and the work instruction diagram 92 are changed.

  Since the work confirmation system according to Embodiment 1 configured as described above can perform pass / fail judgment of work contents by using a glove mark worn by an operator, tools of various shapes are used in the assembly work. The work content can be confirmed even for work and work that does not use tools. Further, by confirming the work position and the work direction, it is possible to confirm the work contents and confirm improvement points.

Embodiment 2. FIG.
FIG. 18 is a flowchart showing the operation of the work confirmation unit of the work confirmation system according to Embodiment 2 of the present invention. In the second embodiment, the work confirmation unit 35 determines the time during which the determination of both the work position and the work direction is OK when the work position and the work direction are determined (the work position or the work direction). The work maintenance time, which is the time until both judgments become NG), is measured, and when it is between the lower limit value of the work maintenance time and the upper limit value of the work maintenance time preset in the work instruction data, the pass / fail judgment value Is OK, and if not, the pass / fail judgment value is NG.

  Therefore, in the work instruction data of the work definition unit 31 in the second embodiment, in addition to the work instruction diagram, the work instruction message, the timeout time, and the detection range determination condition, the lower limit value of the work maintenance time and the work maintenance time are set. The upper limit shall be included. For example, it is assumed that the lower limit value and the upper limit value of the work maintenance time are each expressed as a floating point number in seconds. Since the other configuration is the first embodiment, description thereof will be omitted as appropriate.

  Next, the operation of the work confirmation system in the second embodiment will be described. Since the operations other than those described below of the work confirmation unit 35 are the same as those in the above embodiments, the description thereof will be omitted as appropriate. The work confirmation unit 35 starts measuring the work maintenance time at the beginning of the process (step S31 in FIG. 18). Next, as in the first embodiment, the work position is determined (step S21 in FIG. 18) and the work direction is determined (step S22 in FIG. 18).

  In step S21 and step S22, if the determination is OK, the process returns to step S21 again and the above-described operation is repeated. If the determination at step S21 and step S22 is NG, it is determined that the work has been completed, and the measurement of the work maintenance time is completed (step S32 in FIG. 18). That is, the determination of the work position and the work direction is NG, which indicates that the work content has deviated from the work position and the work direction, and when the work content is completed, or when moving to the next work content, or A case where the work contents do not match is considered.

  Next, the pass / fail determination of the work maintenance time is performed from the minimum value of the work maintenance time and the maximum value of the work maintenance time stored in the work definition unit 31 (step S33 in FIG. 18). If the determination is OK, the work determination is OK. If the determination is NG, the work determination is NG and the work determination ends.

  According to the work confirmation system of the second embodiment configured as described above, the work maintenance time can be determined as well as the same effect as the first embodiment. If it is too short or too long, it is not regarded as work completion, and the work content can be confirmed more accurately.

Embodiment 3 FIG.
FIG. 19 is a diagram showing a display example displayed on the display of the work confirmation system according to Embodiment 3 of the present invention. In the third embodiment, the work confirmation of the work confirmation system can be performed in the same manner as in each of the above-described embodiments, so the description thereof will be omitted as appropriate. In the third embodiment, the work analysis unit using the work result What is provided is described. First, the work analysis unit acquires the start time of each work content and the end time of each work content for the plurality of work objects 12 from the work result data of the result recording unit 36. Next, each work time of each work content is obtained from the difference between each end time and each start time. Next, the standard deviation for a plurality of work objects 12 is calculated, and a bar graph is obtained with the time series order of each work as the horizontal axis and the standard deviation of the work time as the vertical axis, and is displayed on the display 17 (FIG. 19). (A)).

  In addition, the standard deviation of the estimated position of each work content and the standard deviation of the estimated direction are obtained, respectively, and the time series order of each work content is taken as the horizontal axis, and the standard deviation of the estimated position of each work content and the standard deviation of the estimated direction are calculated. A bar graph with the vertical axis is obtained and displayed on the display 17 (FIGS. 19B and 19C). By displaying such a graph, in the series of work, those that have a long work time, those that have a large standard deviation, and those that have a large standard deviation of the estimated position and direction should be extracted and improved. Can clarify the work.

  According to the work confirmation system of the third embodiment configured as described above, the standard deviation of the estimated position accumulated in the result recording unit, as well as the effects similar to those of the above-described embodiments can be obtained. The standard deviation of the direction and the standard deviation of the estimated time can be confirmed.

Embodiment 4 FIG.
In the fourth embodiment, sample work data selected as work data from the work result data stored in the result recording unit 36 in the control device 18 and registered in the work definition unit 31 as sample work data. An example including a selection unit will be described. In this case, at the time of image input (step S3 in FIG. 6), the input image data is included in the work result data of the result recording unit 36 and recorded. This image data is recorded at regular intervals, such as 0.5 seconds, and is recorded as a bitmap image or a moving image using various compression processes such as MJPEG. Since other configurations are the same as those of the above-described embodiments, the description thereof will be omitted as appropriate.

  Next, the operation of the work confirmation system according to Embodiment 4 configured as described above will be described. The operations other than those described below of the sample work selection unit are the same as those in each of the above embodiments, and thus the description thereof is omitted as appropriate. The sample work selection unit has the optimum work time length among the work result data of the result recording unit 36, the optimum work time standard deviation, or the estimated value of the estimated position and the estimated direction. A sample with the optimum standard deviation is selected as a sample operation (step S40 in FIG. 20). Next, the selected work result data is registered in the work definition unit 31 as sample work data (step S41 in FIG. 20).

  Next, the work instruction unit 32 acquires work instruction data and sample work data from the work definition unit 31 (step S42 in FIG. 20). Next, both the work instruction data and the sample work data are displayed alternately or simultaneously on the work instruction screen 42 of the display 17 (step S43 in FIG. 20). For example, the work instruction data is a work instruction diagram and a work instruction message, and the sample work data is a bitmap image or a moving image of the sample work data. Thereby, the sample work data can be shown to the worker 13.

  According to the work confirmation system of the fourth embodiment configured as described above, the sample work data can be displayed as well as the same effects as those of the above-described embodiments. The content can be improved.

Embodiment 5 FIG.
In each of the above embodiments, the case corresponding to the individual difference of the worker wearing the glove is not particularly shown, but in the present embodiment 5, the case corresponding to the individual difference of the worker will be described. FIG. 21 is a diagram showing the configuration of the individual difference correction mark of the work confirmation system in the fifth embodiment of the present invention, and FIG. 22 is a flowchart showing the operation of the work confirmation system in the fifth embodiment of the present invention. In the figure, the same parts as those in the above embodiments are denoted by the same reference numerals, and description thereof is omitted. The individual difference correction mark 100 is for performing processing for correcting individual differences. For example, the individual difference correction mark 100 has characteristics in color, luminance, and pattern, and can be easily detected by the image processing unit 34 by a general image processing method. There is something like that. For example, a red resin ball can be considered.

  In the fifth embodiment, the work definition unit 31 uses the work position reference value, the work position of the worker 13 as the work instruction data, in addition to the work instruction diagram, the work instruction message, the timeout time, and the detection range determination conditions. Including individual difference correction values. For example, the reference value of the work position and the individual difference correction value have a length in pixel units. The individual difference correction value is stored in a data structure of a dictionary format that holds a value corresponding to each key using a character string for identifying the worker 13 as a key, so that the individual difference correction value is obtained for each worker 13. To store.

  Next, the operation of the work confirmation system according to Embodiment 5 configured as described above will be described. The operations other than the operation including the reference value of the work position of the work content and the individual difference correction value of the work position of the worker 13 are the same as those in the above-described embodiments, and thus description thereof will be omitted as appropriate. First, the operator 13 holds the individual difference correction mark 100 with the thumb 23 and the index finger 24 as shown in FIG. Then, this state is photographed by the imaging device 15, and the correction image data obtained by photographing the mark 2 in a state where the operator 13 holds the individual difference correction mark 100 is acquired (step S50 in FIG. 22).

  Next, the length from the detection position of the individual difference correction mark 100 to the work position is obtained by the image processing unit 34, and the individual difference correction value h of the work position is detected from this length (step S52 in FIG. 22). The registration of the individual difference correction value h of the work position is performed at a stage before step S1 of the work instruction preparation described in the above embodiments. If it is already registered, if a character string for identifying the worker 13 is input at the start of the work from the next work, the individual difference correction value h of the work position registered in the past using the character string as a key is obtained. The registration work is omitted until the control device 18 is stopped again.

When determining the work content, after detecting the estimated value of the estimated position and the estimated direction through step S4, the work position reference value b and the individual difference correction value registered in the work definition unit 31 are detected. h is extracted to correct the estimated value. Specifically, the estimated direction angle t, the estimated position (x, y) of the work position, the individual difference correction value h of the work position, and the reference value b are performed as shown in the following formula and reflected estimation A correction value (X ′, Y ′) of the estimated position value is calculated.
X ′ = x− (h−b) × cos (t)
Y ′ = y− (h−b) × sin (t)
The corrected estimated values (X ′, Y ′) are used as estimated values (corrected values) of the estimated position and the estimated direction, and the operation is performed from the subsequent step S5 in the same manner as in each of the above embodiments, and determination is performed.

  According to the work confirmation system of the fifth embodiment configured as described above, the same effects as those of the above-described embodiments can be obtained, and the length of the thumb and index finger can be varied depending on the individual wearing the glove. Pass / fail judgment can be made regardless of individual differences.

Embodiment 6 FIG.
In each of the above embodiments, an example in which the imaging device 15 is fixed is shown. However, the present invention is not limited to this, and in the sixth embodiment, a case where the imaging device is made portable will be described. FIG. 23 is a diagram showing a configuration of a work confirmation system according to Embodiment 6 of the present invention, FIG. 24 is a diagram showing a configuration of an imaging device of the work confirmation system shown in FIG. 23, and FIG. 25 is a work confirmation shown in FIG. It is the figure which showed the position reference mark of the system. In the figure, the same parts as those in the above embodiments are denoted by the same reference numerals, and description thereof is omitted. When a portable imaging device is used in this way, as shown in FIG. 23, the position reference mark 103 is installed on the work object 12 or a jig for positioning the work object 12, and the image pickup apparatus 101 is used. The photographing position is determined by photographing the position reference mark 103.

  As the portable imaging device 101, a head-mounted imaging device 101 that can be worn by the operator 13 can be considered. The imaging device 101 includes a camera body 111, a camera lens 112, a belt 113, and buckles 114 and 115, and can be attached to the forehead or work cap of the worker 13. The position reference marks 103 can be displayed at the four corners of the work object 12 as shown in FIGS. 23 and 25, for example. FIG. 25 shows an enlarged view of the position reference marks 103a, 103b, 103c, and 103d displayed at the four corners in FIG.

  Next, the operation of the work confirmation system according to the sixth embodiment configured as described above will be described. Since the operations other than those described below of the image processing unit 34 are the same as those in the above embodiments, the description thereof will be omitted as appropriate. The image processing unit 34 detects the position reference marks 103a, 103b, 103c, and 103d that appear in the image signal, and estimates the estimated position and estimated direction as the position reference marks 103a, 103b, 103c, and 103d, and the globe. 11 from the positional relationship.

  According to the work confirmation system of the sixth embodiment configured as described above, the same effects as those of the above-described embodiments can be obtained, and the imaging device is portable. Even in a situation where the work position is hidden by an object or the like, an image signal can be obtained by changing the shooting position and direction. In particular, in the case of a head-mounted imaging device, an image signal close to the work situation seen by the operator can be obtained, so that it is considered that the work position is less likely to be hidden by an obstacle or the like.

Embodiment 7 FIG.
In each of the above-described embodiments, an example of one imaging device has been shown. However, the present invention is not limited to this, and a plurality of imaging devices are used so that each imaging device can capture a work situation. Is also possible. FIG. 26 is a diagram showing a configuration of a work confirmation system according to Embodiment 7 of the present invention. In the figure, the same parts as those in the above embodiments are denoted by the same reference numerals, and description thereof is omitted. The imaging device 15 includes a plurality of first imaging devices 15a and second imaging devices 15b.

  According to the work confirmation system of the seventh embodiment configured as described above, the same effects as those of the above-described embodiments can be obtained. In addition, the work position (globe The position of the mark, the direction indicated by the mark) can be detected. In this case, the work determination may be made in advance so as to use the image of the imaging device that is most desirable for position detection, and work determination data may be created.

  It should be noted that the present invention can be freely combined with each other within the scope of the invention, and each embodiment can be appropriately modified or omitted.

2 mark, 3 right hand, 11 glove, 12 work object, 13 worker,
14 working desks, 15 imaging devices, 15a first imaging device, 15b second imaging device,
16 Support base, 17 Display, 18 Control device, 19 Image signal input line,
20 display signal output line, 21 mark, 22 mark, 23 thumb, 24 index finger,
25 middle finger, 26 ring finger, 31 work definition part, 32 work instruction part, 33 image input part,
34 image processing unit, 35 work confirmation unit, 36 result recording unit, 41 setting screen,
42 work instruction screen, 43 work result screen, 50 mark, 51 mark,
52 mark, 53 mark, 54 mark, 55 mark, 56 mark,
57 mark, 58 mark, 59 mark, 60 mark, 61 mark,
70 image data, 71 connector, 72 connector receiving side, 73 substrate,
74 detection range, 81 line segment, 82 line segment, 83 line segment, 84 line segment, 85 directed line segment, 86 midpoint, 87 directed line segment, 88 midpoint, 91 work name, 92 work instruction diagram,
93 Work instruction message, 94 Assembly target / result, 95 Image signal,
96 Work confirmation status display, 97 Work object change button, 98 Work interrupt button,
100 personal difference correction mark, 101 imaging device, 102 wireless communication path,
103 position reference mark, 111 camera body, 112 camera lens,
113 belt, 114 buckle, 115 buckle, 121 position reference mark,
122 position reference mark, 123 position reference mark, 124 position reference mark,
131 imaging device, 132 imaging device, 201 reference directed line segment, 202 midpoint,
203 reference directed line segment, 204 midpoint, 205 judgment condition, 206 judgment condition,
207 Reference directed line segment, 208 Reference directed line segment, 209 determination condition, 210 determination condition.

Claims (11)

In the work confirmation system that confirms the work content performed by the worker,
A glove having a mark attached to the hand by the operator and having directionality from the back of the hand to a fingertip;
An imaging device for photographing the mark of the globe;
A work confirmation system comprising: a control device that performs pass / fail judgment by comparing image data of the mark photographed by the imaging device and the work content. The work confirmation system according to claim 1, wherein the mark of the globe is configured to have the directionality using at least one of color, brightness, shape, and size. The control device is
A work definition unit in which judgment position data for the work position and work direction of the work content for judging the work content is stored;
An image processing unit for detecting an estimated position and an estimated direction of the mark from the image data;
The work according to claim 1, further comprising: a work confirmation unit that compares the estimated position and the estimated direction with the determination condition data of the work definition unit to perform the pass / fail determination of the work content. Confirmation system. A result recording unit that accumulates work result data including the estimated position and the estimated direction detected by the image processing unit, and the pass / fail determination of the work confirmation unit;
The work confirmation system according to claim 3, further comprising a work analysis unit that analyzes a standard deviation of the estimated position and a standard deviation of the estimated direction accumulated in the result recording unit. The work definition unit has a time of the work content in the determination condition data,
The image processing unit detects an estimated time of the mark from the image data;
The work confirmation system according to claim 3, wherein the work confirmation unit performs the pass / fail judgment of the work content by adding the estimated time and comparing the determination condition data of the work definition unit. A result recording unit that accumulates work result data including the estimated position, the estimated direction, and the estimated time detected by the image processing unit, and the pass / fail determination of the work confirmation unit;
The work confirmation system according to claim 5, further comprising: a work analysis unit that analyzes a standard deviation of the estimated position, a standard deviation of the estimated direction, and a standard deviation of the estimated time accumulated in the result recording unit. The work confirmation system according to claim 4 or 6, further comprising a sample work selection unit that selects sample work data as a sample of the work content from the work result data of the result recording unit. The work definition unit has a reference value of the work position of the work content and an individual difference correction value of the work position of the worker,
The said operation confirmation part corrects the said estimated position and the said estimated direction using the said reference value and the said individual difference correction value, and performs the said pass / fail determination. Work confirmation system. The imaging device acquires correction image data obtained by photographing the mark in a state where the operator holds the individual difference correction mark,
The work confirmation system according to claim 8, wherein the image processing unit obtains the individual difference correction value from the correction image data and stores it in the work definition unit. The work confirmation system according to any one of claims 1 to 9, further comprising a display for displaying the work contents and the pass / fail judgment. The work confirmation system according to claim 1, wherein the imaging device is configured to be portable.

Images