JP2015099022A - Operation confirmation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation confirmation system configured to confirm work operation of a worker by means of a simple method, without reducing work efficiency of the worker or increasing cost.SOLUTION: An operation confirmation system includes an imaging apparatus 10 which images work content, an image processing/work determination apparatus 20 which performs image processing and determines whether a work has been successful or not, and a notification apparatus 30 which notifies of a result of the determination. The image processing/work determination apparatus 20 performs edge extraction processing and symbolization processing on image data, and compares the processed image data with reference image data, to determine whether the work content is appropriate or not.

Description

  The present invention relates to an operation confirmation system for confirming whether an assembly operation by an operator is accurately performed in various production lines.

  Conventionally, an operator has prevented the occurrence of work mistakes by looking at the printed work instruction content. Or, a work mistake has been confirmed by displaying the work content on a monitor.

  Moreover, the use of the lamps on the installation shelves of the used parts was used to prevent erroneous use of the parts. In addition, a motion sensor is installed at the outlet of the used parts shelf, and the presence or absence of parts acquisition is determined by detecting the movement of the sensor.

  In addition, for example, techniques related to Patent Documents 1 and 2 already exist as conventional techniques related to the work confirmation system.

  In the technique according to Patent Document 1, a marker is attached to a position that moves when an operator performs work, and standard position information indicating the position of the marker in time series order when the work is correctly performed is stored in advance. Then, the success or failure of the work is confirmed using the movement change of the marker photographed by the camera and the standard position information. In the technique according to Patent Document 2, the marker attached to the worker is detected, and the detection result is compared with the target work state to confirm the success or failure of the work.

JP 2007-156838 A JP 2003-167613 A

  However, attaching a marker to the worker as in the techniques according to Patent Documents 1 and 2 causes a reduction in the work efficiency of the worker and increases the cost. In addition, the work of attaching a marker to an operator is complicated and takes time.

  In addition, as described above, in the method of determining the presence / absence of parts acquisition by installing an operation sensor at the outlet of the used parts shelf, it is possible to acquire only two parts in one operation or not to acquire parts only in operation. Misoperation could not be detected.

  Therefore, an object of the present invention is to provide a work confirmation system that can confirm a worker's work operation by a simple method without causing a reduction in work efficiency and cost of the worker. It is another object of the present invention to provide an operation confirmation system that can confirm a more accurate parts acquisition operation.

  In order to achieve the above object, a work confirmation system according to the present invention includes an imaging device that captures work content, an image processing / work determination device that performs image processing and work success / failure determination, and a result of the work success / failure determination. A notification device for notifying, wherein the image processing / work determination device stores a reference image data group composed of a plurality of reference image data in which a reference shape is defined, and the imaging device is ( A) A series of work contents are imaged, (B) a work image data group composed of a plurality of image data obtained by (A) is transmitted to the image processing / work determination device, and the image processing / work determination is performed. The apparatus performs edge extraction processing on each of the image data constituting the work image data group received in (C) and (B), and (D) based on a result of the edge extraction processing, each A shape pattern indicating a work operation is specified in the image data, (E) the work image data group after (D) is compared with the reference image data group, and (D) the work image data group after In addition, it is determined whether or not there is the same shape pattern as the reference shape, and the notification device notifies the determination result of (E) to the outside.

  Alternatively, the work confirmation system according to the present invention includes an imaging device that captures an inventory state of a component, an image processing / work determination device that performs image processing and component success / failure determination, and a notification device that notifies the result of the component success / failure determination. The image processing / work determination apparatus stores reference image data in which a first inventory state of the part is defined, and the imaging apparatus includes (a) the first inventory. The second stock state of the part after taking out the part from the state is imaged, and (b) image data representing the second stock state obtained by (a) is used as the image processing / work. The image processing / work determination device transmits to the determination device, (c) performs edge extraction processing on the image data received in (b), and (d) based on the result of the edge extraction processing. The image data And (e) comparing the image data after (d) with the reference image data to determine whether or not the component has been used, and the notifying device in (e) The determination result is notified to the outside.

  The work confirmation system according to the present invention includes an imaging device that captures work content, an image processing / work determination device that performs image processing and work success / failure determination, and a notification device that notifies a result of the work success / failure determination. The image processing / work determination device stores a reference image data group including a plurality of reference image data in which a reference shape is defined, and the imaging device images (A) a series of work contents. (B) A work image data group composed of a plurality of image data obtained by (A) is transmitted to the image processing / work determination device, and the image processing / work determination device (C) (B The edge extraction process is performed on each of the image data constituting the work image data group received by (1), and (D) a work operation is performed on each image data based on the result of the edge extraction process. A shape pattern to be shown is specified, (E) the work image data group after (D) is compared with the reference image data group, and the work image data group after (D) is the same as the reference shape. It is determined whether or not the shape pattern is present, and the notification device notifies the determination result of (E) to the outside.

  Therefore, in the work confirmation system, it is not necessary to add a marker to the worker when confirming work suitability, and the configuration is simplified. Therefore, the work confirmation system according to the present embodiment can confirm the work operation of the worker by a simple method without causing a reduction in work efficiency and cost of the worker.

  In addition, the work confirmation system according to the present invention includes an imaging device that captures an inventory state of a component, an image processing / work determination device that performs image processing and component success / failure determination, and a notification device that notifies a result of the component success / failure determination. The image processing / work determination apparatus stores reference image data in which a first inventory state of the part is defined, and the imaging apparatus includes (a) the first inventory. The second stock state of the part after taking out the part from the state is imaged, and (b) image data representing the second stock state obtained by (a) is used as the image processing / work. The image processing / work determination device transmits to the determination device, (c) performs edge extraction processing on the image data received in (b), and (d) based on the result of the edge extraction processing. To the image data And (e) comparing the image data after (d) with the reference image data to determine whether or not the component has been used, and the notification device determines whether or not (e) The result is notified to the outside.

  Therefore, the said work confirmation system can confirm more accurate parts acquisition work rather than parts acquisition success / failure determination by a sensor etc. with a simple structure and a simple method.

It is a block diagram which shows the structure of the work confirmation system which concerns on this invention. 4 is a flowchart for explaining the operation of the work confirmation system according to the first embodiment. It is a figure for demonstrating operation | movement of the work confirmation system which concerns on Embodiment 1. FIG. It is a figure for demonstrating operation | movement of the work confirmation system which concerns on Embodiment 1. FIG. 10 is a flowchart for explaining the operation of the work confirmation system according to the second embodiment.

  The present invention relates to a work confirmation system that can quickly and reliably find work mistakes and confirmation of misuse of assembled parts in assembly work by an operator in various production lines. Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of a work confirmation system according to the present embodiment. As shown in FIG. 1, the work confirmation system according to the present embodiment includes an imaging device 10, an image processing / work determination device 20, and a notification device 30.

  In the present embodiment, the imaging device 10 dynamically images the work content (a series of work operations) of the worker.

  The image processing / work determination device 20 includes an image processing unit 1, a work determination unit 2, and a reference data storage unit 3.

  The image processing unit 1 performs edge extraction processing on image data. Then, since the shape (contour) of the object can be recognized by the edge extraction process, the image processing unit 1 specifies the shape (contour) of the object included in the image data based on the edge extraction process ( Symbolization processing). The reference data storage unit 3 stores reference image data serving as a reference for success or failure of work. In addition, the work determination unit 2 uses the reference image data stored in the reference data storage unit 3 to determine whether or not the work content of the worker imaged by the imaging device 10 is correct.

  The notification device 30 determines the success or failure of the work content determined by the image processing / work determination device 20 (more specifically, the work determination unit 2) (in this embodiment, the success or failure of a series of work operations of the worker). Notice. Here, the notification may be notified by voice, the determination result may be displayed so as to be visible, and any notification means may be adopted as long as the determination result can be recognized from the outside. .

  Here, the image processing unit 1 and the image processing / work determination device 20 are connected to be communicable. Further, the image processing / work determination device 20 and the notification device 30 are connected to be communicable. In the image processing / work determination device 20, the image processing unit 1, the work determination unit 2, and the reference data storage unit 3 are also connected to be communicable with each other.

  First, preparation for the operation of the work confirmation system will be described.

  First, the imaging apparatus 10 is used to image a correct series of work operations performed by an expert or the like. The captured result is transmitted to the image processing / work determination apparatus 20 as a plurality of series of image data.

  In the image processing / work determination device 20, the image processing unit 1 performs edge extraction processing on each transmitted image data. That is, each image data includes an image indicating the motion of the worker, and the edge of the motion shape (contour) of the worker is detected. And by the said edge extraction process, the image process part 1 specifies an operator's operation | movement shape (contour) in each image data (symbolization process).

  Next, the user selects at least two pieces of image data including a characteristic operation (for example, an operation that is definitely performed and an operation that is clearly discriminated) from among the plurality of image data subjected to the symbolization processing. Select above. Then, the selected image data is stored in the reference data storage unit 3 as reference image data.

  Here, the above-mentioned characteristic operation (that is, the reference shape) is defined for each reference image data. As can be seen from the above, the reference data storage unit 3 stores a plurality of reference image data (that is, a reference image data group). Further, the reference image data group is configured by defining a plurality of reference image data in a predetermined order (order according to the order of work of skilled workers).

  When the reference image data group is stored in the reference data storage unit 3, the preparation is completed. Next, the operation of the work confirmation system according to the present embodiment will be described using the flowchart shown in FIG.

  The imaging device 10 starts imaging the worker (step S1). Thereafter, the worker starts a predetermined series of work in units of work (step S2). When the worker finishes a series of work operations, the imaging device 10 finishes imaging the work content of the worker (step S3). Thereafter, the imaging device 10 transmits a plurality of pieces of image data obtained by imaging a series of work contents (work operations) of the worker during steps S1 to S3 to the image processing / work determination device 20 ( Step S4).

  Here, the data group composed of a plurality of image data transmitted in step S4 is referred to as a work image data group. Therefore, the work image data group is configured by arranging (defining) a plurality of image data in time series order of imaging of the imaging apparatus 10 between steps S1 to S3.

  The image processing / work determination device 20 receives the work image data group transmitted in step S4 and stores it in a predetermined storage area of a predetermined storage device (not shown). Then, the image processing unit 1 performs edge extraction processing on each image data constituting the work image data group (step S5). That is, each image data includes an image indicating the worker's motion, but the image processing unit 1 detects the edge of the motion shape (contour) of the worker. Then, based on the data after the edge extraction process, the image processing unit 1 specifies the action shape (contour) of the worker, that is, the shape pattern indicating the action of the work in each image data (symbolization process). Step S5).

  The image processing unit 1 transmits the work image data group after step S5 to the work determination unit 2. The work determination unit 2 reads a reference image data group stored in the reference data storage unit 3. Then, the work determination unit 2 compares the received work image data group with the read reference image data group (step S6).

  The work image data group received by the work determination unit 2 includes a plurality of image data defined in the time series of imaging. In each image data after step S5, various shape patterns indicating the operation of the worker are specified. In addition, the reference image data group read by the work determination unit 2 includes at least two or more reference image data defined in a predetermined order (time-sequential order of work operations of skilled workers). In each reference image data, a reference shape (a characteristic action shape that is surely performed among the work operations of skilled workers) is specified.

  Therefore, in step S6, the work determination unit 2 refers to each image data after step S5 in the time series of imaging, and in a plurality of image data constituting the work image data group, according to a predetermined order of the reference image data. It is determined whether or not a shape pattern that can be identified with the reference shape is included.

  For example, as shown in FIG. 3, it is assumed that the reference image data group includes three reference image data BD1, BD2, and BD3. The predetermined order of the reference image data BD1, BD2, and BD3 defined in the reference image data group is assumed to be the order of BD1 → BD2 → BD3. Further, it is assumed that the reference shape A is specified in the reference image data BD1, the reference shape B is specified in the reference image data BD2, and the reference shape C is specified in the reference image data BD3 by the symbolization processing. .

  On the other hand, as shown in FIG. 4, it is assumed that the work image data group includes N pieces of image data d1, d2,. Then, it is assumed that the time-series order taken by the imaging device 10 is in the order of d1 → d2 →. Further, in each of the image data d1, d2,..., DN, a shape pattern indicating an operator's motion shape is specified by symbolization processing. That is, by moving the image data d1, d2,... DN frame by frame in the order of d1-> d2->. Fwdarw.dN, a series of operations of the operator imaged by the imaging device 10 is reproduced.

  In the above case, in step S6, the work determination unit 2 determines the patterns of the shapes of the image data d1, d2,... DN arranged in the time series of imaging in FIG. 4 and the reference image data BD1, DB2, DB3. The reference shapes A, B, and C specified in the above are compared. In the work image data group shown in FIG. 4 (in the plurality of image data d1, d2,... DN arranged in the time-series order of imaging), the first pattern having a shape pattern that can be identified with the reference shape A. Image data, second image data having a shape pattern that can be identified with the reference shape B, and third image data having a shape pattern that can be identified with the reference shape C are included in this order (the predetermined order). It is determined whether or not. Here, the first image data, the second image data, and the third image data do not need to be continuous and may be intermittent.

  As a result of the image comparison processing in step S6, the work determination unit 2 determines that the reference shapes are included in a predetermined order in the work image data group (in the above example, the work shown in FIG. In the image data group, the first image data, the second image data, and the third image data are included in a predetermined order). In this case, the work determination unit 2 determines “work contents: appropriate” indicating that the series of work contents of the worker performed in step S2 is correct (step S7). Then, the image processing / work determination device 20 transmits the determination result to the notification device 30. Then, the notification device 30 outputs a notification indicating “work content: appropriate” (step S8).

  On the other hand, it is assumed that, as a result of the image comparison process in step S6, the work determination unit 2 determines that the reference shapes are not included in a predetermined order in the work image data group. For example, in the above example, when at least one of the first image data, the second image data, and the third image data is missing in the work image data group shown in FIG. The image data, the second image data, and the third image data are all included, but the work image data group shown in FIG. 4 does not follow the predetermined order. In this case, the work determination unit 2 determines “work contents: unsuitable” indicating that the series of work contents of the worker performed in step S2 is incorrect (step S7). Then, the image processing / work determination device 20 transmits the determination result to the notification device 30. Then, the notification device 30 outputs a notification indicating “work content: unsuitable” (step S8).

  As described above, in the work confirmation system according to the present embodiment, the reference image data group and work image captured by the imaging device 10 and subjected to the edge extraction processing and symbolization processing in the image processing / work determination device 20. By comparing the data groups, the image processing / work determination device 20 determines the suitability of the worker's work.

  As described above, in the work confirmation system according to the present embodiment, it is not necessary to attach a marker to the worker when confirming work suitability, and the configuration is simplified. Therefore, the work confirmation system according to the present embodiment can confirm the work operation of the worker by a simple method without causing a reduction in work efficiency and cost of the worker.

  In addition, the work confirmation system according to the present embodiment makes it possible to determine whether or not there is a problem with the work content immediately after the work is finished in the work performed by the worker. Therefore, inspection or inspection of the workpiece can be reduced, and work efficiency and work quality can be improved.

<Embodiment 2>
According to the first embodiment, the success or failure of the worker's work operation can be determined and confirmed. On the other hand, when the work is an assembly work or the like, for example, the following method can be considered as a method for determining and confirming the suitability of the parts to be assembled. In other words, before the work, the operator's data and the data of the parts to be assembled are read by a barcode reader, and the read data is compared with the pre-managed data, so that the worker uses the appropriate parts in the work. You can check if you did.

  On the other hand, it is also possible to confirm / determine the success / failure of parts to be handled by the worker using the work confirmation system having the configuration shown in FIG. Hereinafter, the work confirmation system (work confirmation system according to the present embodiment) will be described. Here, the work confirmation system according to the present embodiment and the work confirmation system according to the embodiment have the same block configuration (see FIG. 1).

  In the present embodiment, the imaging device 10 images the inventory status of parts.

  Also in the present embodiment, the image processing / work determination device 20 includes the image processing unit 1, the work determination unit 2, and the reference data storage unit 3.

  The image processing unit 1 performs edge extraction processing on image data. Then, since the shape (contour) of the object can be recognized by the edge extraction process, the image processing unit 1 specifies the shape (contour) of the object included in the image data based on the edge extraction process ( Symbolization processing). The reference data storage unit 3 stores reference image data serving as a reference for success / failure of work. Further, the work determination unit 2 determines whether the work content captured by the imaging device 10 is correct using the reference image data stored in the reference data storage unit 3.

  The notification device 30 notifies the success / failure of the work content determined by the image processing / work determination device 20 (more specifically, the work determination unit 2) (success / failure of the used component in the present embodiment). Here, the notification may be notified by voice, the determination result may be displayed so as to be visible, and any notification means may be adopted as long as the determination result can be recognized from the outside. .

  Here, as described in the first embodiment, the image processing unit 1 and the image processing / work determination device 20 are connected to be communicable. Further, the image processing / work determination device 20 and the notification device 30 are connected to be communicable. In the image processing / work determination device 20, the image processing unit 1, the work determination unit 2, and the reference data storage unit 3 are also connected to be communicable with each other.

  Next, the operation of the work confirmation system according to the present embodiment will be described using the flowchart shown in FIG.

  First, preparation for the operation of the work confirmation system will be described.

  First, the image pickup apparatus 10 is used to pick up an image of a stock state of parts before work (hereinafter referred to as a first stock state). The imaged result is transmitted to the image processing / work determination device 20.

  In the image processing / work determination device 20, the image processing unit 1 performs edge extraction processing on the transmitted image data. That is, the image data includes an image indicating the state of the component, and the edge of the shape (contour) of each component is detected. Then, through the edge extraction process, the image processing unit 1 specifies the shape (contour) of the component placement state in the image data (symbolization process). Then, the symbolized image data is stored in the reference data storage unit 3 as reference image data (that is, as image data in which the inventory status of parts is defined).

  When the reference image data is stored in the reference data storage unit 3, the preparation is completed. Next, operation | movement of a work confirmation system is demonstrated using the flowchart shown in FIG.

  The worker performs a work operation for attaching a predetermined part to the product according to the work instruction (step S21). By the work operation, the worker takes out the part from the place where the part is stored. When the worker finishes the work operation, the imaging device 10 captures an image of the inventory status (second inventory status) of the component after taking out the component (step S22). Thereafter, the imaging device 10 transmits the image data obtained in step S22 to the image processing / work determination device 20 (step S23). Here, the image data transmitted in step S23 includes the second stock status.

  The image processing / work determination device 20 receives the image data transmitted in step S23 and stores it in a predetermined storage area of a predetermined storage device (not shown). Then, the image processing unit 1 performs edge extraction processing on the image data (step S24). That is, the image data includes an image indicating the inventory status (second inventory status) of the part after step S21. In the image processing unit 1, the shape (contour) of the component in the second inventory status is included. ) Edge is detected. Then, based on the data after the edge extraction processing, the image processing unit 1 specifies a shape pattern indicating the shape (contour) of the component in the second inventory state in the image data (symbolization processing, Step S24).

  The image processing unit 1 transmits the image data after step S24 to the work determination unit 2. In addition, the work determination unit 2 reads reference image data (image data in which the first inventory state is captured) stored in the reference data storage unit 3. Then, the work determination unit 2 compares the received image data with the read reference image data (step S25).

  The image data received by the work determination unit 2 is the image data captured in step S22. As described above, the shape pattern of the part in the second inventory state is specified in the image data after step S24. . Further, the reference shape (the shape of the part in the first inventory state before step S21) is specified in the reference image data read by the work determination unit 2.

  Accordingly, in step S25, the work determination unit 2 compares the shape pattern in the image data after step S24 with the reference shape of the reference image data, and detects a difference between both shapes.

  For example, if the shape pattern and the reference shape are the same, it means that the operator has not taken out the predetermined part described in the work instruction in step S21 (for example, (the predetermined part and Therefore, in this case, the work determination unit 2 performs the work contents of the worker (parts taken out and assembled) in step S21. ) Is determined to be inappropriate and “part use: inappropriate” is determined (step S26), and the image processing / work determination device 20 transmits the determination result to the notification device 30. 30 outputs a notification indicating “part use: inappropriate” (step S27).

  Further, for example, it is assumed that the work instruction sheet includes an assembly work instruction using one predetermined part. As a result of the comparison in step S25, the shape pattern is different from the reference shape, and the difference is caused by the fact that one predetermined part is lost (for example, the reference pattern). In the data storage unit 3, a symbolized component unit image of one predetermined component is registered in advance, and a difference shape between the shape pattern and the reference shape is determined by using the component unit image. The work determination unit 2 determines that there is no more one). In this case, the work determination unit 2 determines that the work contents of the worker performed in step S21 (the work in which the predetermined parts are taken out and assembled) are appropriate, and the determination of “part use: appropriate” is made. Is performed (step S26). Then, the image processing / work determination device 20 transmits the determination result to the notification device 30. Then, the notification device 30 outputs a notification indicating “component use: appropriate” (step S27).

  Similarly to the above, it is assumed that the work instruction sheet includes an assembly work instruction using one predetermined part. As a result of the comparison in step S25, the shape pattern is different from the reference shape, and the difference is caused by the fact that two or more predetermined parts are lost (for example, In the reference data storage unit 3, a symbolized component unit image of one predetermined component is registered in advance, and using the component unit image, a difference shape between the shape pattern and the reference shape is a predetermined shape. The work determination unit 2 determines that there are no more than two parts). In this case, the work determination unit 2 determines that the work contents of the worker performed in step S21 (the work in which the predetermined parts are extracted and assembled) are inappropriate, and the determination of “part use: inappropriate” is made. Is performed (step S26). Then, the image processing / work determination device 20 transmits the determination result to the notification device 30. And the notification apparatus 30 outputs the notification which shows "component use: improper" (step S27).

  As described above, in the work confirmation system according to the present embodiment, the reference image data and image data captured by the imaging device 10 and subjected to the edge extraction processing and the symbolization processing in the image processing / work determination device 20 are stored. In comparison (that is, by comparing the first stock status and the second stock status), the image processing / work determination device 20 determines the suitability of the parts to be handled in the work.

  Therefore, the work confirmation system according to the present embodiment can confirm a more accurate part acquisition work than a part acquisition success / failure determination by a sensor or the like with a simple configuration and a simple method.

  In addition, the work confirmation system according to the present embodiment makes it possible to immediately determine whether or not there is a problem in the work content in the parts acquisition work by the worker after the work is completed. Therefore, inspection or inspection of the workpiece can be reduced, and work efficiency and work quality can be improved.

DESCRIPTION OF SYMBOLS 1 Image processing part 2 Work determination part 3 Reference | standard data storage part 10 Imaging device 20 Image processing / work determination apparatus 30 Notification apparatus

Claims (3)

An imaging device for imaging work content;
An image processing / work determination device for performing image processing and work success / failure determination;
A notification device for notifying the result of the work success / failure determination,
The image processing / work determination device includes:
A reference image data group consisting of a plurality of reference image data in which a reference shape is defined is stored,
The imaging device
(A) Capture a series of work contents,
(B) A work image data group made up of a plurality of image data obtained by (A) is transmitted to the image processing / work determination device,
The image processing / work determination device includes:
(C) An edge extraction process is performed on each of the image data constituting the work image data group received in (B),
(D) Based on the result of the edge extraction process, identify a shape pattern indicating a work operation in each image data;
(E) The work image data group after (D) is compared with the reference image data group, and whether or not the same shape pattern as the reference shape exists in the work image data group after (D). Determine whether
The notification device includes:
Notifying the determination result of (E) to the outside,
A work confirmation system characterized by that. The reference image data group is:
The plurality of reference image data is configured in a predetermined order,
The working image data group is:
The plurality of image data is configured by being defined in order of time series of imaging,
In the image processing / work determination device, in (E),
(D) In the subsequent work image data group, it is determined whether or not the same shape pattern as the reference shape is present according to the predetermined order.
The work confirmation system according to claim 1, wherein: An imaging device for imaging the inventory status of the parts;
An image processing / work determination device that performs image processing and component success / failure determination;
A notification device for notifying the result of the component success / failure determination,
The image processing / work determination device includes:
Reference image data defining the first inventory status of the part is stored,
The imaging device
(A) Imaging the second inventory status of the part after taking out the component from the first inventory status;
(B) transmitting the image data representing the second inventory state obtained by (a) to the image processing / work determination device;
The image processing / work determination device includes:
(C) performing an edge extraction process on the image data received in (b),
(D) identifying a shape pattern in the image data based on the result of the edge extraction process;
(E) The image data after (d) is compared with the reference image data to determine whether or not the component has been used,
The notification device includes:
Notifying the determination result of (e) to the outside,
A work confirmation system characterized by that.

Images