JP7207029B2 - Work support device and work support program - Google Patents

Description

The present invention relates to a work support device and a work support program that support work performed according to a predetermined work procedure.

2. Description of the Related Art Conventionally, at a manufacturing site, for example, an assembling work of manually assembling parts to a printed circuit board, a device in the middle of manufacturing, or the like may be performed. Although such work is performed according to a predetermined work procedure, it generally includes multiple processes, and assembly work generally involves assembling multiple types of parts. For some reason, parts may be left out or mixed up. Therefore, for example, Patent Literature 1 proposes to provide detection means such as a weight sensor on the parts box side to determine whether or not the parts taken out from the parts box conform to the work procedure. .

In addition, when working on different products, the types and numbers of parts to be assembled are also different. Therefore, if the detection means as described above is provided on the parts box side, the equipment cost increases in proportion to the number of parts. In addition to this, there is a problem that a great deal of labor is required to install the detecting means and associate it with the work procedure. In order to solve this problem, for example, a work support device disclosed in Patent Document 2 below is known. In this work support device, a monitoring region, which is a range for monitoring work within the imaging range of the imaging unit, is set by input operation according to the work procedure, and a portion corresponding to the monitoring region of the captured image and a portion of the image corresponding to the monitoring region are set. By comparing the part corresponding to the monitoring area in the other image taken at the previous time, dynamic detection is performed for the management area, and based on the detection result of this dynamic detection, whether or not the work procedure was followed is determined, and the determination result is notified to the operator.

JP 2010-211623 A JP 2018-156279 A

By the way, in the work support device disclosed in the above-mentioned Patent Document 2, when setting the monitoring area corresponding to each parts box, the operator confirms the position of the parts box in the image captured by the imaging unit. It is necessary to operate the wireless mouse to draw a border around the parts box in the image. This is because the monitoring area cannot be set in advance because not only the type and number of parts boxes but also the positions of the parts boxes may change for each manufacturing lot. Then, for each production lot, the worker must operate the wireless mouse while looking at the screen. This not only takes time for the initial setting work to set the monitoring area, but also causes the operator to make an operation error. There is a problem that an erroneous monitoring area is set.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems, and its object is to provide a configuration in which a monitoring area can be easily and accurately set without imposing a work burden on an operator. be.

In order to achieve the above object, the invention described in claim 1 of the scope of claims includes:
A work support device (10) for supporting work performed according to a predetermined work procedure using parts (20, 20a-20d) housed in a plurality of parts boxes (30, 30a-30d),
an imaging unit (13);
a monitoring area setting unit (11) for setting a monitoring area (Pa to Pd) for each of the parts boxes within the imaging range of the imaging unit;
A portion corresponding to the monitoring area set by the monitoring area setting unit in the captured image captured by the imaging unit and the monitoring area corresponding to another captured image captured at a time before the captured image. a detection unit (11) for detecting removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
a determination unit (11) that determines whether or not the work according to the predetermined work procedure is being performed based on the detection result of the detection unit;
a notification unit (15, 16) for notifying the operator of the determination result by the determination unit;
with
The upper ends (33a to 33d) of the peripheral wall (32) in the parts box are formed in a polygonal annular shape with a plurality of corners (34a to 34d),
The monitoring area setting unit extends from a corner that is touched by a worker's finger among the plurality of corners in the captured image captured by the imaging unit and bends at a plurality of intersections. A boundary line (Pa to Pd) returning to the starting point is detected, and a polygonal area surrounded by the boundary line is set as the monitoring area for each parts box.

The invention according to claim 5,
A work support device (10) for supporting work performed according to a predetermined work procedure using parts (20, 20a-20d) housed in a plurality of parts boxes (30, 30a-30d),
an imaging unit (13);
a monitoring area setting unit (11) for setting a monitoring area (Pa to Pd) for each of the parts boxes within the imaging range of the imaging unit;
A portion corresponding to the monitoring area set by the monitoring area setting unit in the captured image captured by the imaging unit and the monitoring area corresponding to another captured image captured at a time before the captured image. a detection unit (11) for detecting removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
a determination unit (11) that determines whether or not the work according to the predetermined work procedure is being performed based on the detection result of the detection unit;
a notification unit (15, 16) for notifying the operator of the determination result by the determination unit;
with
The monitoring area setting unit is configured so that a circular trajectory (Pa to Pd) drawn by the operator (M)'s finger (F) tracing the upper end (33a to 33d) of the peripheral wall (32) of the parts box by the imaging unit is When the image is captured, an area surrounded by the circular trajectory is set as the monitoring area for each parts box.
It should be noted that the symbols in parentheses above indicate the corresponding relationship with specific means described in the embodiments to be described later.

In the first aspect of the present invention, monitoring of a portion corresponding to a monitoring area set by a monitoring area setting unit in a captured image captured by an imaging unit and another captured image captured at a time before the captured image is monitored. Based on the result of comparison with the portion corresponding to the area, the detection unit detects removal of the part from the parts box corresponding to the monitoring area. Based on the result of detection by the detection unit, the determination unit determines whether or not the work according to the predetermined work procedure is being performed, and the notification unit notifies the operator of the determination result. At that time, in the imaged image captured by the imaging unit, the monitoring area setting unit extends a plurality of corners of the upper end of the peripheral wall of the parts box from the corner where the operator's finger is touching as a starting point. are detected, and a polygonal area surrounded by the boundary lines is set as a monitoring area for each parts box.

As a result, the operator can easily set the monitoring area for each parts box by simply touching one corner of the upper end of the peripheral wall of the parts box with his/her finger before starting work or the like. In particular, since the operator does not need to operate the mouse while looking at the screen, an incorrect monitoring area is not set due to the operator's input operation error. Therefore, the monitoring area can be easily and accurately set without imposing a work burden on the operator.

In the invention according to claim 2, the monitoring area setting unit detects the boundary line so that one of the plurality of corners touched by the finger is the starting point and the remaining corners are at least part of the intersections. As a result, the boundary line returned to the original starting point so as not to pass through the corner touched by the remaining fingers as the intersection can be determined as the erroneously detected boundary line, so that the detection accuracy of the boundary line can be improved.

In the third aspect of the present invention, the boundary line is detected by the monitoring area setting section so that the direction of bending at the intersection is determined in one direction. For example, there is a possibility that an erroneous boundary line is detected because an unintended intersection is detected because the neighboring parts box and the peripheral wall are in contact with each other. On the other hand, for example, when extending the boundary line so that the direction of bending at the intersection is the clockwise direction in plan view from above the parts box, at the unintended intersection, the upper end of the peripheral wall of the adjacent parts box The direction of turning is the opposite counterclockwise direction. For this reason, by setting the bending direction at the intersection to one direction, the boundary line extending from the unintended intersection as described above so as to bend in a direction different from the one direction is directed toward the upper end of the peripheral wall of the adjacent parts box. Can be excluded as curved borders.

In the fourth aspect of the present invention, the monitoring area setting unit detects the boundary line as an annular portion having a predetermined width, and sets the polygonal area surrounded by the inner edge of the annular portion as the monitoring area. Accordingly, the monitoring area can be set more accurately because the adjacent parts box, which may be connected to the outer edge of the annular portion, does not affect the setting of the monitoring area.

In the fifth aspect of the present invention, monitoring of a portion corresponding to the monitoring area set by the monitoring area setting unit in the captured image captured by the imaging unit and other captured images captured at a time prior to the captured image is performed. Based on the result of comparison with the portion corresponding to the area, the detection unit detects removal of the part from the parts box corresponding to the monitoring area. Based on the result of detection by the detection unit, the determination unit determines whether or not the work according to the predetermined work procedure is being performed, and the notification unit notifies the operator of the determination result. At that time, when the monitoring area setting unit captures an image of a circular locus drawn by the operator's finger tracing the upper end of the peripheral wall of the parts box by the imaging unit, the area surrounded by the circular locus becomes the monitoring area. It is set for each parts box.

As a result, the operator can easily set the monitoring area for each parts box by simply tracing the upper edge of the peripheral wall of the parts box before starting work. In particular, since the operator does not need to operate the mouse while looking at the screen, an incorrect monitoring area is not set due to the operator's input operation error. Therefore, the monitoring area can be easily and accurately set without imposing a work burden on the operator.

According to the invention of claim 6, it is possible to realize a work support program having the same effect as that of claim 1.
According to the seventh aspect of the invention, it is possible to realize a work support program that has the same effect as the fifth aspect.

It is an explanatory view showing a schematic structure of a work support device concerning a 1st embodiment. It is explanatory drawing explaining the imaging state where each component box is imaged. 1 is a block diagram illustrating an electrical configuration of a work support device; FIG. 6 is a flowchart illustrating the flow of work support processing performed by a control unit in the first embodiment; FIG. 5 is a flowchart illustrating the flow of monitoring area setting processing in FIG. 4 ; FIG. FIG. 11 is an explanatory diagram illustrating a state in which search targets are switched starting from a corner touched by a stagnant finger; FIG. 7A is an explanatory diagram illustrating a state in which the first monitoring area is set in the captured image, and FIG. 7B is an explanatory diagram illustrating a state in which all monitoring areas are set in the captured image. It is a diagram. FIG. 11 is an explanatory diagram for explaining a case where the influence of detection of an unintended intersection is suppressed by setting the turning direction at the intersection in one direction during searching; FIG. 11 is an explanatory diagram for explaining a monitoring region set by touching two corners with stagnant fingers in the second embodiment; FIG. 12 is an explanatory diagram for explaining a monitoring area set by searching the inner edge of the upper end of the parts box in the third embodiment; FIG. 16 is a flowchart illustrating the flow of monitoring area setting processing in the fourth embodiment; FIG. FIG. 14 is an explanatory diagram illustrating a circular trajectory traced by an operator's finger along the upper end of the parts box in the fourth embodiment;

[First embodiment]
DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment embodying a work support device and a work support program according to the present invention will be described below with reference to the drawings.
As illustrated in FIG. 1, the work support device 10 according to the present embodiment is provided on a workbench 1 or the like, and is operated by a worker M according to a predetermined work procedure using parts stored in a plurality of parts boxes. It is configured as a device that assists the assembly work to be performed.

A work W such as a printed circuit board or a device to be assembled is transferred and placed on the workbench 1 where the assembly work is performed. A plurality of component boxes 30 are arranged on the shelf 2 of the workbench 1 so as to be aligned left and right as seen from the worker M, and each component box 30 contains a plurality of types of components 20 to be assembled to the work W. are stored separately. In this embodiment, as can be seen from FIG. 2, four component boxes 30a to 30d are arranged side by side on the shelf 2, the component box 30a contains the component 20a, the component box 30b contains the component 20b, and the component box 30a contains the component 20b. The component 20c is stored in the box 30c, and the component 20d is stored in the component box 30d.

As shown in FIG. 2, the parts box 30a has a rectangular bottom wall 31 and a peripheral wall 32 which is annularly connected at the lower end to the outer edge of the rectangular bottom wall 31. It is formed to be a quadrangular ring having In the component boxes 30b to 30d, similarly to the component box 30a, the peripheral wall is annularly connected at the lower end to the outer edge of the square bottom wall, and the upper end of the peripheral wall is formed into a square ring having four corners. is formed in

As shown in FIGS. 1 and 3, the work support device 10 includes a control unit 11, a storage unit 12, an imaging unit 13, a display unit 14, a light emitting unit 15, a speaker 16, an operation unit 17, a communication unit 18, and the like. there is The control unit 11 is composed mainly of a microcomputer, performs overall control of the work support device 10 and performs various calculations, and functions, for example, to execute work support processing to be described later. The storage unit 12 is configured by known storage media such as ROM, RAM, HDD, and nonvolatile memory, and stores an application program (hereinafter also referred to as a work assistance program) for executing work assistance processing, a predetermined database, and the like. are stored in advance so that they can be used by the control unit 11 .

The imaging unit 13 is configured as a camera having a light receiving sensor (eg, C-MOS area sensor, CCD area sensor, etc.). In this embodiment, the imaging unit 13 is configured separately from the apparatus main body 10a including the control unit 11, the display unit 14, etc., and the range including at least the component boxes 30a to 30d can be captured by moving or still images. It is arranged on the upper part of the workbench 1 so as to take an image. In this embodiment, the imaging unit 13 is configured to capture continuous still images at, for example, 30 frames/second, and each captured image is stored in the storage unit 12 so as to be analyzed by the control unit 11 .

The display unit 14 is, for example, a liquid crystal display, and is configured to be controlled by the control unit 11 to display an image captured by the imaging unit 13, predetermined information, and the like. The device main body 10a is attached to the back plate of the workbench 1 or the like so that the display screen of the display unit 14 can be visually recognized by the operator M. As shown in FIG.

The light-emitting unit 15 is, for example, an LED, and is configured to be controlled by the control unit 11 to control the color of emitted light and the lighting/blinking state, and is arranged at a position where the operator M can easily see it. The speaker 16 is a known speaker or the like, and is controlled by the control unit 11 to emit various notification sounds such as preset sounds and alarm sounds. Note that the light emitting unit 15 and the speaker 16 may correspond to an example of the "notifying unit".

The operation unit 17 is configured to output an operation signal corresponding to the input operation to the control unit 11, and the control unit 11 receives the operation signal and performs processing according to the input operation. there is The communication unit 18 is configured as a communication interface for performing data communication with an external device such as a host device, and is configured to perform communication processing in cooperation with the control unit 11 .

Next, when the worker M performs the assembling work of individually assembling the parts contained in the parts boxes into the workpiece W according to a predetermined work procedure, the control unit 11 executes for each production lot. The work support processing performed based on the work support program will be described in detail with reference to the flow charts shown in FIGS. 4 and 5. FIG. As a predetermined work procedure, the part 20a in the part box 30a is assembled first, the part 20b in the part box 30b is second, and the part 20c in the part box 30c is third. An example will be described in which the work procedure is determined in advance such that the component 20d in the component box 30d is assembled fourth.

When the operation support processing is started by the control unit 11 by performing a predetermined operation on the operation unit 17, the work procedure information reading processing shown in step S101 of FIG. (hereinafter also referred to as work procedure information) on the above-mentioned predetermined work procedure is read from the storage unit 12 . In this embodiment, the work procedure information is stored in the storage unit 12 each time by reading an information code or a wireless tag attached to a work instruction sheet or the like describing the work procedure for the production lot. For example, the work procedure information may be stored in the storage unit 12 in response to an input operation or the like by the operation unit 17, or may be received in advance from a host device or the like via the communication unit 18 and stored in the storage unit 12. may be stored in

When the work procedure information is read as described above, k indicating the assembly order (pickup order) of the parts is set to 1 (S103), and then the monitoring area setting process shown in step S105 is performed. In this process, an image area corresponding to the parts box of the k-th part in the assembly order specified from the work procedure information is set as the monitoring area in the captured image captured by the imaging unit 13 . done. Immediately after the assembly order k is set to 1 as described above, processing is performed to set the image area corresponding to the parts box 30a of the first part 20a as the monitoring area Pa. Note that the control unit 11 that executes the monitoring area setting process can correspond to an example of a "monitoring area setting unit".

In the monitoring area setting process according to the present embodiment, when the operator M touches one corner on the upper end of the peripheral wall of the parts box with a finger F while the imaging unit 13 is imaging, A boundary line along the upper edge is detected, and a rectangular area surrounded by the boundary line is set and registered as a monitoring area.

The monitor area setting process for setting the monitor area in accordance with the boundary line detected starting from the corner of the parts box touched by the finger will be specifically described below with reference to the flowchart shown in FIG.
First, when the imaging unit 13 becomes capable of imaging (S201 in FIG. 5), it is determined in the determination processing in step S203 whether or not a human finger is detected in the captured image, and the human finger is detected. The determination of No is repeated until .

If a human finger is detected in the captured image during this repeated processing (Yes in S203), it is determined in the determination processing in step S205 whether or not the detected finger is stagnant. Here, if the finger is moving (No in S205), the processing from step S203 is performed, and while the finger movement is detected, the determination is Yes in step S203 and No in step S205. Repeated.

Then, when the detected finger stagnates, a determination of Yes is made in the determination processing of step S205, and timing of the stagnation time during which the detected finger stagnate is started (S207). Subsequently, in the determination process of step S209, it is determined whether or not the stagnation time has passed a preset predetermined time (for example, 5 seconds), and if the stagnation time has not passed the predetermined time (No in S209), in the determination processing in step S211, it is determined whether or not the detected stagnation of the finger is maintained. If the detected finger remains stationary (Yes in S211), the processing from step S209 is performed. If the movement of the finger is detected before the stagnation time elapses (No in S211), the process from step S203 is performed.

On the other hand, as shown in FIG. 6, the finger F of the worker M stays in contact with the corner 34a between the upper end 33a and the upper end 33d of the parts box 30a, and the stay time of the finger F is the predetermined time. (Yes in S209), the starting point detection process shown in step S213 is performed. In this process, two straight lines extending from the position of the stagnant finger F (two straight lines that will intersect closest to the fingertip coordinates) and A point considered to be the intersection of the considered parts is detected as the starting point. As described above, when stagnant while touching the corner 34a, the line segment corresponding to the upper end 33a and the line segment corresponding to the upper end 33d extend from the stagnant position of the finger F. Therefore, the corner 34a touched by the finger F is detected as the starting point.

Next, a search target setting process shown in step S215 is performed, and one of the two straight lines extending from the stagnant finger F is set as a search target. Subsequently, in the search process of step S217, a process for searching the picked-up image for an intersection point at which the straight line set as the search target intersects with another straight line is performed.

In this embodiment, the direction in which the intersection is bent during searching is determined to be the clockwise direction when viewed from above the parts box. A straight line that becomes the first search target. As described above, when the finger F is stagnant at the corner 34a of the parts box 30a, a portion of the line segment corresponding to the upper end 33a and a portion of the line segment corresponding to the upper end 33d are separated from each other by the finger. Since it is detected as two straight lines extending from F, the line segment corresponding to the upper end 33a ahead in the clockwise direction becomes the search target (see arrow Sa in FIG. 6).

Then, when the tip (intersection point) of the straight line to be searched is detected in the search process of step S217, it is determined whether or not the detected intersection point matches the starting point in the determination process of step S219. be. Here, when a search is made for an intersection that is the tip of the line segment corresponding to the upper end 33a and that intersects the line segment corresponding to the upper end 33b, this intersection corresponds to the corner portion 34b and does not match the starting point. A determination of No is made in step S219.

In this case, the search target switching process of step S221 is performed, and the search target is switched to another straight line extending from the searched intersection. As described above, when the intersection corresponding to the corner portion 34b is searched, the line segment corresponding to the upper end 33b is switched to the search target (see arrow Sb in FIG. 6). Then, in the processing of step S217, a point of intersection which is the tip of the line segment corresponding to the upper end 33b and which intersects the line segment corresponding to the upper end 33c is searched. does not match, the determination at step S219 is No. For this reason, the line segment corresponding to the upper end 33c is switched to be the search target (S221: see arrow Sc in FIG. 6), and the line segment corresponding to the upper end 33d that is the tip of the line segment corresponding to the upper end 33c is searched. is searched (S217). Since this intersection corresponds to the corner portion 34d, it does not match the starting point (No in S219), so the line segment corresponding to the upper end 33d is switched to the search target (S221: see arrow Sd in FIG. 6). ), and a point of intersection at which the tip of the line segment corresponding to the upper end 33d intersects with the line segment corresponding to the upper end 33a is searched (S217). Since this intersection corresponds to the corner portion 34a and therefore coincides with the starting point, it is determined as Yes in the determination processing of step S219.

In this case, the boundary line setting process shown in step S223 is performed. In this process, a polygonal loop line connecting each line segment to be searched is detected and set as a boundary line extending from the starting point, bending at a plurality of intersections, and returning to the starting point. A polygonal area surrounded by is set as a monitoring area. As described above, when the boundary lines connecting the line segments corresponding to the upper ends 33a to 33d are detected and set, the quadrangular area surrounded by the boundary lines becomes the parts box containing the first component 20a. 30a is set as the monitoring area Pa (see FIG. 7A). Therefore, in FIG. 7A, the boundary line can be indicated by the symbol Pa. In addition, even when the finger F of the worker M is stagnant while touching the corner 34b (34c, 34d) of the parts box 30a, the corner 34b (34c, 34d) is used as a starting point and extended from this starting point to form a plurality of A quadrangular area surrounded by boundary lines that bend at intersections and then return to the starting point is set as the monitoring area Pa.

Here, the reason why the direction of bending at the intersection during searching is determined to be one direction (the clockwise direction when viewed from above the parts box) will be described in detail with reference to FIG.
As shown in FIG. 8, for example, it is assumed that the parts box 30e and the parts box 30f are arranged so that the peripheral wall of the parts box 30e and the peripheral wall of the parts box 30f are in contact with each other, and the monitoring area of the parts box 30e is set. do. In this case, in the search process, an unintended intersection point (see symbol Sp1 in FIG. 8) is detected because the upper end of the parts box 30e and the upper end of the parts box 30f are connected at a position close to the parts box 30f. , the upper end of the parts box 30f may be erroneously searched as a straight line extending from this intersection (see arrow Sf in FIG. 8). On the other hand, for example, when extending the boundary line so that the direction of bending at the intersection point is the clockwise direction when viewed from above the parts box, the unintended intersection point Sp1 bends toward the upper end of the parts box 30f. The direction is reversed to the counterclockwise direction.

Therefore, in this embodiment, the turning direction at the intersection is determined as the clockwise direction (one direction) as described above. As a result, as shown in FIG. 8, a straight line corresponding to the upper end of the parts box 30e extending from the unintended intersection Sp1 (see arrow Se in FIG. 8) is ahead of the intersection Sp1 in the clockwise direction, and extends from the unintended intersection Sp1. Since the straight line corresponding to the upper end of the parts box 30f (see arrow Sf in FIG. 8) is ahead of the intersection Sp1 in the counterclockwise direction, the upper end of the parts box 30f in the opposite direction is suppressed from being erroneously searched. be able to.

When the monitoring area setting process is completed by setting the monitoring area Pa for the parts box 30a containing the first part 20a as described above, in the determination process of step S107, the assembly order k is determined by the work procedure information. is equal to the total number N of parts boxes specified from . In the above example, since the total number N is set to 4, the determination in step S107 is No, and after the assembly order k is incremented by one (S109), the process of step S105 is performed. be. After the monitoring area is set for the parts box 30a containing the first part 20a as described above, k=2 is set, so the monitoring area is set for the parts box 30b containing the second part 20b. The process for doing so is started. Similarly, when a finger touches one of the corners of the parts box 30b containing the second part 20b and stays there, a monitoring area Pb is set for the parts box 30b. After that, since k is not equal to N (No in S107), after k is set to 3 (S109), the finger that touches any corner of the component box 30c containing the third component 20c stays. Thus, a monitoring area Pc is set for this parts box 30c (S105). After that, since k is not equal to N (No in S107), after k is set to 4 (S109), the finger that touches any corner of the component box 30d containing the fourth component 20d stagnate. By doing so, a monitoring area Pd is set for this parts box 30d (S105).

When the monitoring areas Pa to Pd are set for each of the four component boxes 30a to 30d in this way (see FIG. 7B), k=N, and it is determined Yes in step S107. A setting completion notification process shown in S111 is performed. In this process, the worker M is notified that all the monitoring areas have been set for the production lot according to the light emission state of the light emission unit 15 and the notification sound state of the speaker 16 . In the monitoring area setting process, each time one monitoring area is set, notification indicating completion of individual setting may be performed using the light emitting unit 15 or the speaker 16 .

When completion of setting of each monitoring area is notified in this manner, the monitoring process shown in step S113 is started. In this process, each of the monitoring areas Pa to Pd set as described above is monitored by performing dynamic detection for each of the monitoring areas Pa to Pd. Note that the control unit 11 that executes the dynamic detection can correspond to an example of a “detection unit”.

In the present embodiment, a method of observing time-series changes in each of the monitoring areas Pa to Pd using differences in images is adopted as the dynamic detection. Specifically, in the monitoring process, by comparing the latest captured image captured by the imaging unit 13 with another previously captured image (for example, the captured image captured immediately before), Dynamic detection of the monitored area is performed.

Based on the dynamic detection, for example, using the method disclosed in Japanese Patent Application Laid-Open No. 2018-156279, the operation is performed based on whether the change occurring in the monitoring area is large enough to be determined as a hand. It is determined whether or not the hand of person M has been detected. Note that the control unit 11 that determines whether or not work according to a predetermined work procedure is being performed based on the detection result of the dynamic detection can correspond to an example of a "determining unit."

For example, when a worker M who has finished touching one of the corners of the component boxes 30a to 30d in order takes out the component 20a from the component box 30a in order to actually start the assembly work, The hand of the worker M is detected within the monitoring area Pa based on the difference between the latest captured image and the immediately preceding captured image. Then, according to the work procedure information that has been read, when the detection result of detecting the hand of worker M is obtained in the order of monitoring area Pa, monitoring area Pb, monitoring area Pc, and monitoring area Pd, correct assembly work is performed. However, as a result of the determination, a notification indicating normal operation is made. As this normal notification, for example, green lighting by the light-emitting unit 15 to make the user aware of the correct answer, or the sound of "ping-pong" by the speaker 16 to make the user aware of the correct answer, or the like can be adopted. The worker M who has received this normal notification can recognize that the parts have been assembled according to the correct work procedure. The notification of normality may be made for each monitored area, or may be made by detecting the hand of the worker M in all the monitored areas according to the work procedure information.

On the other hand, for example, after the hand of worker M is detected in the monitoring area Pa, the hand of the worker M is detected in the monitoring area Pc, so that the hand of the worker M is detected in the monitoring areas in the wrong order. is detected, it is judged that an erroneous assembly operation has been performed, and a notification indicating an abnormal operation is made as a determination result. As the abnormality notification, for example, red lighting by the light-emitting unit 15 to make the user aware of the abnormality, or a sound such as "Please pick up the correct part" by the speaker 16, or the like can be adopted. The worker M who has received this anomaly report can immediately recognize that the part has been taken out in an erroneous work procedure.

As described above, in the work support device 10 according to the present embodiment, the part corresponding to the monitoring area (Pa to Pd) set by the monitoring area setting process in the captured image captured by the imaging unit 13 and the corresponding image Parts (20a to 20d) from parts boxes (30a to 30d) corresponding to the monitoring area based on the result of comparison with parts corresponding to the monitoring area in other captured images captured at a time before the image. removal is detected. Based on this detection result, it is determined whether or not the work according to the predetermined work procedure is being performed, and the operator M is notified of this determination result. At that time, in the captured image captured by the imaging unit 13 in the monitoring area setting process, the operator M's finger F Boundary lines (Pa to Pd) are detected, extending from the corners touched by , bending at a plurality of intersections, and then returning to the starting points, and a polygonal area surrounded by the boundary lines is used as a monitoring area. , is set for each parts box.

The work support program according to the present embodiment is a work support device 10 that supports work performed according to a predetermined work procedure using parts (20a to 20d) housed in a plurality of parts boxes (30a to 30d). A monitoring area setting process (S105) for setting a monitoring area (Pa to Pd) for each parts box within the imaging range of the imaging unit 13 in the control unit 11; Corresponding to the monitoring area based on the result of comparison between the part corresponding to the monitoring area set by the area setting process and the part corresponding to the monitoring area in another captured image captured at a time before the current captured image. a detection process (S113) for detecting removal of parts from a parts box, a determination process (S113) for determining whether or not work according to a predetermined work procedure is being performed based on the detection result of the detection process (S113); It is a program for executing a notification process (S113) for notifying the operator of the determination result by the process. In the monitoring area setting process, in the captured image captured by the imaging unit 13, one of the plurality of corners 34a to 34d on the upper end 33a to 33d of the peripheral wall 32 of the parts box is touched by the finger F of the worker M. Boundary lines (Pa to Pd) are detected that extend from the starting point and return to the starting point after bending at a plurality of intersections, and a polygonal area surrounded by the boundary line is used as a monitoring area for each parts box. set.

As a result, the operator M can easily set the monitoring area for each parts box by simply touching one corner of the upper end of the peripheral wall of the parts box with the finger F before starting work. In particular, since the operator M does not need to operate the mouse while looking at the screen, an erroneous monitoring area is not set due to an input operation error of the operator M or the like. Therefore, the monitoring area can be easily and accurately set without imposing a work burden on the worker M.

In particular, in the monitoring area setting process, since the boundary lines (Pa to Pd) are detected so that the direction of bending at the intersection is determined in one direction, the direction from the unintended intersection (Sp1) as described above is different from the one direction. A line that extends to bend to the side can be excluded as a line that bends to the top edge of the peripheral wall of the adjacent parts box.

[Second embodiment]
Next, a work support device and a work support program according to the second embodiment will be described with reference to the drawings.
The second embodiment is mainly different from the first embodiment in that a plurality of corners of one parts box are touched with a stagnant finger when setting a monitoring area. Therefore, the same reference numerals are assigned to substantially the same components as in the first embodiment, and the description thereof will be omitted.

In this embodiment, by touching a plurality of corners of one parts box with a stagnant finger F, the detection accuracy of the boundary line for setting the monitoring area is enhanced. Specifically, in the monitoring area setting process, the boundary line is detected so that one of the plurality of corners touched by the finger F in the stagnant state is set as the starting point and the rest are at least part of the intersections. As a result, the boundary line returned to the original starting point so as not to pass through the corner touched by the rest of the finger F in the stagnant state can be determined as the erroneously detected boundary line, so that the detection accuracy of the boundary line can be improved. can be done. For example, when the finger F touches the corner portion 34a and the corner portion 34c in the order of the corner portion 34a and the corner portion 34c with respect to the parts box 30a, the boundary line starting from the corner portion 34a (corner portion 34c) becomes the corner portion 34c (corner portion 34c). If the position corresponding to the portion 34a) is not set as the intersection, it can be determined as an erroneously detected boundary line.

In addition, by predetermining in advance the corner touched by the stagnant finger F corresponding to the intersection to be detected, the detection accuracy of the boundary line for setting the monitoring area can be improved. can be enhanced. For example, it is determined in advance that after the stagnant finger F first touches the corner corresponding to the intersection to be detected second, the stagnant finger F touches the corner serving as the starting point. . Then, as illustrated in FIG. 9, after first touching the corner 34c of the parts box 30a with the stagnant finger F, by touching the corner 34a with the stagnant finger F, the parts box Even if an unintended intersection point Sp2 is detected by the part 20a in the part 30a, the second intersection point is set at a position corresponding to the corner 34c, so that an erroneous search is not performed to bend at the intersection point Sp2. . Therefore, it is possible to improve the detection accuracy of the boundary line for setting the monitoring area. In addition, in FIG. 9, for convenience, the parts box 30a is planarly viewed and one component 20a is illustrated in an exaggerated state.

[Third embodiment]
Next, a work support device and a work support program according to the third embodiment will be described with reference to the drawings.
The main difference between the third embodiment and the first embodiment is that the boundary line is detected as an annular portion having a predetermined width in the monitoring area setting process. Therefore, the same reference numerals are assigned to substantially the same components as in the first embodiment, and the description thereof will be omitted.

In this embodiment, the upper end of the peripheral wall of the parts box is detected as an annular portion having a predetermined width, and the inner edge of the annular portion is set as the search target. Specifically, when the corner 34a of the parts box 30a is touched by the stagnant finger F, in the starting point detection process shown in step S213, as shown in FIG. A point regarded as an intersection Spa between the corresponding line segment and the line segment corresponding to the inner edge 35d of the upper end 33d is detected as the starting point. Then, in the search target setting process shown in step S215 described above, first, the line segment corresponding to the inner edge 35a of the upper end 33a and the intersection point Spb at the tip thereof are the search targets. In the search target switching process of step S221 described above, the line segment corresponding to the inner edge 35b of the upper end 33b and the intersection point Spc that is the tip thereof, the line segment that corresponds to the inner edge 35c of the upper end 33c and the intersection point Spd that is the tip thereof, the upper end The search target is switched in the order of the line segment corresponding to the inner edge 35d of 33d. In FIG. 10, for the sake of convenience, the parts boxes 30a and 30b are viewed from above, the parts 20a and 20b are omitted, and the widths of the respective upper ends are exaggerated.

As a result, in the boundary line setting process shown in step S223, a rectangular area surrounded by the inner edges 35a to 35d is set as the monitoring area Pa of the parts box 30a. In this way, the boundary line is detected as an annular portion having a predetermined width, and the polygonal area surrounded by the inner edges (35a to 35d) of the annular portion is set as the monitoring area, as can be seen from FIG. Thus, the adjacent parts box (30b), which may be connected to the outer edge of the annular portion, does not affect the setting of the monitoring area (Pa), so the monitoring area can be set more accurately.

[Fourth embodiment]
Next, a work support device and a work support program according to the fourth embodiment will be described with reference to the drawings.
In the fourth embodiment, in the monitoring area setting process, the monitoring area is set according to the circular trajectory drawn by the operator's finger tracing the upper end of the peripheral wall of the parts box, unlike the first embodiment. different. Therefore, the same reference numerals are assigned to substantially the same components as in the first embodiment, and the description thereof will be omitted.

In the monitoring area setting process of the present embodiment, when the imaging unit 13 captures an image of a circular trajectory drawn by the finger F of the worker M tracing the upper edge of the peripheral wall of the parts box, the area surrounded by the circular trajectory is monitored. Set as a region. That is, by tracing the upper end of the peripheral wall of each parts box, the monitoring area can be set.

The monitoring area setting process in this embodiment will be described in detail below with reference to the flowchart shown in FIG.
As in the first embodiment, when the stagnation time of the finger F of the worker M who has touched the corner of the parts box has passed the predetermined time (Yes in S209 of FIG. 11), the finger F is touching. When the corner is detected as the starting point (S213), locus imaging processing shown in step S225 is performed. In this process, when the stagnant finger moves circularly, a process is performed to capture an image of the trajectory of the movement.

Then, for example, as shown in FIG. 12, the worker M moves his finger F from the corner 34a of the parts box 30a to the upper end 33a, the corner 34b, the upper end 33b, the corner 34c, the upper end 33c, and the corner 34d. , the upper end 33d, and the corner 34a in this order to capture an image of an annular trajectory (Pa), the area surrounded by this annular trajectory is set as the monitoring area Pa. Similarly, operator M traces the top edge of the peripheral wall of parts box 30b with finger F to set monitoring area Pb, and traces the top edge of the peripheral wall of parts box 30c to set monitoring area Pc. A monitoring area Pd is set by tracing the upper end of the peripheral wall 30d. In addition, in FIG. 12, for convenience, the parts box 30a is shown in plan view and the parts 20a are omitted.

As described above, in the work support device 10 according to the present embodiment, the part corresponding to the monitoring area (Pa to Pd) set by the monitoring area setting process in the captured image captured by the imaging unit 13 and the corresponding image Parts (20a to 20d) from parts boxes (30a to 30d) corresponding to the monitoring area based on the result of comparison with parts corresponding to the monitoring area in other captured images captured at a time before the image. removal is detected. Based on this detection result, it is determined whether or not the work according to the predetermined work procedure is being performed, and the operator M is notified of this determination result. At that time, in the monitoring area setting process, when the imaging unit 13 captures an image of a circular trajectory drawn by the finger F of the worker M tracing the upper end (33a to 33d) of the peripheral wall of the parts box, the circular trajectory An enclosed area is set for each parts box as a monitoring area.

The work support program according to the present embodiment is a work support device 10 that supports work performed according to a predetermined work procedure using parts (20a to 20d) housed in a plurality of parts boxes (30a to 30d). A monitoring area setting process (S105) for setting a monitoring area (Pa to Pd) for each parts box within the imaging range of the imaging unit 13 in the control unit 11; Corresponding to the monitoring area based on the result of comparison between the part corresponding to the monitoring area set by the area setting process and the part corresponding to the monitoring area in another captured image captured at a time before the current captured image. a detection process (S113) for detecting removal of parts from a parts box, a determination process (S113) for determining whether or not work according to a predetermined work procedure is being performed based on the detection result of the detection process (S113); It is a program for executing a notification process (S113) for notifying the operator of the determination result by the process. In the monitoring area setting process, when the imaging unit 13 captures an image of a circular locus drawn by the finger F of the worker M tracing the upper end (33a to 33d) of the peripheral wall of the parts box, the area is surrounded by the circular locus. An area is set as a monitoring area for each parts box.

As a result, the operator M can easily set the monitoring area for each parts box by simply tracing the upper edge of the peripheral wall of the parts box with the finger F before starting the work. In particular, since the operator does not need to operate the mouse while looking at the screen, an erroneous monitoring area is not set due to an input operation error by the operator M or the like. Therefore, the monitoring area can be easily and accurately set without imposing a work burden on the worker M.

Note that in the monitoring region setting process in the present embodiment, when an annular trajectory is imaged by the finger F of the worker M without detecting the stagnation of the finger F of the worker M, the area surrounded by this annular trajectory is detected. may be set as the monitoring area.

It should be noted that the present invention is not limited to the above-described embodiments and modifications thereof, and may be embodied as follows, for example.
(1) The parts box 30 to be monitored by being placed within the imaging range of the imaging unit 13 is not limited to the four parts boxes 30a to 30d, and may be one to three parts boxes. , may be five or more parts boxes. Further, the component boxes 30 are not limited to being arranged in a row so as to be horizontally aligned within the imaging range of the imaging unit 13, and may be arranged so as to be horizontally aligned in multiple stages such as an upper stage and a lower stage, for example. . In addition, the upper end of the peripheral wall of the parts box 30 to be monitored is not limited to being formed in a quadrangular ring shape having four corners 34a to 34d, and may have a plurality of corners such as a trapezoid. may be formed in a polygonal annular shape. In addition, if the parts box 30 to be monitored has a shape in which the upper end of the peripheral wall can be recognized as an annular line or an annular portion having a predetermined width from the captured image, the arrangement method, etc., such as a storage bag, etc. Depending on the configuration, the shape of the upper end of the peripheral wall may be changed.

(2) The present invention is not limited to being adopted for work support when assembling parts to a work W such as a printed circuit board according to a predetermined work procedure. It may be employed for work support when assembling parts according to a predetermined work procedure.

10 work support device 11 control unit (monitoring area setting unit, detection unit, determination unit)
13... Imaging unit 15... Light emitting unit (notifying unit)
16...Speaker (notification unit)
20, 20a to 20d Parts 30, 30a to 30d Parts box 32 Peripheral wall 33a to 33d Upper edge 34a to 34d Corner 35a to 35d Inner edge F Finger M Operator Pa to Pd Monitoring area

Claims (7)

A work support device that supports work performed according to a predetermined work procedure using parts stored in a plurality of parts boxes,
an imaging unit;
a monitoring area setting unit that sets a monitoring area for each component box within an imaging range of the imaging unit;
A portion corresponding to the monitoring area set by the monitoring area setting unit in the captured image captured by the imaging unit and the monitoring area corresponding to another captured image captured at a time before the captured image. a detection unit that detects removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
a determination unit that determines whether or not the work according to the predetermined work procedure is being performed based on the detection result of the detection unit;
a notification unit that notifies a worker of the determination result by the determination unit;
with
The upper end of the peripheral wall of the parts box is formed in a polygonal annular shape with a plurality of corners,
The monitoring area setting unit extends from a corner that is touched by a worker's finger among the plurality of corners in the captured image captured by the imaging unit and bends at a plurality of intersections. A work support device, wherein a boundary line returning to the starting point is detected, and a polygonal area surrounded by the boundary line is set as the monitoring area for each parts box. 3. The monitoring area setting unit detects the boundary line so that one of the plurality of corners touched by the finger is the starting point and the remaining corners are at least part of the intersection. 1. The work support device according to 1. 3. The work support device according to claim 1, wherein the monitoring area setting unit detects the boundary line so as to define a bending direction at the intersection as one direction. The monitoring area setting unit detects the boundary line as an annular portion having a predetermined width, and sets a polygonal area surrounded by an inner edge of the annular portion as the monitoring area. 4. The work support device according to any one of 3. A work support device that supports work performed according to a predetermined work procedure using parts stored in a plurality of parts boxes,
an imaging unit;
a monitoring area setting unit that sets a monitoring area for each component box within an imaging range of the imaging unit;
A portion corresponding to the monitoring area set by the monitoring area setting unit in the captured image captured by the imaging unit and the monitoring area corresponding to another captured image captured at a time before the captured image. a detection unit that detects removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
a determination unit that determines whether or not the work according to the predetermined work procedure is being performed based on the detection result of the detection unit;
a notification unit that notifies a worker of the determination result by the determination unit;
with
When the imaging unit captures an image of a circular trajectory drawn by a worker's finger tracing the upper edge of the peripheral wall of the parts box, the monitoring region setting unit sets the region surrounded by the circular trajectory as the monitoring region, A work support device characterized by setting for each parts box. A control unit of a work support device that supports work performed according to a predetermined work procedure using parts stored in a plurality of parts boxes,
a monitoring area setting process for setting a monitoring area for each of the parts boxes within an imaging range of an imaging unit;
A part corresponding to the monitoring area set by the monitoring area setting process in the captured image captured by the imaging unit and the monitoring area corresponding to the monitoring area in another captured image captured at a time before the captured image. a detection process for detecting removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
Determination processing for determining whether or not work according to the predetermined work procedure is being performed based on the detection result of the detection processing;
A notification process for notifying the operator of the determination result of the determination process;
A work support program for executing
The upper end of the peripheral wall of the parts box is formed in a polygonal annular shape with a plurality of corners,
In the monitoring area setting process, in the captured image captured by the imaging unit, the corner portion that is touched by the operator's finger among the plurality of corner portions is extended as a starting point, and then bent at a plurality of intersection points. A work support program characterized by detecting a boundary line returning to the starting point, and setting a polygonal area surrounded by the boundary line as the monitoring area for each parts box. A control unit of a work support device that supports work performed according to a predetermined work procedure using parts stored in a plurality of parts boxes,
a monitoring area setting process for setting a monitoring area for each of the parts boxes within an imaging range of an imaging unit;
A part corresponding to the monitoring area set by the monitoring area setting process in the captured image captured by the imaging unit and the monitoring area corresponding to the monitoring area in another captured image captured at a time before the captured image. a detection process for detecting removal of the parts from the parts box corresponding to the monitoring area based on the result of comparison with the part to be monitored;
Determination processing for determining whether or not work according to the predetermined work procedure is being performed based on the detection result of the detection processing;
A notification process for notifying the operator of the determination result of the determination process;
A work support program for executing
In the monitoring area setting process, when a ring-shaped trajectory drawn by an operator's finger tracing the upper end of the peripheral wall of the parts box is imaged by the imaging unit, an area surrounded by the ring-shaped trajectory is set as the monitoring area, A work support program characterized by being set for each parts box.

Images