WO2013099373A1

WO2013099373A1 - Work management apparatus and work management system

Info

Publication number: WO2013099373A1
Application number: PCT/JP2012/073904
Authority: WO
Inventors: 浩延松下; 誠海田; 隆紀清水; 篤片山
Original assignee: 三菱電機エンジニアリング株式会社
Priority date: 2011-12-27
Filing date: 2012-09-19
Publication date: 2013-07-04
Also published as: JP2013132736A; CN104023906A; JP5930708B2; CN104023906B

Abstract

A camera (3) captures a starting point marker attached to a workpiece and a test marker attached to a torque wrench. A marker recognition unit (12) of the work management apparatus (10) recognizes the markers from the image data taken by the camera (3). A marker position-calculating unit (13) calculates the three-dimensional position of the test marker with respect to the three dimensional position of the starting point marker. A work correctness-assessing unit (17) assesses the work as being performed in the correct position if the three-dimensional position of the test marker calculated by the marker position-calculating unit (13) is within a screw tightening work range previously set in a work procedure storage unit (16) and notifies the worker via a reporting unit (18) that the work position is incorrect if the test marker position is not in the previously set range.

Description

Work management device and work management system

The present invention relates to a work management apparatus and a work management system that detect a work position using a captured image of an AR marker and determine whether the work is correct or incorrect.

For example, Patent Document 1 discloses a technique for specifying a position where screw tightening is performed when an operator performs screw tightening with a hand-held tightening tool. The tightening position detection device according to Patent Document 1 includes a deformable tape member that connects a reference position and a tightening tool, an optical fiber sensor that detects torsion and bending of the tape member, and an optical fiber provided in the tape member. A calculation unit that calculates a three-dimensional position of the tightening tool with respect to a reference position from the entire shape of the tape member based on a signal from the sensor.

JP 2010-85390 A

Since the conventional tightening position detection device is configured as described above, the tape member that connects the reference position and the tightening tool may come into contact with the member to be tightened and the operator, and there is a problem that workability is poor. It was. Moreover, since the optical fiber sensor provided in the tape member is always bent during work, there is a problem that the possibility of failure is high. Furthermore, since a dedicated tightening tool and a reference position are required for each tightening position detection device, there is a problem that the cost becomes high.

The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide a work management apparatus and a work management system that can be constructed at low cost without interfering with the work of an operator. And

The work management device according to the present invention acquires image data obtained by capturing a work area where a work is performed on a workpiece using a tool provided with a first marker, and the first marker included in the image is acquired. A marker recognizing unit for recognizing, a marker position calculating unit for calculating a three-dimensional position of the first marker recognized by the marker recognizing unit with respect to an origin set at a predetermined three-dimensional position, based on the image data; The work procedure storage unit in which information on the three-dimensional position of the tool to be performed is set, and the three-dimensional position of the first marker calculated by the marker position calculation unit are collated with the three-dimensional position of the tool in the work procedure storage unit. And an operation correctness determination unit for determining correctness of the position where the operation is performed.

Further, the work management device according to the present invention performs image processing on a workpiece using a tool provided with a first marker, and image data obtained by imaging a work area provided with a second marker at a predetermined position. And a marker recognition unit that recognizes the first marker and the second marker included in the image, and a three-dimensional position of the second marker recognized by the marker recognition unit based on the image data, A marker position calculation unit for calculating the three-dimensional position of the first marker recognized by the marker recognition unit with respect to the origin, a work procedure storage unit in which information on the three-dimensional position of the tool when performing the work is set, and a marker The apparatus includes a work correctness determination unit that compares the three-dimensional position of the first marker calculated by the position calculation unit with the three-dimensional position of the tool included in the work procedure storage unit to determine whether the position where the work is performed is correct.

Further, the work management system according to the present invention includes at least a marker attached to a tool, one or more imaging devices that image a work area in which work is performed on a workpiece using the tool, and an imaging device. The image processing apparatus includes the above-described work management apparatus that acquires image data and determines correctness of work.

According to the present invention, the three-dimensional position of this marker is calculated using image data obtained by imaging the marker attached to the tool, and collated with a preset three-dimensional position of the tool at the time of work, and correctness of work is confirmed. By determining, since it is not necessary to wire-connect the tool and the work management device, the worker can move to the action range without limitation. Moreover, since a general-purpose tool can be used, a system can be constructed at low cost.

It is a block diagram which shows the structure of the work management system which concerns on Embodiment 1 of this invention. 2 is a perspective view for explaining a management target of the work management system according to Embodiment 1. FIG. It is a figure which shows the example of the AR marker used with the work management system which concerns on Embodiment 1. FIG. 6 is a diagram illustrating an example of a captured image used by the work management apparatus according to Embodiment 1. FIG. 3 is a flowchart illustrating an operation of the work management apparatus according to the first embodiment. 6 is a diagram illustrating an example of a work history stored in a work history storage unit of the work management apparatus according to Embodiment 1. FIG. It is a figure explaining the process of the work correctness determination part of the work management apparatus which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the work management system which concerns on Embodiment 2 of this invention. It is a perspective view explaining the management object of the work management system concerning Embodiment 3 of this invention. It is a perspective view explaining the management object of the work management system concerning Embodiment 4 of this invention.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
The work management system shown in FIG. 1 takes an image of a worker attaching a fastening component such as a screw to a workpiece (work) using a hand-held tool, and monitors and manages the screw tightening work based on the image data. It is a system to do. FIG. 2 shows an example of the management target. Here, with respect to an operation in which an operator uses a torque wrench 2 to fasten a screw (not shown) in the screw holes 1a to 1d provided at the upper four corners of the work 1, A case of monitoring and managing the order of screw tightening will be described.

As shown in FIG. 2, screw holes 1a to 1d for fastening screws are provided at four corners on the upper surface of the workpiece 1. In addition, an origin marker 20 is attached to a predetermined position (for example, the center) of the upper surface of the work 1.
The shape of the workpiece 1 and the positions and number of the screw holes 1a to 1d are not limited to the illustrated example, and may be arbitrary.

An inspection marker 21 is attached to the torque wrench 2 which is a tool. In addition, a communication device 2a is attached to the torque wrench 2. When the tightening torque exceeds a predetermined torque value, that is, when the torque is increased, a signal indicating that the specified torque has been reached (hereinafter referred to as torque increase). Signal) and the torque value at that time. The torque-up signal and the torque value are received by the wirelessly connected receiver 2 b and output to the tool state detection unit 15 of the work management device 10.
The torque wrench 2 may not be configured to transmit both the torque up signal and the torque value at that time, but may be configured to transmit only the torque up signal or only the torque value.
Moreover, although the torque wrench 2 was illustrated as a tool, it is not limited to this, For example, an electric driver etc. may be sufficient. In this case as well, information such as a torque-up signal, torque value, and rotation speed can be output to the work management apparatus 10.

The origin marker 20 and the inspection marker 21 are so-called AR (Augmented Reality) markers. FIG. 3 shows another example of the AR marker.
As the AR marker, for example, a square two-dimensional image in which a black pattern is arranged on a white background is used. Different black patterns are drawn for the origin marker 20 and the inspection marker 21 on the white background portion surrounded by the black frame of the AR marker. In addition, this pattern has a shape that is not line-symmetric to distinguish the top, bottom, left, and right of the AR marker. Furthermore, the ratio of black frame: white background portion: black frame = 1: 2: 1 is set in the vertical direction and the horizontal direction. Further, since the AR marker is imaged by the camera 3 and image processing is performed by the work management apparatus 10, it is preferable to devise so that the surface of the AR marker does not shine.
The AR marker may have a shape other than those shown in FIGS. 2 and 3, and colors other than black and white may be used.

The camera 3 is fixed at a predetermined position with respect to the work 1 placed on the pedestal. The work area including the screw holes 1a to 1d of the work 1 is imaged at a predetermined sampling time interval to obtain image data. Output to the work management apparatus 10. As the camera 3, a general-purpose imaging device such as a WEB camera that can output image data to the work management device 10 in real time may be used.
In the example of FIG. 2, the upper surface of the work 1 is the XY plane, and the camera 3 is installed at a predetermined height position in the Z direction perpendicular to the XY plane.

FIG. 4 shows an example of a captured image. This image is an image taken at the timing when the screw tightening of the screw hole 1a is completed and the next screw hole 1b is being tightened. The origin marker 20 attached to the work 1 and the torque wrench 2 The inspection marker 21 attached to is shown.

Note that a polarizing filter may be attached to the camera 3 to reduce reflection of light, or a neutral density filter may be attached to lower the illuminance. Thereby, the origin marker 20 and the inspection marker 21 can be reliably imaged.

The work management apparatus 10 monitors and manages the screw tightening work using the torque wrench 2 by using the image data of the camera 3, the torque-up signal of the receiver 2b and the torque value at that time as input. For example, a personal computer is used as the work management apparatus 10, and a CPU (Central Processing Unit) executes a program stored in a program memory (not shown) to perform a marker recognition unit 12, a marker position calculation unit 13, a tool state detection. The processing contents of the unit 15, the work correctness determination unit 17, the notification unit 18, and the tool drive control unit 19 are executed.

Hereinafter, an example of the operation of the work management apparatus 10 will be described with reference to the flowchart shown in FIG. The work management apparatus 10 repeats the processes of steps ST1 to ST9 shown in FIG. 5 every predetermined sampling time.

In the marker definition storage unit 11, definition information for each AR marker is stored in advance. As the definition information of the origin marker 20, information on a pattern drawn on a white background portion of the origin marker 20, information indicating the origin position, and the like are set. As the definition information of the inspection marker 21, information on a pattern drawn on a white background portion of the inspection marker 21, information indicating the position of the torque wrench 2, information on the capability range of the torque wrench 2, and the like are set. Further, if there is a tool other than the torque wrench 2, an AR marker is attached, and definition information such as a tool type is set for each AR marker. When it is necessary to distinguish between torque wrenches with different capability ranges, different AR markers are attached to the respective torque wrenches, and definition information is set for each.

The marker recognizing unit 12 acquires image data from the camera 3 (step ST1), recognizes the image of the AR marker on the acquired image, calculates information such as a two-dimensional position, size, and shape, and Based on the definition information stored in the definition storage unit 11, the recognized AR marker is identified as the origin marker 20 or the inspection marker 21 (step ST2). Then, the recognition result is output from the marker recognition unit 12 to the marker position calculation unit 13.

The marker position calculation unit 13 calculates the three-dimensional position and orientation (direction, inclination, etc.) of the origin marker 20 and the inspection marker 21 based on the recognition result of the marker recognition unit 12 (step ST3). As a method for calculating the three-dimensional position and orientation of the AR marker, a known technique (for example, Japanese Patent Laid-Open No. 7-98214) may be used, and detailed description thereof is omitted.
At this time, the marker position calculation unit 13 uses the three-dimensional position of the origin marker 20 as the origin, and calculates the three-dimensional position of the inspection marker 21 with respect to this origin. Thereafter, the calculated three-dimensional position of the inspection marker 21 is treated as the position of the torque wrench 2.
Then, the position coordinates of the torque wrench 2 are output from the marker position calculation unit 13 to the work history storage unit 14.

The tool state detection unit 15 is an interface for inputting / outputting information to / from the receiver 2b. If the receiver 2b has received the torque-up signal and the torque value at that time, the tool state detection unit 15 acquires the information (step ST4) and outputs it to the work history storage unit 14.
Note that the timing of performing the process of step ST4 is not limited to this, and may be performed at an arbitrary timing before the process of step ST6 is performed. Further, in the operation example of FIG. 5, the processing of steps ST1 to ST3 is repeated every sampling time, and the process proceeds to processing after step ST6 with the acquisition of the torque-up signal as a trigger.

The work history storage unit 14 is input at any timing from the position coordinates (X, Y, Z coordinates with respect to the origin) of the torque wrench 2 for each sampling time input from the marker position calculation unit 13 and the tool state detection unit 15. Torque up signal and the torque value at that time are stored (step ST5).
FIG. 6 shows an example of the work history. As the position of the torque wrench 2 calculated by the marker position calculation unit 13, the values of the X coordinate, the Y coordinate, and the Z coordinate are registered. Further, “*” is registered at the time when the tool state detection unit 15 acquires the torque-up signal, and the torque value at that time is also registered. The screw tightening work position and the correctness / incorrectness determination result will be described later.
When the torque wrench 2 is a type that outputs only one of the torque-up signal and the torque value, only one of the information may be registered in the work history storage unit 14.

Note that the image data captured by the camera 3 may be output from the marker recognition unit 12 to the work history storage unit 14, and the image data may be accumulated in the work history storage unit 14. By saving all image data imaged at every sampling time or by saving image data imaged at the timing of receiving the torque-up signal, the work procedure can be confirmed after the screw tightening operation. Further, it can be left as evidence that the screws are fastened to all the screw holes 1a to 1d.
Alternatively, only the image data when the screw tightening operation for the screw holes 1a to 1d is completed may be stored. In this case, for example, when the screw tightening work is completed, a signal is automatically or manually output to the work management apparatus 10, and on the work management apparatus 10 side, image data when a work completion signal is input from the outside. Is stored in the work history storage unit 14. With this configuration, it is possible to leave as evidence that the screws are fastened to all the screw holes 1a to 1d while suppressing the data capacity.

Furthermore, when image data is accumulated in the work history storage unit 14, a search key associated with AR marker information may be assigned to the image data and stored. For example, when the marker recognition unit 12 recognizes the inspection marker 21, a search key representing the inspection marker 21 is assigned to the image data at that time, and the work history storage unit 14 records the search key and the image data. Thus, image data obtained by imaging the screw tightening process using the torque wrench 2 can be easily searched from the image data of all manufacturing processes of the workpiece 1 using the search key representing the inspection marker 21. it can.

The work procedure storage unit 16 stores in advance position information and order information of screw tightening work.
In the example of FIG. 2, the positional relationship between the camera 3 and the workpiece 1 is fixed, and the origin marker 20 is attached at a predetermined position of the workpiece 1, so that the positions of the screw holes 1a to 1d with respect to the origin on the captured image Can be obtained in advance. Based on the positions of the screw holes 1a to 1d obtained in advance, the coordinates of the screw tightening work position for performing the screw tightening work are determined and registered in the work procedure storage unit 16. In consideration of an error in calculating the three-dimensional position of the AR marker, a screw tightening work area may be provided by giving a width to the screw tightening work position. Here, the screw tightening work area for the screw hole 1a is A, the screw tightening work area for the screw hole 1b is B, the screw tightening work area for the screw hole 1c is C, and the screw tightening work area for the screw hole 1d is D.
Further, it is assumed that the order of the screw tightening work areas A, B, C, and D is registered as the order information.

The work correctness determination unit 17 refers to the work history storage unit 14 and the work procedure storage unit 16 to determine whether the position and order of the screw tightening work performed by the worker is correct (step ST6). Then, when it is determined that the position and order of the screw tightening work are correct (step ST7 “YES”), the work correctness determination unit 17 outputs the determination result to the work history storage unit 14 (step ST8). On the other hand, when it is determined that the position and order of the screw tightening work are wrong (step ST7 “NO”), information to that effect is output to the notification unit 18 (step ST9), and the result of the determination is stored in the work history storage unit 14. (Step ST8).

Specifically, the work correctness determination unit 17 collates the position of the torque wrench 2 stored in the work history storage unit 14 with the screw tightening work area stored in the work procedure storage unit 16 to determine the position of the torque wrench 2. Is in the screw tightening work area (or if it matches the screw tightening work position), it is determined that the screw tightening work is being performed at this position, and this screw tightening work area is stored in the work history storage unit. 14 to output. Thereby, in the work history shown in FIG. 6, position information such as A and B is registered in the item of the screw tightening work area at the time when it is determined that the screw tightening work is being performed.

FIG. 7 is a diagram for explaining correctness determination processing of the work correctness determination unit 17. The image in FIG. 7 corresponds to the captured image in FIG. 4, and the black circle is the origin calculated by the marker position calculation unit 13 based on the origin marker 20. The gray circle is the position of the torque wrench 2 at each time calculated by the marker position calculation unit 13 based on the inspection marker 21, and corresponds to the X, Y, and Z coordinates of the work history in FIG. A hatched area is a screw tightening work area with respect to the origin, which is registered in advance in the work procedure storage unit 16. For reference, positions corresponding to the screw holes 1a to 1d are indicated by crosses.
If the position of the torque wrench 2 indicated by the gray circle is within the screw tightening work area indicated by the oblique lines, the work correctness determination unit 17 determines that the screw tightening work is being performed. Although not shown, the screw tightening work area is set in the Z direction in addition to the X and Y directions, and when the torque wrench 2 is at a predetermined height or higher from the upper surface of the workpiece 1. It is determined that the screw tightening operation is not performed.

Further, the work correctness / incorrectness determination unit 17 performs the order of the screw tightening work area (the screw tightening work position A at time 8:01:00 in the example of FIG. 6) at the torque up signal acquisition time stored in the work history storage unit 14. Are compared with the order information stored in the work procedure storage unit 16, and if they match, it is determined that the screw tightening work has been performed in the correct order. As a result, it is possible to detect a case where the operator makes a mistake in the order of the screw tightening work or a case where the operator forgets the tightening.
In this example, the acquisition of the torque up signal is regarded as the completion of the screw tightening operation. However, when there is no information indicating the completion of the screw tightening operation such as a torque up signal, the position of the torque wrench 2 is the screw tightening operation. It may be determined whether or not the screw tightening operation has been performed according to the length of time matching the region. Further, if the tool generates a sound during the screw tightening operation such as an electric screwdriver, it may be determined whether or not the screw tightening operation is completed depending on the presence or absence of the sound. Or you may use the rotation speed of an electric driver, the signal of completion | finish of rotation, etc.

Subsequently, when the work correctness determination unit 17 determines that the order of the screw tightening work is correct, the work correctness determination unit 17 registers “◯” indicating correctness in the item of the correctness determination result in the work history storage unit 14. On the other hand, if it is determined that it is incorrect, “x” indicating an error is registered in the item of the correctness / incorrectness determination result in the work history storage unit 14, and screw tightening operations are performed in an incorrect order to the notification unit 18. The information which shows that is output. Further, information indicating the position of correct screw tightening work may be output.

The work correctness determination unit 17 may further determine whether the item is other than the position and the order. For example, if the torque wrench 2 is a type that outputs a torque value, the work correctness determination unit 17 compares the torque value stored in the work history storage unit 14 with the torque value set in the work procedure storage unit 16 in advance. Then, it may be determined whether or not the screw tightening operation has been performed with a correct torque value, and the appropriateness determination result of the torque value may be registered in the operation history storage unit 14.

Further, for example, the work correctness determination unit 17 determines which tool is used (torque wrench, electric screwdriver, etc.) based on the definition information of the inspection marker 21, and registers the tool type in the work history storage unit 14. You may make it do. Further, even for the same type of tool, the capability range (a torque wrench for 50 Nm, a torque wrench for 100 Nm, etc.) is determined based on the definition information of the inspection marker 21, and the suitability of the tool is registered in the work history storage unit 14. You may do it.
For example, in FIG. 6, the tool ID “2” of the torque wrench 2 is registered in the tool ID item.

Also, for example, correct / incorrect determination can be made when the screw tightening operation is performed in two stages of temporary tightening and final tightening. In this case, as the order information in the work procedure storage unit 16, the order of the temporary tightening screw tightening work areas A, B, C, and D, the torque value at that time, and the final tightening screw tightening work areas A, B, C, The order of D and the torque value at that time are stored in advance. Then, the work correctness determination unit 17 determines whether each screw tightening work of temporary tightening and final tightening has been performed according to the information of the work procedure.

The notification unit 18 is an interface that outputs an appropriate signal to a notification device such as a display device or a speaker, and presents information to the worker (step ST9). When the information is input from the work correctness determination unit 17, the notification unit 18 displays information indicating that the screw tightening work is performed in the wrong order or the wrong torque value, or outputs the information. . Moreover, the alerting | reporting part 18 may display the information which shows the position of the correct screwing operation | work, or a correct torque value on a screen, or may output by audio | voice.

In the operation example of FIG. 5, the correctness / incorrectness of the screw tightening work position is determined after obtaining the torque-up signal, but the determination timing is not limited to this. For example, every time the position of the torque wrench 2 is calculated in step ST3 of FIG. 5, the work correctness determination unit 17 may perform the correctness determination of this position and notify the operator of the determination result from the notification unit 18. Thereby, the operator can know whether the position of the screw tightening work is correct or wrong before the start of the screw tightening work.

Next, an example of using a tool that is driven by receiving a drive permission signal from the outside will be described. Here, the torque wrench 2 is driven only when the communication device 2a receives a drive permission signal from the outside (that is, the work management device 10), and when the torque is increased, either the torque up signal or the torque value is set. Alternatively, both are transmitted from the communication device 2a to the receiver 2b.

The tool drive control unit 19 of the work management apparatus 10 refers to the tool suitability determination result registered in the work history storage unit 14, so that the capability range of the torque wrench 2 is preset in the work procedure storage unit 16. If it is within the capability range, a drive permission signal is output to the communication device 2a. As a result, the screw tightening operation can be started only when the operator selects the torque wrench 2 in an appropriate capability range, and the screw tightening operation using the torque wrench 2 in the incorrect capability range can be prevented.

Further, the tool drive control unit 19 determines whether or not the position of the torque wrench 2 for each sampling time registered in the work history storage unit 14 is in an appropriate screw tightening work position, and at the appropriate screw tightening work position. If there is, a drive permission signal is output to the communication device 2a. As a specific example, for example, when the position coordinate of the screw tightening work position A is registered at time 8:00:20 in FIG. 6, the tool drive control unit 19 compares the position coordinate with the position information in the work procedure storage unit 16. Then, it is determined whether or not the position of the torque wrench 2 is correct. If it is correct, a drive permission signal is output to the torque wrench 2. Subsequently, the tool drive control unit 19 stops the output of the drive permission signal based on the torque up signal at time 8: 00: 1: 00, and further, based on the position coordinates of the screw tightening work position B at time 8:01:30. A drive permission signal is output. As a result, the screw tightening work can be started only when the operator moves the torque wrench 2 to an appropriate screw tightening work position, and the screw tightening work to the wrong screw holes 1a to 1d can be performed in the wrong order. Can be prevented.

As described above, according to the first embodiment, the work management device 10 is a work area where the work 1 is screwed to the work 1 using the torque wrench 2 provided with the inspection marker 21, and the origin marker is at a predetermined position. A marker recognizing unit 12 that acquires image data obtained by imaging a work area with 20 attached from the camera 3 and recognizes the origin marker 20 and the inspection marker 21 included in the image based on the definition information in the marker definition storage unit 11; Based on the image data, a marker position calculation unit that calculates the three-dimensional position of the inspection marker 21 recognized by the marker recognition unit 12 with respect to the origin using the three-dimensional position of the origin marker 20 recognized by the marker recognition unit 12 as an origin. 13, the screw tightening work areas A to D as the information of the three-dimensional position of the torque wrench 2 when performing the screw tightening work, and the order of performing the screw tightening work; The work procedure storage unit 16 in which information indicating the movement order of the screw tightening work areas A to D is set, the three-dimensional position of the inspection marker 21 calculated by the marker position calculation unit 13 and the movement order thereof are stored in the work procedure storage unit 16. The screw tightening work areas A to D and the moving order thereof are collated with each other, and a work correct / incorrect determination unit 17 for determining the correctness of the position and order of the screw tightening work is provided. For this reason, it is not necessary to wire-connect the torque wrench 2 and the work management device 10 in order to determine whether the position and order of the screw tightening work are correct, and the worker can move without limitation to the action range. In addition, since a general-purpose tool such as a torque wrench 2 and an electric driver and a general-purpose camera 3 can be used, a system can be constructed at low cost.

Further, according to the first embodiment, the work management apparatus 10 notifies the worker of the determination result of the work correctness determination unit 17 and moves the screw tightening work areas A to D included in the work procedure storage unit 16 in the moving order. Based on this, a notification unit 18 for instructing the position of the next screw tightening operation is provided. For this reason, an operator can be made to perform a screw fastening operation | work in a correct procedure.

In addition, according to the first embodiment, the work management apparatus 10 acquires a torque value signal indicating that the torque value at the time of completion of the tightening or a specified torque output from the torque wrench 2 has been reached. And the three-dimensional position of the inspection marker 21 calculated by the marker position calculation unit 13 and the torque value at the time of completion of fastening acquired by the tool state detection unit 15 or a torque up signal indicating that the specified torque has been reached as date information Since it is configured to include the work history storage unit 14 that stores it together, traceability can be managed.
Further, torque value information is set for each screw tightening work area A to D of the torque wrench 2 when the screw tightening work is performed in the work procedure storage unit 16, and the work correctness determination unit 17 determines whether the screw tightening work is correct or incorrect. Since the torque value at the completion of fastening acquired by the tool state detection unit 15 and the torque value of the work procedure storage unit 16 are collated when determining the position of the screw tightening work, The correctness / incorrectness of the torque value can also be determined.

Further, according to the first embodiment, since the work history storage unit 14 is configured to store image data obtained by imaging the work area, the work procedure can be confirmed after the work.
Furthermore, since the work history storage unit 14 is configured to store the image data together with the search key associated with the inspection marker 21 included in the image, the target image is checked when the work procedure is confirmed. You can search easily.

Further, according to the first embodiment, the camera 3 that captures the work area is configured to be attached with the filter that reduces the reflection of light in the work area, so that the origin marker 20 and the inspection marker 21 can be reliably imaged. Can do.

In the first embodiment, the origin marker 20 is attached to the work 1. However, the present invention is not limited to this, and is not limited to this, but a place where the positional relationship with the work 1 is fixed, such as a pedestal on which the work 1 is placed. It may be attached.

In addition, the three-dimensional position of the inspection marker 21 is calculated using the three-dimensional position of the origin marker 20 as the origin, but the origin marker 20 for defining the origin is not essential. When the positional relationship between the camera 3 and the workpiece 1 is fixed as in the first embodiment, the origin marker 20 can be omitted and the three-dimensional position of the virtual origin can be set in advance. is there. In this case, the marker position calculation unit 13 may calculate the three-dimensional position of the inspection marker 21 with respect to the virtual origin.

Embodiment 2. FIG.
FIG. 8 is a block diagram showing the configuration of the work management system according to Embodiment 2 of the present invention. The same or corresponding parts as those in FIG. In the second embodiment, a plurality of cameras 3-1 and 3-2 are used to image the workpiece 1 from a plurality of directions.

The work management apparatus 10 performs the same processing as that in the first embodiment on each of the image data captured by the plurality of cameras 3-1 and 3-2 to check whether the position and order of the screw tightening work are correct. Determine.
Note that the origin marker 20 for the camera 3-1 and the origin marker 20 for the camera 3-2 may be attached to one work 1, or the common imaging of the camera 3-1 and the camera 3-2. One origin marker 20 may be attached within the range. Alternatively, as described in the first embodiment, the origin marker 20 may be omitted and a virtual origin may be set.
Further, the number of cameras 3-1 and 3-2 may be three or more.

As described above, according to the second embodiment, the marker recognizing unit 12 of the work management apparatus 10 acquires a plurality of image data obtained by imaging a work area from a plurality of directions, and at least one image among the plurality of images. The marker position calculation unit 13 is configured to calculate the three-dimensional position of the inspection marker 21 with respect to the origin based on the image data in which the inspection marker 21 is recognized. Thus, by arranging a plurality of cameras 3-1 and 3-2 so that there is no blind spot with respect to the work 1, it is possible to take an image regardless of the position of the worker and the work 1. Whether the screw tightening operation is correct or incorrect can be determined.
In addition, by imaging the workpiece 1 from a plurality of angles, there is an effect of suppressing a decrease in image processing accuracy of the workpiece 1 due to light reflection.

Embodiment 3 FIG.
FIG. 9 shows an example of a management target of the work management system according to Embodiment 3 of the present invention. In the third embodiment, the workpiece 1 is freely rotatable in the clockwise direction, and the worker rotates the workpiece 1 to change the posture and perform the screw tightening operation. When the posture of the workpiece 1 is changed, two origin markers 20-1 and 20-2 are attached at predetermined positions of the workpiece 1 as shown in FIG.

The work management apparatus according to the third embodiment has the same configuration as that of the work management apparatus 10 shown in FIG. 1 and will be described below with reference to FIG.
In the work management apparatus 10, definition information of the origin markers 20-1 and 20-2 and the inspection marker 21 is stored in advance in the marker definition storage unit 11. The marker recognizing unit 12 recognizes the AR marker on the image acquired from the camera 3 and determines whether the AR marker is the origin marker 20-1 or the origin marker 20-2 based on the definition information in the marker definition storage unit 11. The inspection marker 21 is identified.

The marker position calculation unit 13 calculates the three-dimensional positions and orientations of the origin markers 20-1 and 20-2 and the inspection marker 21 based on the recognition result of the marker recognition unit 12. At this time, the marker position calculation unit 13 uses the three-dimensional position of either one of the origin markers 20-1 and 20-2 as the origin, and determines the three-dimensional position relative to the other origin of the origin markers 20-1 and 20-2. calculate. Further, the marker position calculation unit 13 calculates the three-dimensional position of the inspection marker 21 relative to the origin based on the triangulation principle based on the three-dimensional positions of the origin markers 20-1 and 20-2. The calculated three-dimensional position of the inspection marker 21 is stored in the work history storage unit 14 as the position of the torque wrench 2.
The subsequent processing is the same as that in the first embodiment, and a description thereof will be omitted.

In the illustrated example, two origin markers 20-1 and 20-2 are used to calculate the three-dimensional position of the inspection marker 21 using the principle of triangulation, but three or more origin markers may be used.
Further, although the method of the third embodiment is applied when the workpiece 1 moves with respect to the fixed camera 3, it can also be applied when the camera 3 moves with respect to the fixed workpiece 1. is there.

As described above, according to the third embodiment, the marker recognizing unit 12 of the work management apparatus 10 acquires image data obtained by capturing the work area to which the origin markers 20-1 and 20-2 are attached, and includes them in the image. The marker position calculation unit 13 recognizes the origin markers 20-1 and 20-2 and the inspection marker 21 to be detected, and uses the three-dimensional positions of the origin markers 20-1 and 20-2 recognized by the marker recognition unit 12 as origins. The three-dimensional position of the inspection marker 21 is calculated using the principle of triangulation. For this reason, also when changing the attitude | position of the workpiece | work 1 at the time of screw fastening operation | work, a work position can be detected and the correctness of the position and order of work can be determined.

In the third embodiment, the position of the torque wrench 2 can be detected when the positions of the work 1 and the camera 3 change relatively, but the work 1 as in the first embodiment. This is also applicable when the positional relationship between the camera 3 and the camera 3 is fixed. In this case, the detection accuracy of the position of the torque wrench 2 is improved.

Embodiment 4 FIG.
FIG. 10 shows an example of a management target of the work management system according to Embodiment 4 of the present invention. In the fourth embodiment, it is assumed that screws are fastened to the

screw holes

1e and 1f in the previous process, and screws are fastened to the screw holes 1a to 1d in this process. If there is a deficiency such as forgetting to tighten the screw in the previous process, a caution marker 22 is attached to the workpiece 1 as shown in FIG.
In this process, the operator first performs screw tightening of the

screw holes

1e and 1f on the work 1 provided with the caution marker 22, and then screw tightens of the screw holes 1a to 1d as in the first embodiment. We will do the work.

Since the work management apparatus according to the fourth embodiment has the same configuration as that of the work management apparatus 10 shown in FIG. 1, the following description will be given with reference to FIG.
In the work management apparatus 10, definition information of the origin marker 20, the inspection marker 21, and the attention marker 22 is stored in the marker definition storage unit 11 in advance. As the definition information of the caution marker 22, information of a pattern drawn on a white background part for identifying the caution marker 22, information indicating that the work 1 is defective in the previous process, and the like are set.

Further, in the work procedure storage unit 16, in addition to information on the work procedure indicating the position and order of the screw tightening work for the screw holes 1a to 1d in this process, the screw tightening work is performed for the

screw holes

1e and 1f of the previous process. Information on the work procedure indicating the position and the order is stored in advance. Furthermore, for the work 1 with the attention marker 22 attached thereto, information indicating that the work procedure is performed in the order of the work procedure of the previous process and the work procedure of this process, and the work 1 without the attention marker 22 Information indicating that only the work procedure of this process is performed is stored in advance.

The marker recognizing unit 12 recognizes the AR marker on the image acquired from the camera 3 and determines whether the AR marker is the origin marker 20, the inspection marker 21, or the attention marker 22 based on the definition information in the marker definition storage unit 11. Identify. At this time, when the attention marker 22 is identified, attention information indicating that the attention marker 22 is attached to the work 1 is output from the marker recognition unit 12 to the work history storage unit 14. The work history storage unit 14 is input from the marker recognition unit 12 in addition to the torque wrench 2 position input from the marker position calculation unit 13 and the torque up signal input from the tool state detection unit 15 and the torque value at that time. Remember attention information.

The work correctness determination unit 17 switches the information of the work procedure according to whether or not the caution information is registered in the work history storage unit 14, and performs the correctness determination of the screw tightening work. Specifically, the work correctness determination unit 17 first checks whether or not the caution information is registered in the work history storage unit 14, and if the caution information is not registered, the same as in the first embodiment. Then, it is determined whether or not the screw tightening work is performed in the order of the screw holes 1a to 1d, and the correctness / incorrectness determination result is output to the work history storage unit 14. On the other hand, when the caution information is registered, the information of the work procedure to be referred is switched, it is determined whether or not the screw tightening work is performed in the order of the

screw holes

1e, 1f, 1a to 1d, and the correct / incorrect determination result Is output to the work history storage unit 14.

When the information indicating that the screw tightening operations are performed in the wrong order is input from the work correctness determination unit 17, the notification unit 18 presents information indicating that to the worker. Thereby, for example, when the worker does not notice the defect of the work 1 and starts the screw tightening operation of the screw hole 1a according to the normal work procedure, the correct screw tightening work position (screw hole 1e) is notified to the worker. be able to.
Since other processes are the same as those in the first embodiment, description thereof is omitted.

Note that the work performed on the workpiece 1 provided with the caution marker 22 is not limited to the example described above.

As described above, according to the fourth embodiment, the work procedure storage unit 16 of the work management apparatus 10 has a plurality of pieces of information for determining the correctness of the screw tightening work, and the marker recognizing unit 12 adds new attention to the image. When the marker 22 is included, the attention marker 22 is recognized, and the work correctness determination unit 17 performs the screw tightening operation depending on whether the marker recognition unit 12 recognizes the attention marker 22 or not. It is configured to perform the determination by switching information for determining correctness. For this reason, the correctness of the work can be determined not only in the normal work procedure but also in an irregular work procedure such as when the work 1 is defective.

In the present invention, within the scope of the invention, any combination of the embodiments, any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

The work management device and the work management system according to the present invention acquire image data obtained by imaging a work area, recognize a first marker included in the image, and a first recognition recognized by the marker recognition unit. Marker position calculation unit for calculating the three-dimensional position of the marker, work procedure storage unit in which information on the three-dimensional position of the tool when performing the work is set, and the three-dimensional of the first marker calculated by the marker position calculation unit It is not necessary to connect the tool to the work management device by wire, because the position is collated with the three-dimensional position of the tool in the work procedure storage unit and the work correctness judgment unit for judging the correctness of the work position is provided. Since the person can move without limitation to the action range, the present invention can be applied to a work management apparatus and a work management system that determine whether work is correct or incorrect.

1 work (workpiece), 1a to 1f screw hole, 2 torque wrench (tool), 2a communication device, 2b receiver, 3, 3-1, 3-2 camera, 10 work management device, 11 marker definition storage unit, 12 marker recognition unit, 13 marker position calculation unit, 14 work history storage unit, 15 tool state detection unit, 16 work procedure storage unit, 17 work correctness determination unit, 18 notification unit, 19 tool drive control unit 20, 20-1, 20-2 Origin marker (second and third markers), 21 Inspection marker (first marker), 22 Caution marker.

Claims

A marker recognizing unit that obtains image data obtained by imaging a work area in which a work is performed on a workpiece using a tool provided with a first marker, and recognizes the first marker included in the image;
A marker position calculation unit that calculates a three-dimensional position of the first marker recognized by the marker recognition unit with respect to an origin set at a predetermined three-dimensional position based on the image data;
A work procedure storage unit in which information of the three-dimensional position of the tool when performing the work is set;
An operation correctness determination unit that compares the 3D position of the first marker calculated by the marker position calculation unit with the 3D position of the tool included in the operation procedure storage unit and determines whether the operation is performed correctly A work management apparatus comprising:
Work is performed on a workpiece using a tool provided with a first marker, image data obtained by imaging a work area provided with a second marker at a predetermined position is acquired, and the first data included in the image is obtained. A marker recognition unit for recognizing one marker and the second marker;
Based on the image data, the three-dimensional position of the second marker recognized by the marker recognition unit is used as the origin, and the three-dimensional position of the first marker recognized by the marker recognition unit with respect to the origin is calculated. A marker position calculator,
A work procedure storage unit in which information of the three-dimensional position of the tool when performing the work is set;
An operation correctness determination unit that compares the 3D position of the first marker calculated by the marker position calculation unit with the 3D position of the tool included in the operation procedure storage unit and determines whether the operation is performed correctly A work management apparatus comprising:
The work procedure storage unit has information indicating the movement order of the three-dimensional position of the tool as the order of performing the work,
The work correctness determination unit compares the movement order of the three-dimensional position of the first marker calculated by the marker position calculation unit with the movement order of the three-dimensional position of the tool included in the work procedure storage unit, and The work management apparatus according to claim 1, wherein correctness / incorrectness of the order of performing the determination is determined.
The work procedure storage unit has information indicating the movement order of the three-dimensional position of the tool as the order of performing the work,
The work correctness determination unit compares the movement order of the three-dimensional position of the first marker calculated by the marker position calculation unit with the movement order of the three-dimensional position of the tool included in the work procedure storage unit, and The work management apparatus according to claim 2, wherein correctness / incorrectness of the order of performing the determination is determined.
The marker recognition unit acquires a plurality of image data obtained by imaging the work area from a plurality of directions, recognizes the first marker included in at least one of the plurality of images,
The work management apparatus according to claim 1, wherein the marker position calculation unit calculates a three-dimensional position of the first marker with respect to the origin based on image data in which the first marker is recognized. .
The marker recognition unit acquires a plurality of image data obtained by imaging the work area from a plurality of directions, recognizes the first marker included in at least one of the plurality of images,
The work management apparatus according to claim 2, wherein the marker position calculation unit calculates a three-dimensional position of the first marker with respect to the origin based on image data in which the first marker is recognized. .
The marker recognizing unit acquires image data obtained by capturing a work area provided with the second marker and the third marker, and recognizes the second marker and the third marker included in the image. ,
The marker position calculation unit uses the respective three-dimensional positions of the second marker and the third marker recognized by the marker recognition unit as the origin, and uses the triangulation principle to calculate the three-dimensional of the first marker. The work management apparatus according to claim 2, wherein the position is calculated.
In addition to notifying the operator of the determination result of the work correctness determination unit, the notification unit includes a notification unit that indicates the next work position based on the movement order of the three-dimensional position of the tool of the work procedure storage unit. The work management apparatus according to claim 3.
In addition to notifying the operator of the determination result of the work correctness determination unit, the notification unit includes a notification unit that indicates the next work position based on the movement order of the three-dimensional position of the tool of the work procedure storage unit. The work management apparatus according to claim 4.
A tool state detection unit that acquires information indicating that a torque value at the time of fastening completion or a predetermined torque has been reached, which is output from the tool for fastening a part to the workpiece;
Together with the date and time information, the three-dimensional position of the first marker calculated by the marker position calculation unit and the information indicating that the torque value at the completion of fastening or the prescribed torque obtained by the tool state detection unit has been reached have been reached. The work management apparatus according to claim 1, further comprising a work history storage unit that stores the work history.
A tool state detection unit that acquires information indicating that a torque value at the time of fastening completion or a predetermined torque has been reached, which is output from the tool for fastening a part to the workpiece;
Together with the date and time information, the three-dimensional position of the first marker calculated by the marker position calculation unit and the information indicating that the torque value at the completion of fastening or the prescribed torque obtained by the tool state detection unit has been reached have been reached. The work management apparatus according to claim 2, further comprising a work history storage unit that stores the work history.
The work procedure storage unit has information on torque values set for each three-dimensional position of the tool when performing the work,
The work correctness determination unit, when determining the correctness of the work, collates the torque value at the completion of fastening acquired by the tool state detection unit with the torque value of the work procedure storage unit. The work management apparatus according to claim 10.
The work procedure storage unit has information on torque values set for each three-dimensional position of the tool when performing the work,
The work correctness determination unit, when determining the correctness of the work, collates the torque value at the completion of fastening acquired by the tool state detection unit with the torque value of the work procedure storage unit. The work management apparatus according to claim 11.
The work management apparatus according to claim 10, wherein the work history storage unit stores image data obtained by imaging the work area.
The work management apparatus according to claim 11, wherein the work history storage unit stores image data obtained by imaging the work area.
15. The work management apparatus according to claim 14, wherein the work history storage unit stores the image data together with a search key associated with a marker included in the image.
16. The work management apparatus according to claim 15, wherein the work history storage unit stores the image data together with a search key associated with a marker included in the image.
At least markers attached to the tool,
One or more imaging devices for imaging a work area in which work is performed on a workpiece using the tool;
A work management system comprising: the work management apparatus according to claim 1, wherein image data captured by the imaging device is acquired to determine whether the work is correct or incorrect.
At least markers attached to the tool,
One or more imaging devices for imaging a work area in which work is performed on a workpiece using the tool;
A work management system comprising: the work management apparatus according to claim 2, which acquires image data captured by the imaging device and determines whether the work is correct or incorrect.