JP6923429B2 - Work method for work - Google Patents

Description

The present invention relates to a working method for a work. In particular, the present invention relates to a method of generating a work track for performing work on a work and performing work along this work track.

Conventionally, for example, in an automobile manufacturing factory or the like, in order to automatically perform a predetermined work on a work, it is desired to appropriately generate a work track on the work in advance. For example, when applying an adhesive on a window glass by a robot or the like, it is desirable to place the window glass on a table and take an image of the window glass with a camera to appropriately generate an orbit for applying the adhesive in advance. Is done.

Patent Document 1 is provided with a calibration means for obtaining the internal parameters of the camera from the image of the linear marker captured by the camera in view of the fact that the error due to the influence of the camera parameters becomes large when the object is detected by the camera. Is disclosed.

Japanese Unexamined Patent Publication No. 2006-135621

However, when generating a work trajectory on the work, it is necessary to consider not only the error of the image recognition system by the camera (lens distortion, deviation of the camera installation position, etc.) but also the mechanical error of the device (assembly error, etc.). Therefore, it was difficult to obtain sufficient accuracy of the work track even if the calibration was performed individually for these errors.

The present invention has been made in view of the above points, and an object of the present invention is to provide a work method for a work capable of performing work along a work trajectory on the work on which sufficient accuracy has been obtained. There is.

A working method for a work according to the present invention for achieving the above object is to mount a camera, a table that can move relative to the camera, a dot device that can move relative to the table, and a working unit on the table. A calibration work to be placed, a main work, a moving unit for moving the table, a control unit for controlling the movement of the table by the moving unit, and a storage unit for storing real space coordinate values and image coordinate values. A trajectory information storage device that stores trajectory information of work positions suitable for each of the plurality of types of the work, a work discrimination device that stores images of the plurality of types of the work in advance, and a temporary work trajectory generator. The proofreading work placed on the table is moved together with the table, and the proofreading point is struck on the proofreading work by the scoring device. A calibration information acquisition step of storing the spatial coordinate values and the coordinate values on the image of each of the calibration dots captured by the camera in the storage unit, and the present work on the table instead of the calibration work. The exchange step of placing the work and the work discriminating device receive the data of the actual image of the work on the table captured by the camera, and from the actual image, the plurality of types of the book stored are stored. A process of determining the type of the work placed on the table among the works, and a step of determining the type of the work.
The temporary work track generator extracts track information suitable for the discriminated type of the work from the track information storage device , and generates a temporary work track for the work according to the trajectory information on the image. The actual operation is performed from the generation step, the coordinate values on the image stored in the storage unit near the temporary work trajectory, and the real space coordinate values stored in the storage unit corresponding to the coordinate values on the image. Until the work unit works on the actual work position, which is the real space coordinate value, and reaches the end of the temporary work track, and the real space coordinate value calculation process, which calculates the real space coordinate value which is the work position. After repeating the calculation of the work position and the work of the work unit or calculating the work position until the end of the temporary work track is reached, the work unit works on the work position. It is characterized by having a work process to be performed.

Due to this specific matter, each coordinate system is acquired based on the optical error of the camera, the error due to the relative position between the camera and the table, and the mechanical error of the moving part. It is possible to perform reduced calibration at once. Further, since the entire coordinates are not calibrated at once but only the points near the temporary work trajectory are sequentially calibrated, the local distortion of the camera lens can be eliminated and the reliability of calibration can be improved.

In the present invention, the actual work position, which is the real space coordinate value calculated from the coordinate value on the image in the vicinity of the temporary work trajectory for the work and the real space coordinate value corresponding to the coordinate value on the image, is set to the actual work position. On the other hand, the work department is trying to do the work. Therefore, calibration can be performed by collectively reducing the optical error of the camera, the error due to the relative position between the camera and the table, and the mechanical error of the moving portion. In addition, local distortion of the camera lens can be eliminated, and the certainty of calibration can be improved.

It is a figure which shows the schematic structure of the adhesive coating apparatus which concerns on embodiment. It is a block diagram which shows the structure of the orbit generation system. It is a figure which shows the schematic structure of the dot making apparatus. It is a flowchart which shows the procedure of the trajectory generation operation and the adhesive application operation. It is a top view which shows the state which the dot is attached to the flat plate for a marker. It is a figure corresponding to FIG. 5 which shows the proximity point of the pixel coordinates on a plate for a marker.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present embodiment describes a case where the present invention is applied as a method for applying an adhesive to a wind glass applied to a vehicle.

-Outline configuration of adhesive coating device-
FIG. 1 is a diagram showing a schematic configuration of an adhesive coating device 1 according to the present embodiment. Further, FIG. 2 is a block diagram showing the configuration of the trajectory generation system 2 provided in the adhesive coating device 1.

As shown in these figures, the adhesive coating device 1 includes a track generation system 2 and a working unit 3. Further, the trajectory generation system 2 includes a trajectory generation device system 4 and an image recognition system 5.

The trajectory generating device system 4 includes a table 41, a moving mechanism (moving unit) 42, a control device (control unit) 43, and a spotting device 44. The image recognition system 5 includes a camera 51, an image calibration device (storage unit) 52, a trajectory information storage device 53, a work discrimination device 54, a target trajectory generation device (temporary work trajectory generation unit) 55, and a command conversion for the trajectory generation device. It is configured to include the device 56. Each will be described below.

On the table 41, a work to be described later (one of a marker flat plate W1 as a calibration work (see FIG. 3) and a window glass W2 as the main work (see FIG. 1)) is placed, and the work W1, W2 is gripped by means such as adsorption.

The moving mechanism 42 rotates the table 41 around a vertical axis or slides it in the horizontal direction, and moves the table 41 using a moving motor or the like (not shown) as a power source. .. That is, the table 41 can be moved relative to the camera 51 by the moving mechanism 42. A rack and pinion mechanism 41a is applied as a mechanism for sliding the table 41 in the horizontal direction. The configuration for moving the table 41 and the direction for moving the table 41 are not limited to those described above, and can be set arbitrarily. Further, the moving mechanism 42 also moves or rotates the adhesive coating nozzle 31 constituting the working portion 3 up and down. The adhesive application nozzle 31 may be moved up and down or rotated manually.

The control device 43 receives a command signal from the orbit generating device command conversion device 56, outputs a movement control signal to the moving mechanism 42 in accordance with this command signal, thereby moving the table 41 and the adhesive coating nozzle 31. Control. The control device 43 also controls the movement of the dot device 44.

The doting device 44 spots a marker on the marker flat plate W1 held by the table 41. The marker (dotting point) on the marker flat plate W1 is performed for calibration between the camera 51 and the adhesive coating device 1. The spotting device 44 is moved by the moving mechanism 42 to adjust the position where the marker is spotted. That is, the dot device 44 can move relative to the table 41.

FIG. 3 is a diagram showing a schematic configuration of the dot device 44. As shown in FIG. 3, the spotting device 44 is arranged above the table 41, and moves up and down while holding the marker pen 44a for drawing a marker on the marker flat plate W1 and the marker pen 44a. It has an elevating mechanism 44b that can move up and down). As the configuration of the elevating mechanism 44b, a well-known mechanism using a motor, air, or the like as a power source is applied. The spotting device 44 is a device for spotting a marker on the marker flat plate W1 when the marker pen 44a is attached, but the marker pen 44a is removed and an adhesive application nozzle 31 is provided. In the state, the working unit 3 is configured. As described above, in the present embodiment, the functional portion as the elevating mechanism 44b and the functional portion as the working portion 3 are partially shared.

The camera 51 is arranged above the table 41, and images the marker flat plate W1 and the window glass W2 held by the table 41 from above.

The image calibration device 52 acquires and stores as calibration information the corresponding positional relationship between the image (camera image) captured by the camera 51 and the marker drawn on the marker flat plate W1. That is, the image proofreading device 52 stores the real space coordinate value representing the position of the actual marker and the coordinate value on the image representing the position of the marker in the camera image. Further, as the image proofreading device 52, a plurality of the cameras 51 may be prepared to obtain and process a delicate image.

The orbit information storage device 53 stores target orbit information such as teaching and CAD with respect to the position and posture of the window glass W2 in a state where the window glass W2 is gripped on the table 41.

The work discriminating device 54 stores images of a plurality of types of works (wind glass W2) to which the adhesive is applied in the adhesive coating device 1, and also receives image data from the camera 51, and the camera 51 receives the image data. From the captured actual image of the window glass W2, the type of the window glass W2 held by the table 41 is determined.

The target orbit generation device 55 receives the work type information from the work discriminating device 54, adapts the orbit information from the orbit information storage device 53 to the type of the window glass W2 (to the orbit information storage device 53). The orbit information conforming to the window glass W2 is extracted from the stored orbit information), and the target orbit is generated on the camera image. This generated orbit information becomes tentative target orbit information.

The command conversion device 56 for the trajectory generator receives the provisional target trajectory information from the target trajectory generator 55, and controls the information based on the calibration information stored in the image calibration device 52. It is converted into a command signal to the device 43, and this command signal is transmitted to the control device 43.

-Orbit generation operation, adhesive application operation-
Next, the track generation operation by the track generation system 2 and the adhesive application operation by the working unit 3 will be described with reference to the flowchart of FIG.

As the trajectory generation operation, a calibration operation, a window glass discrimination operation, and a target trajectory generation operation are performed in order.

(Calibration operation)
First, the calibration operation will be described.

In this calibration operation, as shown in FIG. 3, the marker flat plate W1 is placed on the table 41, and the marker flat plate W1 is gripped on the table 41. The marker flat plate W1 preferably has the same height dimension and size as the window glass W2.

Next, the marker pen 44a is attached to the elevating mechanism 44b of the spotting device 44, and the marker pen 44a points (draws) a marker on the marker flat plate W1 by the elevating operation of the elevating mechanism 44b.

Then, the marker pen 44a is used to make dots on the marker flat plate W1 at arbitrary intervals (step ST1; “The calibration work placed on the table is moved together with the table and is placed on the calibration work by the dot device. "Doting process for making proofreading dots"). This dot is for calibration, and in the present embodiment, the marker is dotd fully automatically by moving the table 41 according to the movement control signal from the control device 43 and operating the dot device 44. There is.

Next, the marker flat plate W1 on which the marker is hit is imaged by the camera 51 (step ST2), and each hit point on the marker flat plate W1 is extracted as a feature point. FIG. 5 is a plan view showing a state in which each hitting point is attached on the marker flat plate W1. As a method for extracting this feature point, general image processing can be used. In this case, since it is not possible to determine which feature point is the hitting point drawn by the trajectory generator system 4, the center point O1 and the start point O2 on the marker flat plate W1 are specified.

Then, based on the coordinates of the center point O1 and the start point O2, the coordinates of the marker (striking point) are searched as feature points in a predetermined order while masking a predetermined range, and the image captured by the camera 51 ( It is stored in association with the coordinates of the camera image) (step ST3). As shown by the broken arrow in FIG. 5, the order of searching for these feature points is the hitting points on the marker flat plate W1 on the side close to the center point O1 and at the same distance from the center point O1. A plurality of hit points are searched along a predetermined rotation direction (clockwise direction or counterclockwise direction) starting from the start point O2 and associated with the coordinates of the camera image, and then sequentially, a flat plate for a marker. A similar search is performed on a hit point located far from the center point O1 on W1 and associated with the coordinates of the camera image.

By the above operation, the direct correspondence table between the coordinate data [xi, yi] group of the hitting point drawn by the trajectory generator system 4 and the pixel coordinate data [Xi, Yi] group of the camera image becomes the calibration information. It will be stored in the image calibration device 52 (“the real space coordinate value of each calibration dot and the coordinate value on the image of each calibration dot captured by the camera” in the present invention is stored in the storage unit. Calibration information acquisition process ").

The above is the calibration operation.

After the calibration operation is completed, the marker flat plate W1 is removed from the table 41, and the marker pen 44a is removed from the elevating mechanism 44b.

(Wind glass discrimination operation)
Next, the window glass discrimination operation will be described.

In this window glass discrimination operation, as shown in FIG. 1, the window glass W2 is placed on the table 41, and the window glass W2 is gripped on the table 41 (step ST4; instead of the “calibration work” referred to in the present invention. Replacement process for placing this work on the table ").

After that, the window glass W2 gripped on the table 41 is imaged by the camera 51 to acquire image data (step ST5), and then any of the plurality of types of window glass W2 stored in advance is used. (Type of window glass W2) is determined (step ST6). Then, the position and orientation of the window glass W2 on the table 41 are calculated based on the camera image. Any pattern matching method can be used for the method for discriminating the window glass W2 and the calculation of the position and orientation of the window glass W2.

Then, from the various trajectory information stored in the work discriminating device 54, the trajectory information corresponding to the discriminated type of window glass W2 is extracted, and a temporary target trajectory to which the adhesive is applied according to the trajectory information is captured by the camera. It is generated on the pixel coordinates of the image (step ST7; "temporary work trajectory generation step in which the temporary work trajectory generation unit generates a temporary work trajectory for the present work preset on the image" in the present invention).

The above is the window glass discrimination operation.

(Target orbit generation operation)
Next, the target trajectory generation operation will be described.

In this target trajectory generation operation, a proximity point is extracted from the calibration information obtained in the calibration operation with respect to the command point on the pixel coordinates of the camera image (step ST8). For example, find four points surrounding the command point as proximity points. The proximity points may be three or more points.

FIG. 6 is a diagram corresponding to FIG. 5 showing the proximity points of the pixel coordinates on the marker flat plate W1. In FIG. 6, when the command point exists in the area A1, the proximity points are four points a1, b1, c1, and d1. Further, it indicates that the proximity points when the command points exist in the area A2 are four points a2, b2, c2, and d2.

Then, the number of the proximity points is n, the coordinate data of the hit points drawn by the trajectory generator system 4 as described above is [xi, yi], and the pixel coordinate data of the camera image is [Xi, Yi] (i). = 1 to n), the projective transformation matrix H is obtained by the following equations (1) to (3) (step ST9).

The equation (1) is a matrix created from the coordinate values of the hit points drawn by the trajectory generator system 4, and the equation (2) is a matrix created from the pixel coordinate values of the camera image.

Using the projection conversion matrix H obtained in this way, the coordinate data [x, y] of the trajectory generator system 4 is calculated from the pixel coordinate data [X, Y] of the command point by the following equation (4) ( Step ST10; The actual work is performed from the "coordinate values on the image stored in the storage unit near the temporary work trajectory and the real space coordinate values stored in the storage unit corresponding to the coordinate values on the image" in the present invention. Real space coordinate value calculation process for calculating the real space coordinate value which is the position). This coordinate data [x, y] becomes the coordinates (orbit generation coordinate data) for determining the final target trajectory.

The coordinate data [x, y] calculated in this way is set as the passing position of the adhesive coating nozzle 31, that is, the adhesive coating trajectory, and the adhesive is applied from the adhesive coating nozzle 31 toward the window glass W2. The adhesive coating work to be applied is performed (step ST11). That is, each proximity point is weighted using the above calculation formula, the coordinates in the region surrounded by each proximity point (coordinates on the target trajectory) are determined, and the trajectory in this region (target trajectory). ), Adhesive application work will be performed.

Such an operation is repeated until the adhesive coating nozzle 31 reaches the end point of the adhesive coating track (until a YES determination is made in step ST12). That is, if the adhesive application nozzle 31 has not reached the end point of the adhesive application trajectory and a NO determination is made in step ST12, the process returns to step ST8, and the new command point on the pixel coordinates of the camera image is contacted. Proximity points are extracted from the calibration information obtained in the calibration operation. For example, as described above with reference to FIG. 6, proximity points are extracted with respect to command points in different regions, a new projective transformation matrix H is calculated, and an orbital generator system using this projective transformation matrix H. The coordinate data [x, y] of 4 is calculated, and the adhesive application work is performed according to the coordinate data [x, y]. A work process in which the calculation of the work position and the work of the work part are repeated until the end of the temporary work track is reached. ")

Then, when the adhesive application nozzle 31 reaches the end point of the adhesive application trajectory and a YES determination is made in step ST12, the process proceeds to step ST13, and it is determined whether or not the next work (wind glass W2) exists. .. Then, if the next window glass W2 exists and a YES determination is made in step ST13, the process returns to step ST4, the next window glass W2 is placed on the table 41, and the window glass W2 is placed on the table 41. The operation of steps ST5 to ST12 described above is repeated by grasping the glass.

Then, when the next window glass W2 does not exist, NO is determined in step ST13, and this routine ends.

As described above, in the present embodiment, the optical error of the camera 51, the error due to the relative position of the camera 51 and the table 41 (the flat plate W1 for the marker and the window glass W2), and the machine of the moving mechanism 42 Each coordinate system is acquired based on the error, and calibration can be performed with these errors collectively reduced. In addition, instead of calibrating the entire coordinates at once, only points near the temporary target trajectory (temporary work trajectory) are sequentially calibrated, so local distortion of the camera lens can be eliminated and calibration can be performed. Certainty can be improved.

(Modification example)
Next, a modified example will be described. In the above embodiment, the work unit works with respect to the actual work position which is the actual space coordinate value, and the calculation of the work position and the work of the work unit are repeated until the end of the temporary work track is reached. rice field. In this modification, the work unit is made to work with respect to the work position after the work position is calculated until the end of the temporary work track is reached.

As a specific control procedure, in the flowchart shown in FIG. 4, the trajectory generation coordinate data calculated in step ST10 is stored without performing the adhesive application work yet, and between steps ST12 and ST13. , The adhesive application work is performed based on the stored trajectory generation coordinate data.

-Other embodiments-
The present invention is not limited to the above-described embodiment, and all modifications and applications included in the claims and the range equivalent to the claims can be applied.

For example, in the above-described embodiment, the case where the present invention is applied as a method for applying an adhesive to the window glass W2 has been described. The present invention is not limited to this, and can be applied to a method of generating a work track for other workpieces and performing work along the work track.

The present invention is applicable to a method of generating a working track for applying an adhesive to window glass and applying the adhesive along the working track.

1 Adhesive coating device 2 Track generation system 3 Working unit 31 Adhesive coating nozzle 4 Track generating device system 41 Table 42 Moving mechanism (moving part)
43 Control device (control unit)
44 Doting device 5 Image recognition system 51 Camera 52 Image proofing device (storage unit)
53 Track information storage device 54 Work discrimination device 55 Target track generator (temporary work track generator)
56 Command converter for orbit generator W1 Flat plate for marker (calibration work)
W2 wind glass (this work)

Claims (1)

A camera, a table that can move relative to the camera, a dot device and a working unit that can move relative to the table, a calibration work and this work that are placed on the table, and a moving unit that moves the table. , A control unit that controls the movement of the table by the moving unit, a storage unit that stores the real space coordinate value and the coordinate value on the image, and a storage unit that stores the trajectory information of the work position suitable for each of the plurality of types of the present work. A work trajectory information storage device, a work discrimination device that stores a plurality of types of images of the present work in advance, and a temporary work track generation unit are provided.
A doting step of spotting a calibration dot on the calibration work by the doting device while moving the calibration work placed on the table together with the table.
A calibration information acquisition step of storing the real-space coordinate values of each of the calibration dots and the coordinate values on the image of each of the calibration dots captured by the camera in the storage unit.
A replacement process in which the work is placed on the table in place of the calibration work, and
The work discriminating device receives the data of the actual image of the work on the table captured by the camera, and from the actual image, the work is placed on the table among the plurality of types of the memorized works. The process of determining the type of the mounted work and
The temporary work track generator extracts track information suitable for the discriminated type of the work from the track information storage device , and generates a temporary work track for the work according to the track information on the image. Generation process and
It is an actual work position from the coordinate values on the image stored in the storage unit near the temporary work trajectory and the real space coordinate values stored in the storage unit corresponding to the coordinate values on the image. The real space coordinate value calculation process for calculating the real space coordinate value, and
The calculation of the work position and the work of the work unit are repeated or the work of the work unit is repeated until the work unit works on the actual work position which is the real space coordinate value and reaches the end of the temporary work track. A work method for a work, which comprises a work process in which the work unit works on the work position after calculating the work position until the end of the temporary work track is reached.

Images