JP6025400B2 - Work position detection device and work position detection method - Google Patents

Description

  The present invention relates to a work position detection apparatus and a work position detection method for detecting the position of a work from image data picked up by an image pickup apparatus.

  2. Description of the Related Art Conventionally, in a factory that produces office equipment and the like, a plurality of works (parts) are sent from a component manufacturer to a production line in a state where they are put in a distribution box called a return box. The work put in the return box is usually arranged in an area partitioned for each work. However, since each work is not fixed to the return box, each work moves within the area where it is placed, and the position and orientation are not uniquely determined. For this reason, the position and orientation of each workpiece is recognized from the image data captured (captured) by the imaging apparatus, and the workpiece is sequentially taken out from the return box by using the result.

  In order to recognize the position of each workpiece in the return box with high accuracy from the image data captured by the imaging device, it is necessary to accurately capture the contour information of each workpiece. However, it is difficult to uniformly irradiate a plurality of works in the return box. In the image data, if the part is too bright or too dark, the outline information of the workpiece in that part cannot be obtained accurately. Therefore, there is a possibility that the position of the workpiece (including the posture of the workpiece based on the position of the workpiece) cannot be accurately recognized.

  In order to solve such a problem, there has been proposed a photographing method for obtaining good image data in which the difference in brightness is eliminated (see Patent Document 1). In the imaging method of this patent document 1, it is possible to obtain good image data even under illumination conditions such as backlight or over-order light by image processing with exposure and weighting focusing on the subject and the background. It is possible.

Japanese Patent No. 4600684

  However, if it is attempted to obtain image data for detecting the positions of all workpieces including a part that is too bright or a part that is too dark by the photographing method described in Patent Document 1, it is necessary to photograph the entire region three times or more. There is. That is, the entire region needs to be photographed first under normal exposure conditions, then under subject extraction exposure conditions, and finally under background extraction exposure conditions. For this reason, in consideration of the shooting time, the time for synthesizing three or more images, etc., there is a problem that it takes a long time to detect the position of the workpiece.

  Further, in the photographing method described in Patent Document 1, since the image composition processing is performed so that the contrast of the image data as a whole becomes good, there is a case where it is not improved locally. If the contrast is not improved locally, there is a problem that the position of the workpiece in that portion cannot be accurately detected.

  Therefore, an object of the present invention is to provide a workpiece position detection apparatus and a workpiece position detection method that can accurately detect the position of a workpiece in a short time.

  The workpiece position detection apparatus according to the present invention has an exposure adjustment function, and captures a plurality of workpieces including a reference workpiece to acquire image data, a storage unit that stores the image data, and the plurality of the plurality of workpieces. One workpiece is selected from the workpieces, a selection region corresponding to the selected workpiece is set, a luminance histogram relating to the selection region is generated from the image data, and the luminance histogram relating to the selection region is added to the image data. Calculating an exposure condition in the selected area so as to approach a luminance histogram relating to a reference area corresponding to the reference workpiece included, and causing the imaging device to re-image the selected area under the exposure condition to obtain partial image data; A control unit that detects the position of the selected workpiece based on the partial image data, That.

  In the workpiece position detection method according to the present invention, the control unit instructs an imaging apparatus having an exposure adjustment function to capture a plurality of workpieces including a reference workpiece and acquire image data, and the control A storage step of storing the image data in a storage unit, a selection step in which the control unit selects one workpiece from the plurality of workpieces, and the control unit corresponds to the selected workpiece. A selection area setting step for setting a selection area, and the control unit generates a luminance histogram relating to the selection area from the image data, and the luminance histogram relating to the selection area corresponds to the reference work included in the image data. An exposure condition calculation step of calculating an exposure condition in the selected area so as to approach a luminance histogram relating to a reference area; and A partial image data acquisition step for instructing the apparatus to re-image the selected area under the exposure conditions to acquire partial image data, and the control unit detects the position of the selected workpiece based on the partial image data And a position detecting step.

  According to the present invention, for example, the position of a workpiece in a portion that is too bright or too dark can be detected only by acquiring image data of a plurality of workpieces and partial image data of a selected region, that is, by only two imaging operations. For this reason, it is possible to shorten the imaging time, the time for image synthesis, and the like, and the position of the workpiece can be detected in a short time. In addition, partial image data of the selected area corresponding to the selected workpiece is acquired under exposure conditions that approach the luminance histogram to the reference area, so that the contrast can be improved locally and the position of the workpiece in that area can be accurately detected. it can. As a result, the position detection of the workpiece can be speeded up, and robust against changes in the external environment such as lighting conditions.

1 is a schematic diagram showing a robot system according to the present embodiment. The block diagram which shows a control computer. The flowchart which shows the preset setting control of a reference | standard area | region. The flowchart which shows workpiece | work position detection control. The figure which shows the image data of the whole area. It is a figure explaining a binarization process and a labeling process, (a) is a figure before a process, (b) is a figure after a process. It is a figure explaining each area | region setting, (a) is a figure which shows the image data of the whole area, (b) is a figure which shows the selection area | region. The figure which shows re-imaging data. FIG. 5A is a diagram showing a luminance histogram of each region, where FIG. 5A is a luminance histogram of a reference region, FIG. 5B is a luminance histogram of a selected region in the entire image data, and FIG.

  Embodiments according to the present invention will be described below with reference to FIGS. First, the robot system 50 according to the present embodiment will be described with reference to FIGS.

  As shown in FIG. 1, a robot system 50 according to the present invention is a robot that grips and takes out a plurality of workpieces 8 that are partitioned and housed in a return box 9 and supplies them to the subsequent process of the production line. An arm 7 is provided. An illuminating device 6 that illuminates the return box 9 is disposed above the robot arm 7. The illumination device 6 is required to be installed at a distance from the return box 9 because the robot arm 7 needs to take out the workpiece 8 from the return box 9. For this reason, it is difficult to create uniform illumination conditions throughout the returnable box 9.

  A camera 5 serving as an imaging device of the workpiece position detection device 1 is disposed above the illumination device 6 or in parallel with the illumination device 6 and at a position where the illumination device 6 does not interfere when the passing box 9 is photographed. is set up. At that time, the illumination device 6 is installed so as not to enter the field of view of the camera 5. The workpiece position detection apparatus 1 roughly includes a camera 5 having an exposure adjustment function and a control computer 10 having a control unit 102, a storage unit 103, a communication unit 116, and the like, which will be described in detail later, in a state where they are connected to each other. ing. The control computer 10 is connected to the robot controller 20 and transmits the position data of the workpiece 8 detected from the image data of the camera 5 as described in detail later. The robot control unit 20 drives and controls the robot arm 7 based on the received position data of the work 8 and grips and takes out the work 8.

  As shown in FIG. 2, the control computer 10 includes a control unit 102, a storage unit 103, and a communication unit 116, and is configured by connecting the camera 5. Specifically, the control computer 10 includes the image processing apparatus 102b and the sound processing apparatus 102c, mainly the CPU 102a, and constitutes the control unit 102. In addition to the image processing device 102b and the sound processing device 102c, a ROM 103a and a RAM 103b are connected to the CPU 102a via a bus 111. The ROM 103a stores programs necessary for basic control of the computer, as well as various programs for executing the steps of workpiece position detection control, which will be described later, and data such as shape data of the workpiece 8. The various programs include, for example, a program that calculates and generates a histogram, a program that performs binarization processing, a program that performs labeling processing on a binarized image, a program that sets a selection area, a program that calculates exposure conditions, and the like. Furthermore, a program for re-imaging and acquiring partial image data of the selected area, a program for detecting the position of the workpiece, and the like are also included. In the present embodiment, these various programs are collectively referred to as a work position detection program. A work area for the CPU 102a is secured in the RAM 103b, and image data captured by the camera 5 can be temporarily stored.

  The image processing apparatus 102b controls the display 107 in accordance with a drawing instruction from the CPU 102a to display a predetermined image on the screen. The sound processing device 102c generates a sound signal according to a sound generation instruction from the CPU 102a and outputs the sound signal to the speaker 109. A keyboard 106a and a mouse 106b as an input device 106 are connected to the CPU 102a via an interface 106c connected to the bus 111 so that designation information necessary for workpiece position detection or other instructions can be input. Yes.

  In addition, the camera 5 is connected to the bus 111 and a recording media reading device 117 is connected to the bus 111. For this reason, the recording medium reading device 117 can read the recording medium 110 on which a program or the like is recorded, and store it in, for example, the ROM 103a or the RAM 103b. The storage unit 103 described above may include an external storage device in addition to the ROM 103a and the RAM 103b which are main storage devices.

  And the communication part 116 is connected to the bus | bath 111, and the communication part 116 can transmit the position data of the workpiece | work 8 detected to the robot control part 20 (refer FIG. 1) mentioned above. Note that the communication unit 116 may also be connected to a robot control unit connected to another robot via a network or the like. In this case, the control computer 10 can be a host computer that sends operation commands to a plurality of robots, and each robot control unit can be a lower computer that operates based on the commands of the host computer. Further, the communication unit 116 may be connected to the Internet or the like to download a program.

  In the present embodiment, the camera 5 and the control computer 10 are separately described. However, the work position detection apparatus 1 may be provided with a computer that is integrated with the camera 5.

  Next, workpiece position detection control by the workpiece position detection apparatus 1 configured as described above will be described. First, preset control as advance preparation (sub control) of workpiece position detection control will be described with reference to FIG.

  As shown in FIG. 3, in the preset control, first, a rectangular area is drawn using the mouse 106b on the image of the entire return box 9 displayed on the display 107 by the image processing apparatus 102b. The rectangular area is drawn so as to cover the entire area of one work 8 in the return box 9 (S21). This is used for registering the contour information of the workpiece 8 in the region as a reference workpiece as a sample when searching for another workpiece 8. An example of setting the master pattern area is shown in FIG. In FIG. 7A, the central portion of the return box 9 is set as the master pattern area AreaMP. By setting the master pattern area AreaMP at the center of the passing box 9, the work 8 is illuminated more uniformly than the vicinity of the outer periphery of the passing box 9, so that the contour information of the work 8 can be accurately captured. An image captured by the camera 5 displayed on the display 107 is composed of a plurality of pieces of pixel information in the RAM 103b. The pixel information is composed of numerical data from 0 to 255, and the numerical value is higher as white and lower as black. This numerical data is called a luminance value.

  Next, the average value of the luminance in the master pattern area AreaMP when set is calculated by the control unit 102 as the prior luminance average value, and the value is stored in the storage unit 103 (S22).

  Subsequently, a plurality of luminance histograms of the master pattern area AreaMP when the exposure amount of the camera 5 is changed are created, and the distribution change of the luminance histogram when the exposure amount is changed is calculated by the control unit 102. Then, the control unit 102 determines an exposure adjustment amount calculation parameter for obtaining an exposure condition of a selected region, which will be described later, from the distribution change of the luminance histogram when the exposure amount is changed (S23). In other words, based on the exposure adjustment amount calculation parameter, the exposure condition when photographing an excessively bright part or an excessively dark part is calculated, and image data in an appropriate exposure state is obtained by imaging under the exposure condition. Can do.

  Next, workpiece position detection control which is the main control of the present embodiment will be described with reference to FIG. When this control is started, first, since the camera 5 includes the entire box 9 in its field of view, the image data of the entire box 9 is read by incorporating all the image sensor information in the camera 5 (S1) (image Data acquisition process). An example of an image read in the entire image sensor is shown in FIG. Each grid in FIG. 5 represents a pixel, the entire area AreaALL represents the entire pixel, and the pixels are arranged in the x direction and the y direction that form orthogonal coordinates from the image origin O. The coordinates (x, y) can be used as a coordinate system for position information of each workpiece 8 and position control information of the robot arm. The read image sensor information of the entire area (image data of the entire area) is image data obtained by imaging all the workpieces 8 (a plurality of workpieces) including the reference workpiece of the master pattern area described above, and is stored in, for example, the RAM 103b of the storage unit 103. It is memorized (memory process). In addition, the black lump which has entered each grid | lattice in the return box 9 in FIG. 5 represents the shape which looked at the workpiece | work 8 taken out by the robot arm 7 in FIG. The shading of the workpiece 8 in FIG. 5 represents how the light is hit, and shows a state in which it is uniform and bright near the center, and becomes uneven and dark as it goes to the periphery.

  Next, the control unit 102 determines whether or not the return box 9 has just been replaced (S2). If it is not immediately after replacement (No in S2), operations such as updating of a master pattern, which will be described later, are unnecessary, and the process proceeds to step S6. If it is immediately after replacement of the returnable box 9 (Yes in S2), the process proceeds to the next step S3.

  In step S3, the workpiece 8 in the master pattern area AreaMP defined by the preset control (see FIG. 3) is used as a reference workpiece as a sample for detecting other workpieces, and its contour information (reference workpiece shape data). ) Is extracted by the control unit 102. The extracted information is stored in the storage unit 103. The master pattern update process in step S3 is performed only once immediately after the return box 9 is replaced.

  Next, in the image data of the entire area acquired in step S1, the control unit 102 calculates a luminance histogram in the master pattern area AreaMP defined by the preset control (see FIG. 3) (S4).

  Next, the prior average brightness value stored at the time of setting by the preset control (see FIG. 3) is compared with the average brightness value in the master pattern area AreaMP obtained from the current image data, and the difference between the average brightness values is compared. Is determined to be greater than or equal to a specified value (S5). Here, the following will be described assuming that the specified value is 10 as an example. Note that the value of the specified value may be changed according to a ratio, a knitting difference, or the like.

  In step S5, when the difference between the previous average brightness value and the current average brightness value is 10 or more (Yes in S5), the process proceeds to step S16. In step S16, the exposure amount of the camera 5 is adjusted so as to approach the pre-brightness average value stored in the pre-set control (less than 10), and the image sensor information (image data) of the entire area is obtained again. As a result, it is possible to cope with a decrease in luminance due to the passage of time of the lighting device 2, and the robustness is improved. Further, if the master pattern is updated every time the box 9 is replaced in step S3, it is possible to reduce the influence of the lot difference of the work 8 or to cope with the change of the shape of the work 8.

  If it is determined in step S5 that the difference between the previous average brightness value and the current average brightness value is less than 10, the process proceeds to step S6. Here, using the contour information of the master pattern registered in step S3, the control unit 102 starts calculating the presence / absence of the workpiece 8 in the image sensor information of the entire area. First, in step S6, the entire area of the acquired image sensor information (image data) is binarized. In addition, by making the color of the return box 9 easy to generate a contrast difference with the color of the work 8, it is possible to determine the presence or absence of the work 8 only by a simple image processing algorithm such as binarization. .

FIG. 6 shows image changes before and after binarization. In this embodiment, since the work 8 is a member that is close to black, the contrast difference is increased by making the returnable box 9 side a member that is close to white. Therefore, by the binarization process using the intermediate value as a threshold value, FIG. 6A, which is an image before binarization, changes the workpiece 8 as shown in FIG. It can be extracted as a black mass. Next, a labeling process is performed on the binarized image in step S6 to assign the same number to pixels with continuous black portions (S7). A lump with the same number is called a label. The work 8 labeled “8 ₁ ” to “8 ₂₅ ” in FIG. 6B is represented.

  Next, the coordinates of the barycentric position viewed from the image origin O in FIG. 5 are calculated for each block of each work 8 labeled in step S7 (S8). The priority order of the labels is determined based on the calculated coordinates (S9). Here, as priority, it is assumed that extraction is performed preferentially as the distance from the image origin O in FIG. 5 increases. Accordingly, in the first take-out operation, the work 8 at the lower right of the return box 9 shown in FIG. 7A is selected (selection process).

  Subsequently, as shown in FIG. 7B, the control unit 102 calculates a circumscribed rectangular area AreaCir of the label selected as the next extraction target, and sets a rectangular area obtained by expanding the circumscribed rectangular area AreaSEL as the selected area AreaSEL. (S10). Although the specified amount is 20% here, it may be changed depending on the size of the work 8 and the size of the return box 9. Specifically, if the circumscribed rectangular area AreaCir circumscribing the workpiece 8 of the binarized image has a length Lx in the x direction and a length Ly in the y direction, the selected area AreaSEL is in the x direction. The region has a length Lx × 1.2 and a length Ly × 1.2 in the y direction. As described above, in step S10, the selection area AreaSEL for the selected workpiece 8 is set (selection area setting step).

  Next, the control unit 102 calculates a luminance histogram from the distribution of luminance values of the pixels in the set selection area AreaSEL (S11). Subsequently, the most frequent gray value of each luminance histogram calculated in the master pattern area (reference area) AreaMP and the selected area AreaSEL is obtained. The exposure adjustment amount suitable for the imaging of the selected workpiece 8 is calculated by multiplying the difference between the obtained most frequent gray values by the exposure adjustment amount calculation parameter determined by the preset control (see FIG. 3). (S12). That is, the exposure condition for re-imaging the selected area AreaSEL is calculated so that the brightness histogram related to the selected area AreaSEL approaches the brightness histogram related to the master pattern area AreaMP (exposure condition calculating step).

  That is, in the master pattern area AreaMP of FIG. 9A, the luminance distribution is not biased at both ends, and an image with high contrast and high resolution is obtained. On the other hand, in the case of an image with a luminance distribution biased to a low luminance as shown in FIG. 9B, even if an attempt is made to detect the position and orientation of the workpiece 8 as it is, the outline information is lost due to black skipping. Even if such image processing is performed, it is difficult to calculate position and orientation information with high accuracy. On the other hand, in the case of an image with a luminance distribution biased to high luminance, the outline information is partially lost due to overexposure. For example, when the selected area AreaSEL has a luminance distribution biased toward low luminance as shown in FIG. 9B, there are many places where the detection target is black. Therefore, in order to approximate the luminance distribution of FIG. 9A, it is possible to approximate the good luminance distribution as shown in FIG. 9C by increasing the exposure time by the camera 5 based on the calculated exposure condition.

  After calculating the exposure conditions for re-imaging the selected area AreaSEL in this way, only the image sensor information (partial image data) of the camera 5 corresponding to the selected area AreaSEL is re-imaged under the calculated exposure conditions. Obtain (S13) (partial image data obtaining step). An example of the acquired partial image data is shown in FIG. As shown in FIG. 8, by capturing only the image element information of the selected area AreaSEL near the work 8 to be taken out next to the entire area ALLALL that is the field of view of the camera 5, the taking time is greatly increased as compared with the whole taking-in time. It can be shortened.

  Then, using the master pattern information updated in step S3, the position of the workpiece 8 (posture based on the position) extracted from the partial image data acquired in step S13 is detected with high accuracy by the control unit 102 (S14) (position Detection step).

  Finally, by transmitting the position information (and posture information) of the workpiece 8 detected in step S14 from the communication unit 116 to the robot control unit 20 (S15), control is performed so that the workpiece 8 is gripped by the robot arm 7. It can be taken out. After completion of the removal, the workpiece position detection control in FIG. 4 is started again, and the remaining workpiece 8 is taken out to the robot arm 7 through the same flow. Thereafter, the same flow is carried out until the work 8 in the box 9 disappears, and then the return box 9 is replaced.

  As described above, according to the workpiece position detection apparatus 1, for example, the position of the workpiece 8 in an excessively bright portion or an excessively dark portion is obtained by acquiring only the image data of the plurality of workpieces 8 and the partial image data of the selected region, that is, 2 It can be detected with only one imaging. For this reason, it is possible to shorten the imaging time, the time for image synthesis, and the like, and the position of the workpiece 8 can be detected in a short time. In addition, partial image data of the selected region corresponding to the selected workpiece 8 is acquired under the exposure condition that approaches the luminance histogram to the reference region, so that the contrast can be improved locally and the position of the workpiece in that portion can be accurately determined. It can be detected. As a result, the position detection of the workpiece can be speeded up, and robust against changes in the external environment such as lighting conditions.

  Further, since the control unit 102 detects the position of the workpiece 8 selected based on the shape data (contour information) of the master pattern (reference workpiece) and the partial image data, it acquires the contour information only from the partial image data and detects the position. The position can be detected with higher accuracy than in the case of doing so.

  In the present embodiment described above, the workpiece position detection is used for the picking-up operation by the robot arm, but an inspection device that simply ends the workpiece position detection may be used. It is also conceivable to perform processing such as processing on the production line using workpiece position detection.

  In the present embodiment, when one workpiece is selected from a plurality of workpieces, binarization processing is performed to determine the presence or absence of the workpiece, and one workpiece is selected based on the barycentric coordinates. Explained things. However, the present invention is not limited to this, for example, a method for determining the presence / absence of a work by performing other image processing such as gray scale, and a method for determining the presence / absence of a work by acquiring an edge component and roughly acquiring an outline. Such a selection method may be used.

  The camera 5 only needs to have an exposure adjustment function, and may be a monochrome camera or a color camera. Furthermore, the present invention can be applied to a device that uses two or more cameras 5 to perform stereo imaging and measure a three-dimensional position.

DESCRIPTION OF SYMBOLS 1 ... Work position detection apparatus: 5 ... Imaging device (camera): 7 ... Robot arm: 8 ... Work: 50 ... Robot system: 102 ... Control part: 103 ... Memory | storage part: AreaMP ... Reference | standard area | region (master pattern): AreaSEL ... Selection area

Claims (7)

An imaging device having an exposure adjustment function and capturing a plurality of workpieces including a reference workpiece to obtain image data;
A storage unit for storing the image data;
One workpiece is selected from the plurality of workpieces, a selection region corresponding to the selected workpiece is set, a luminance histogram related to the selection region is generated from the image data, and the luminance histogram related to the selection region is An exposure condition in the selected area is calculated so as to approach a luminance histogram related to a reference area corresponding to the reference work included in the image data, and the selected area is re-imaged with the exposure condition by the imaging device to obtain partial image data. A control unit for acquiring and detecting the position of the selected workpiece based on the partial image data,
A workpiece position detecting device characterized by that. The storage unit stores shape data of the reference workpiece,
The control unit detects the position of the selected workpiece based on the shape data of the reference workpiece and the partial image data;
The work position detecting apparatus according to claim 1, wherein The control unit acquires shape data of the reference workpiece from the image of the reference workpiece included in the image data, and stores the shape data of the reference workpiece in the storage unit.
The workpiece position detecting apparatus according to claim 2, wherein The workpiece position detecting device according to any one of claims 1 to 3,
A robot arm controlled to hold the workpiece based on the workpiece position detected by the workpiece position detection device,
A robot system characterized by this. An image data acquisition step in which a control unit instructs an imaging apparatus having an exposure adjustment function to capture a plurality of workpieces including a reference workpiece and acquire image data;
A storage step in which the control unit stores the image data in a storage unit;
A selection step in which the control unit selects one workpiece from the plurality of workpieces;
A selection region setting step in which the control unit sets a selection region corresponding to the selected workpiece;
The control unit generates a luminance histogram related to the selected area from the image data, and the selection is performed such that the luminance histogram related to the selected area approaches a luminance histogram related to a reference area corresponding to the reference workpiece included in the image data. An exposure condition calculation step for calculating an exposure condition in the region;
The control unit instructs the imaging device to acquire a partial image data by reimaging the selected area under the exposure condition; and
A position detecting step for detecting the position of the selected workpiece based on the partial image data;
A workpiece position detection method characterized by the above.   A work position detection program for causing a computer to execute each step of the work position detection method according to claim 5.   A computer-readable recording medium on which the work position detection program according to claim 6 is recorded.

Images