JP2014159988A - Object detector, robot system, and object detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detector, a robot system, and an object detection method that are capable of significantly shortening the time required for detection processing and improving processing efficiency.SOLUTION: A sensor unit 300 comprises: a laser light source 321 for radiating laser slit light; a rotation mirror 322 for moving, in a scanning direction, a projection portion of the laser slit light radiated from the laser light source 321 toward a workpiece; a camera 310 that consecutively takes images of the appearance of the workpiece including the moving projection portion at a predetermined frame rate and outputs a plurality of corresponding image frames when the rotation mirror 322 moves the projection portion of the laser slit light on the workpiece; a distance image generation unit 333 for generating one distance image using a plurality of image frames for each round of scanning; and a synthetic image generation unit 336 for synthesizing a plurality of distance images on the basis of a plurality of rounds of scanning temporally different from one another to generate a synthetic image.

Description

  Embodiments disclosed herein relate to an object detection device, a robot system, and an object detection method.

  Patent Document 1 describes a technique for detecting an object by a light cutting method. In the light cutting method, the projection part of the light irradiated from the light source onto the object is moved in the scanning direction, and the appearance of the object including the moving projection part is continuously imaged at a desired interval to obtain the three-dimensional shape of the object. Is detected.

JP 2009-250844 A

  In the light cutting method as described above, when continuous imaging is performed by moving the projection part of the light onto the object in the scanning direction as described above, it is necessary to perform imaging uniformly over almost the entire area along the scanning direction on the object. Don't be. For this reason, it takes a relatively long time to perform the continuous imaging, and it is difficult to improve the efficiency of the detection process.

  The present invention has been made in view of such a problem, and provides an object detection device, a robot system, and an object detection method capable of greatly shortening detection processing and improving processing efficiency. The purpose is to provide.

  In order to solve the above-described problem, according to one aspect of the present invention, an object detection apparatus that detects an object to be detected, the laser light source that irradiates a slit-shaped laser beam, and the laser light source that is irradiated from the laser light source A scanning unit that moves the projection part of the laser beam onto the object in a predetermined scanning direction, and the projection part that moves when the scanning unit moves the projection part of the laser beam on the object A camera that continuously captures the appearance of the object including a desired interval and outputs a plurality of corresponding image data, and a plurality of the image data for each imaging step in which the camera performs a plurality of consecutive images. A plurality of three-dimensional images generated by the distance image generation unit based on a plurality of imaging steps that are different from each other in time, and a distance image generation unit that generates one three-dimensional shape image of the object using Jo image by synthesizing, and generates a composite image to detect the three-dimensional shape of the object, a composite image generating unit, and having an object detection apparatus is applied.

  In order to solve the above problem, according to another aspect of the present invention, a slit-shaped laser beam is irradiated from a light source, and a projected portion of the irradiated laser beam on an object is set in a predetermined scanning direction. And moving the projection portion of the laser light, continuously imaging the appearance of the object including the moving projection portion at a desired interval, and outputting a plurality of corresponding image data Based on a plurality of the imaging steps that are temporally different from each other, generating one three-dimensional shape image of the object using a plurality of the image data for each imaging step in which a plurality of continuous imaging is performed. A plurality of the three-dimensional shape images generated respectively to detect the three-dimensional shape of the object, and an object detection method is applied.

  According to the present invention, since the detection process can be greatly shortened, the processing efficiency can be improved.

1 is a system configuration diagram illustrating an example of an overall configuration of a robot system according to an embodiment. It is a side view expressed in the state where a part of example of composition of a sensor unit was seen through. It is a top view represented in the state which partially saw through an example of the composition of the sensor unit. It is explanatory drawing for demonstrating an example of a structure of a sensor unit. It is explanatory drawing for demonstrating an example of a structure of a sensor unit. It is explanatory drawing showing an example of the imaging frame output from the camera. It is a block diagram showing an example of a functional composition of a sensor controller. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the 1st scan. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the 1st scan. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the Nth scan. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the Nth scan. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the (N + 1) th scan. It is explanatory drawing for demonstrating the process performed by control of a cooperation control part at the time of the (N + 1) th scan. It is explanatory drawing showing an example of the synthesized image produced | generated by the synthesized image production | generation part. It is explanatory drawing for demonstrating an example of the exclusion process by a synthesized image generation part. It is explanatory drawing for demonstrating an example of the exclusion process by a synthesized image generation part. It is explanatory drawing explaining an example of the synthesized image produced | generated by the synthesized image production | generation part, whenever each scan is complete | finished. It is explanatory drawing explaining an example of the synthesized image produced | generated by the synthesized image production | generation part, whenever each scan is complete | finished. It is explanatory drawing explaining an example of the synthesized image produced | generated by the synthesized image production | generation part, whenever each scan is complete | finished. It is a flowchart showing an example of the control procedure of the object detection method which a sensor controller performs.

  Hereinafter, an embodiment will be described with reference to the drawings.

  First, an example of the entire configuration of the robot system according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 1, the robot system 1 of this embodiment includes a stocker 20 (container), a robot 100, a conveyor 30, a sensor unit 300 (object detection device), and a robot controller 200 (controller). .

  The stocker 20 is a box-shaped container made of, for example, resin or metal, and is disposed on a pedestal 21 disposed in the vicinity of the robot 100. Inside the stocker 20, a plurality of workpieces W (objects) to be detected are randomly placed. Each workpiece W may not be placed in a container such as the stocker 20 or may be placed on an appropriate placement surface such as on the base 21. Moreover, it does not specifically limit as each workpiece | work W and those shapes, Various are considered. At this time, each workpiece | work W and those shapes may mutually correspond or are similar, and may mutually differ. However, in each drawing, the shape of each workpiece W is simplified and shown in an elliptical shape.

  The robot 100 performs a transfer operation (so-called random bin picking) for sequentially holding and transferring a plurality of workpieces W in the stocker 20. The robot 100 is not particularly limited as long as it can perform the transfer operation as described above. For example, a vertical articulated robot or a horizontal articulated robot (scalar robot) can be used. . In this example, a vertical articulated robot is applied as the robot 100, and the robot 100 includes a base 101 fixed to an appropriate fixing surface (for example, a floor portion) and an arm unit rotatably provided on the base 101. 102. The arm part 102 has a plurality of indirects from the base 101 side to the opposite tip side. The arm portion 102 incorporates a plurality of servo motors (not shown) for driving the plurality of indirects, respectively. A holding device 103 capable of holding the workpiece W is provided at the tip of the arm unit 102.

  The holding device 103 is not particularly limited as long as it can hold the workpiece W. For example, a holding device that can hold the workpiece W with a finger member (an aspect of holding), air, electromagnetic force, or the like It is possible to use a suction device or the like that can be driven by this to suck the workpiece W (one mode of holding). In this example, a gripping device is applied as the holding device 103, and the holding device 103 has a pair of finger members 103a capable of gripping the workpiece W. The pair of finger members 103a is driven by an appropriate actuator (not shown) built in the holding device 103, and opens / closes by expanding / reducing the mutual distance. The holding device 103 is not limited to the one having the above-described structure. For example, the holding device 103 has a plurality of finger members, and the plurality of finger members swing to grip the workpiece W. A gripper device or the like can also be used.

  Such a robot 100 sequentially grips a plurality of workpieces W in the stocker 20 one by one with the finger members 103a of the gripping device 103, and transfers and places the workpieces to a preset placement position on the conveyor 30. , Perform the transfer work.

  The conveyor 30 conveys the workpiece W placed at a preset placement position to equipment related to the next step.

  The sensor unit 300 detects a plurality of workpieces W in the stocker 20 by a light cutting method, and each of the plurality of workpieces W has a three-dimensional outer shape (including a three-dimensional position and orientation; the same applies hereinafter). A three-dimensional shape is detected. The sensor unit 300 is supported by an appropriate support member 50 so as to be positioned above the stocker 20. The sensor unit 300 may be attached to an appropriate part of the robot 100 (for example, the distal end side of the arm unit 102). The sensor unit 300 will be described in more detail later.

  The robot controller 200 is composed of, for example, a computer having an arithmetic unit, a storage device, an input device, and the like, and is connected to the robot 100 and the sensor unit 300 so as to communicate with each other. The robot controller 200 operates based on the three-dimensional shape information representing the three-dimensional shape of each of the plurality of workpieces W in the stocker 20 detected by the sensor unit 300 (for example, each indirect of the arm unit 102). And the opening / closing operation of the finger member 103a of the gripping device 103 are controlled. The robot controller 200 will be described in detail later.

  Next, an example of the configuration of the sensor unit 300 will be described with reference to FIGS.

  As shown in FIGS. 2 to 5, the sensor unit 300 includes a laser scanner 320, a camera 310, and a sensor controller 330.

  The laser scanner 320 includes a laser light source 321, a rotating mirror 322 (scanning unit), a motor 323, and an angle detector 324. The laser light source 321 emits slit-shaped laser light (hereinafter referred to as “laser slit light”) L. The rotating mirror 322 receives the laser slit light L emitted from the laser light source 321, reflects it toward the workpiece W and the like, and projects it onto the workpiece W and the like. The motor 323 rotates the rotating mirror 322 and changes the rotation angle of the rotating mirror 322. By rotating the rotating mirror 322 by the motor 323, the projection part of the laser slit light L on the work W etc. in the entire projection area TT composed of a plurality of projection areas to be projected such as the work W is indicated by an arrow A. It can be moved in the direction shown (predetermined scanning direction; hereinafter referred to as “scanning direction A”). The angle detector 324 detects the rotation angle of the rotary mirror 322.

  When the projection part of the laser slit light L on the workpiece W or the like in the entire projection area TT moves in the scanning direction A as described above, the camera 310 in the entire projection area TT including the projection part to move is moved. The external appearance of the workpiece W or the like is continuously imaged at a desired frame rate (interval). Specifically, the camera 310 scans in the scanning direction A after the projected portion of the laser slit light L occurs in the projection region at the end of the entire projection region TT in the opposite scanning direction (the direction opposite to the direction indicated by the arrow A). Are sequentially moved, and continuous imaging is performed until the projection area reaches the projection area at the end in the scanning direction A side of the entire projection area TT (that is, over the entire area of the entire projection area TT). The camera 310 then outputs a plurality of imaging frames F (image data; see FIG. 6 described later) corresponding to the continuous imaging. FIG. 6 shows an example of the imaging frame F output from the camera 310. As shown in FIG. 6, in the imaging frame F, the appearance of the work W etc. in the entire projection area TT and the projection part of the laser slit light L on the work W etc. moving every moment as described above. And behaviors are included.

  The sensor controller 330 is composed of, for example, a computer having an arithmetic unit, a storage device, and the like, and controls the operation of the entire sensor unit 300. The sensor controller 330 detects the plurality of workpieces W in the stocker 20 based on the plurality of imaging frames F output from the camera 310, and detects the three-dimensional shape of each of the plurality of workpieces W. The sensor controller 330 will be described in more detail later.

  Next, an example of a functional configuration of the sensor controller 330 will be described with reference to FIG.

  As shown in FIG. 7, the sensor controller 330 includes a light source control unit 341, a motor control unit 342, an image acquisition unit 331, an image storage unit 332, a distance image generation unit 333, a distance image storage unit 334, A cooperation control unit 335, a detection unit 340, a composite image generation unit 336, and a composite image storage unit 337 are provided.

  The light source control unit 341 controls the laser light source 321 to irradiate the laser slit light L. The motor control unit 342 receives the rotation angle information of the rotary mirror 322 detected by the angle detector 324, controls the motor 323 based on the input rotation angle information, and rotates the rotary mirror 322.

  The image acquisition unit 331 acquires a plurality of imaging frames F output from the camera 310. The plurality of imaging frames F acquired by the image acquisition unit 331 are stored in the image storage unit 332.

  The distance image generation unit 333 uses a plurality of imaging frames F in the scan stored in the image storage unit 332 for each time a single scan (imaging process) in which the camera 310 performs a plurality of continuous imagings, Based on the surveying principle, the distance between the camera 310 and the workpiece W is calculated. Then, the distance image generation unit 333 includes one distance image DP (three-dimensional shape image, which will be described later with reference to FIGS. 9 and 9) in which the calculated distance information is included in the appearance image of the workpiece W or the like in the entire projection area TT. 11 and FIG. 13). In the following, for convenience of explanation, in each scan, the imaging frame F corresponding to the first imaging is referred to as “imaging frame F1,” and the imaging frame F corresponding to the second imaging is referred to as “imaging frame F2.” The imaging frame F corresponding to the imaging is referred to as “imaging frame F3” (hereinafter the same).

  In each of the multiple scans, the cooperation control unit 335 causes the camera 310 to perform multiple continuous imaging, and the distance image generation unit 333 generates multiple imaging frames F corresponding to the multiple continuous imaging by the camera 310. The camera 310 and the distance image generation unit 333 are controlled in cooperation so that the one distance image DP is generated. The linkage control unit 335 performs a first scan (initial imaging process) executed first among a plurality of scans, and a second and subsequent scans executed after the first scan (thinning imaging process; 1, 2, 3, 4, 5, 6, 7, 8, 9... Further, among the second and subsequent scans, the cooperation control unit 335 performs an N-th scan (N is an even number equal to or greater than 2) (2, 4, 6, 8... Scan) and an N + 1-th scan ( In the third, fifth, seventh, ninth... Scan, different control processes are executed.

  Hereinafter, the process executed by the control of the cooperation control unit 335 at the first scan, the process executed by the control of the cooperation control unit 335 at the Nth scan, and the control of the cooperation control unit 335 at the N + 1th scan. The processes to be executed will be described sequentially. In the following, for convenience of explanation, the entire projection area TT is composed of 16 projection areas T1 to T16 (see FIGS. 8, 10, 12, etc., which will be described later) having the same area. Is K times (K is a positive integer) 16 times (that is, K = 16) and K / n times (n is an integer greater than or equal to 2) that the camera 310 continuously images during the second and subsequent scans Will be described 8 times (that is, n = 2). However, in practice, the entire projection area TT is composed of a larger number of projection areas (for example, 480), and the camera 310 continuously captures images more frequently during the first scan and the second and subsequent scans.

  First, with reference to FIGS. 8 and 9, a process executed by the control of the cooperation control unit 335 at the first scan will be described.

  8 and 9, in the first scan, the camera 310 sequentially captures 16 consecutive images for all of the 16 projection areas T1 to T16 included in the entire projection area TT under the control of the cooperation control unit 335. I do. That is, the camera 310 has a timing at which the projected portion of the moving laser slit light L becomes the projection region T1, a timing at which the projection region T2 becomes, a timing at which the projection region T3 becomes,. The appearance of the work W or the like in the entire projection area TT including the projection part is imaged at each of the timing of the projection area T15 and the timing of the projection area T16. Then, the camera 310 outputs 16 imaging frames F1 to F16 (see the upper part of FIG. 9) corresponding to the 16 consecutive imagings.

  Then, by the control of the cooperation control unit 335, the distance image generation unit 333 uses the 16 imaging frames F1 to F16 in the first scan stored in the image storage unit 332, so that the entire projection region TT is used. One distance image DP (see the lower part of FIG. 9) corresponding to all of the 16 projection regions T1 to T16 included is generated. Hereinafter, for convenience of explanation, the distance image DP in the first scan is referred to as “distance image DP1”. That is, the distance image generation unit 333 uses the 16 imaging frames F1 to F16 in the first scan, and the work W in the projection area T1 to T16 (that is, the entire projection area TT) in the camera 310. Calculate the distance to etc. The distance image generation unit 333 then includes portions p1 to p16 corresponding to the projection regions T1 to T16 (that is, the entire pp corresponding to the entire projection region TT) of the appearance image such as the workpiece W in the entire projection region TT. The distance image DP1 in which the calculated distance information is included is generated. As shown in the lower part of FIG. 9, the distance image DP1 includes distance information in the entire appearance pp of the external appearance image such as the workpiece W in the entire projection area TT. Therefore, by using this distance image DP1, the detection unit 340 can detect the three-dimensional shape of each of the plurality of workpieces W in the stocker 20 (described later). The distance image DP1 generated by the distance image generation unit 333 during the first scan is stored in the distance image storage unit 334 and is output to the detection unit 340.

  Next, with reference to FIGS. 10 and 11, a process executed by the control of the cooperation control unit 335 at the N-th scan will be described.

  10 and 11, in the N-th scan, the camera 310 has a total area of the total projection region TT among the sixteen projection regions T1 to T16 included in the total projection region TT under the control of the cooperation control unit 335. Of the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 that have been thinned and divided so as to be ½ of the sequential imaging. That is, the camera 310 has a timing at which the projected portion of the moving laser slit light L becomes the projection region T1, a timing at the projection region T3, a timing at the projection region T5, a timing at the projection region T7, and a projection region T9. The appearance of the workpiece W or the like in the entire projection area TT including the projection part is imaged at each of the timing, the timing of the projection area T11, the timing of the projection area T13, and the timing of the projection area T15. Then, the camera 310 outputs eight imaging frames F1 to F8 (see the upper part of FIG. 11) corresponding to the eight consecutive imagings.

  Then, by the control of the cooperation control unit 335, the distance image generation unit 333 uses the eight imaging frames F1 to F8 in the N-th scan stored in the image storage unit 332, so that the entire projection region TT is used. Among the 16 projection areas T1 to T16 included, one distance image DP (see the lower part of FIG. 11) corresponding to the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 is generated. To do. In the following, for convenience of explanation, the distance image DP in the Nth scan is referred to as “distance image DPN”. That is, the distance image generation unit 333 uses the eight imaging frames F1 to F8 in the N-th scan, and uses the camera 310 and the projection areas T1, T3, T5, T7, T9, T11, T13, and T15. The distance from the workpiece W etc. is calculated. The distance image generation unit 333 then includes portions p1, p3, p5 corresponding to the projection areas T1, T3, T5, T7, T9, T11, T13, and T15 of the external appearance image of the workpiece W or the like in the entire projection area TT. The distance image DPN in which the calculated distance information is included in p7, p9, p11, p13, and p15 is generated. As shown in the lower part of FIG. 11, the distance image DPN includes distance information in parts p1, p3, p5, p7, p9, p11, p13, and p15 of the appearance image such as the workpiece W in the entire projection area TT. The distance image DPN generated by the distance image generation unit 333 at the N-th scan is stored in the distance image storage unit 334.

  Next, with reference to FIGS. 12 and 13, processing executed by the control of the cooperation control unit 335 at the time of the N + 1th scan will be described.

  12 and 13, in the (N + 1) th scan, the camera 310 has a total area of the total projection region TT among the sixteen projection regions T1 to T16 included in the total projection region TT under the control of the cooperation control unit 335. The eight projection areas T2, T4 are thinned and divided so as to be ½ of the above and are arranged so as to fill the spaces between the eight projection areas T1, T3, T5, T7, T9, T11, T13, T15. , T6, T8, T10, T12, T14, and T16 are sequentially captured eight times. That is, the camera 310 has a timing at which the projected portion of the moving laser slit light L becomes the projection region T2, a timing at the projection region T4, a timing at the projection region T6, a timing at the projection region T8, and a projection region T10. The appearance of the workpiece W and the like in the entire projection area TT including the projection part is imaged at each of the timing, the timing of the projection area T12, the timing of the projection area T14, and the timing of the projection area T16. That is, under the control of the cooperation control unit 335, the imaging timing of the camera 310 in one scan (one process) in the N + 1th scan and the imaging timing of the camera 310 in one scan in the Nth scan are as follows. These are shifted so that the timings are alternate with respect to the start of each scan. Then, the camera 310 outputs eight imaging frames F1 to F8 (see the upper part of FIG. 13) corresponding to the eight consecutive imagings.

  Then, by the control of the cooperation control unit 335, the distance image generation unit 333 uses the eight imaging frames F1 to F8 in the N + 1th scan stored in the image storage unit 332, so that the entire projection region TT is used. Among the 16 projection areas T1 to T16 included, one distance image DP (see the lower part of FIG. 13) corresponding to the eight projection areas T2, T4, T6, T8, T10, T12, T14, and T16 is generated. To do. In the following, for convenience of explanation, the distance image DP in the N + 1th scan is referred to as “distance image DPN1”. That is, the distance image generation unit 333 uses the eight imaging frames F1 to F8 in the N + 1th scan, and uses the camera 310 and the projection areas T2, T4, T6, T8, T10, T12, T14, and T16. The distance from the workpiece W or the like is calculated. The distance image generation unit 333 then includes portions p2, p4, p6 corresponding to the projection regions T2, T4, T6, T8, T10, T12, T14, and T16 of the external appearance image of the workpiece W or the like in the entire projection region TT. The distance image DPN1 in which the calculated distance information is included in p8, p10, p12, p14, and p16 is generated. As shown in the lower part of FIG. 11, the distance image DPN1 includes distance information in parts p2, p4, p6, p8, p10, p12, p14, and p16 of the appearance image such as the workpiece W in the entire projection area TT. The distance image DPN1 generated by the distance image generation unit 333 during the N + 1th scan is stored in the distance image storage unit 334.

  When the N-th scan is made to correspond to the first thinning imaging process, the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 are divided projection areas and first divisions. One distance image DPN corresponding to the projection areas and corresponding to the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 corresponds to the first three-dimensional shape image, and the N + 1th time. Scan corresponds to the second thinning imaging step, and the eight projection areas T2, T4, T6, T8, T10, T12, T14, and T16 correspond to the divided projection area and the second divided projection area, and the eight One distance image DPN1 corresponding to the projection regions T2, T4, T6, T8, T10, T12, T14, and T16 corresponds to the second three-dimensional shape image. Conversely, when the N + 1th scan is made to correspond to the first thinning imaging process, the eight projection areas T2, T4, T6, T8, T10, T12, T14, and T16 are divided into the divided projection areas and the first projection area. One distance image DPN1 corresponding to the divided projection areas and corresponding to the eight projection areas T2, T4, T6, T8, T10, T12, T14, and T16 corresponds to the first three-dimensional shape image, and the N The second scan corresponds to the second thinning imaging step, and the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 correspond to the divided projection area and the second divided projection area, and the 8 One distance image DPN corresponding to the projection areas T1, T3, T5, T7, T9, T11, T13, and T15 corresponds to the second three-dimensional shape image.

  As illustrated in FIG. 7, after the first scan is completed, the detection unit 340 acquires the distance image DP1 in the first scan output from the distance image generation unit 333. And the detection part 340 detects the three-dimensional shape of each of the some workpiece | work W in the stocker 20 using the acquired distance image DP1. Thereafter, the detection unit 340 is based on the distance image DP based on the latest scan and the immediately preceding scan that are output from the composite image generation unit 336 as described later each time the second and subsequent scans are completed. A composite image CP (described later) obtained by combining with the distance image DP is acquired. And the detection part 340 detects the three-dimensional shape of each of the some workpiece | work W in the stocker 20 using the said synthesized image CP acquired. A detection signal representing a detection result by the detection unit 340 is output to the robot controller 200.

  Thereby, the robot controller 200 acquires the detection signal output from the detection unit 340. Then, the robot controller 200 holds and transfers one workpiece W (for example, the workpiece W that is most easily held) based on the three-dimensional shape information represented by the acquired detection signal. To work. In this example, when each scan is completed, the robot 100 transfers one work W (that is, scan → transfer one work W → scan → transfer one work W...). Although described, the disclosed embodiments are not limited to this. For example, as each scan ends, the robot 100 sequentially transfers a plurality of (for example, two) workpieces W (that is, scan → transfer one workpiece W → transfer one workpiece W → scan → one workpiece W). May be transferred → one workpiece W may be transferred ...).

  Each time the second and subsequent scans are completed, the composite image generation unit 336 performs multiple (this in this example) based on multiple scans (two times in this example) that are stored in the distance image storage unit 334 in time. In the example, two distance images DP are combined to generate a combined image CP (see FIG. 14 described later). The composite image CP is an image for detecting the three-dimensional shape of each of the plurality of workpieces W in the stocker 20. Specifically, the composite image generation unit 336 combines the distance image DP based on the latest scan and the distance image DP based on the immediately preceding scan every time the second and subsequent scans are completed, and combines them. An image CP is generated. In other words, the composite image generation unit 336 combines the portion including the distance information in the distance image DP based on the latest scan and the portion of the distance image DP based on the immediately preceding scan. As a result, the composite image generation unit 336 supplements a portion that does not include distance information in the distance image DP based on the latest scan with a portion that includes distance information in the distance image DP based on the immediately preceding scan. In this way, the composite image generation unit 336 generates a composite image CP. As shown in FIG. 14, the composite image CP includes distance information in the entire appearance pp of the appearance image such as the workpiece W in the entire projection area TT. Therefore, by using this composite image CP, the detection unit 340 can detect the three-dimensional shape of each of the plurality of workpieces W in the stocker 20.

  Here, in the present embodiment, as described above, after the first scan is finished, the distance image DP1 in the first scan is used, and thereafter each second and subsequent scans are finished. In addition, the robot 100 uses the synthesized image CP obtained by synthesizing the distance image DP based on the latest scan and the distance image DP based on the immediately preceding scan based on the three-dimensional shape information detected by the detection unit 340. One workpiece W is transferred. Therefore, when the robot 100 transfers one workpiece W as described above, the distance image DP in the immediately preceding scan used for the synthesis with the distance image DP in the latest scan executed immediately after that is used. Does not reflect the transfer of the single workpiece W (that is, the single transferred workpiece W remains in its original position). For this reason, when the composite image generation unit 336 generates the composite image CP by using the distance image DP in the previous scan unconditionally and combining it with the distance image DP in the latest scan, a reliable image is obtained. The composite image CP still contains low content, which is not preferable.

  Therefore, in the present embodiment, when the robot 100 transfers one workpiece W as described above, the composite image generation unit 336 displays the distance image DP in the latest scan executed immediately thereafter and the immediately preceding distance image DP. A synthesized image CP is generated by synthesizing the distance image DP in the scan with the image excluding the portion corresponding to the transferred workpiece W. That is, the composite image generation unit 336 excludes a portion corresponding to the transferred one workpiece W from the distance image DP in the immediately preceding scan. Then, the composite image generation unit 336 synthesizes the portion including the distance information in the distance image DP in the latest scan and the portion including the distance information in the distance image DP based on the immediately preceding scan after the exclusion process. Then, a composite image CP is generated.

  For example, as shown in FIG. 15, after the N + 1th (for example, third) scan is completed, the distance image DPN1 in the N + 1th scan and the distance image in the Nth (for example, second) scan immediately before the N + 1th scan Based on the three-dimensional shape information detected by the detection unit 340 using the composite image CP (herein referred to as “composite image CPN1”) obtained by combining with the DPN, the robot 100 has one work W (here, “work Wa” ”) Is transferred. In the example shown in FIG. 15, for convenience of explanation, it is assumed that the work W is not transferred by the robot 100 after the Nth scan and before the N + 1th scan. In this case, a portion corresponding to the transferred work Wa is excluded from the distance image DPN1 in the (N + 1) th scan. In FIG. 15, in the distance image DPN1 after the exclusion process (herein referred to as “distance image DPN1 ′”), a portion where data is missing (a portion corresponding to the excluded workpiece Wa) is indicated by a symbol Pa. Yes. Then, portions p1, p3, p5, p7, p9, and p11 including distance information in the distance image DP (herein referred to as “distance image DPN2”) in the N + 2 (for example, fourth) scan immediately after the (N + 1) th scan. , P13, p15 and the distance images DPN1 ′ based on the (N + 1) th scan after the exclusion process are combined with the portions p2, p4, p6, p8, p10, p12, p14, p16 including the distance information. CP (herein referred to as “composite image CPN2”) is generated. The composite image CPN2 includes a part Paa in which data is lost due to a part Pa in which data is lost in the distance image DPN1 ′ based on the N + 1th scan after the exclusion process. The part Paa is a part of the composite image CPN2 Are excluded from the above detection performed by the detection unit 340.

  Here, after the last scan and before the latest scan, for example, when the workpiece W is transferred by the robot 100, misalignment or load collapse has occurred in another workpiece W that is not transferred. There may be a case where the arrangement state of the workpiece W has changed due to some reason. At this time, there is a possibility that the portion where the change of the arrangement state of the workpiece W has occurred is relatively wide. In this case, the distance image DP in the immediately preceding scan used for synthesis with the distance image DP in the latest scan does not reflect the change in the arrangement state of the workpiece W (that is, the arrangement of the workpiece W). The situation has not changed and it remains the same). For this reason, if the composite image generation unit 336 generates the composite image CP for a portion where the arrangement state of the workpiece W has changed, the composite image CP including the content with low reliability is not preferable.

  Therefore, in this embodiment, when it is estimated that a change has occurred in the arrangement state of the workpiece W, specifically, the distance image DP in the latest scan and a composite image generated based on the scan immediately before the distance image DP. When the deviation of the data content in the corresponding part (or the vicinity part of the part; the same applies hereinafter) with the CP is greater than a predetermined threshold, the composite image generation unit 336 displays the distance image DP in the latest scan. And the image excluding the portion where the deviation of the data content is larger than the threshold and the image excluding the portion where the deviation of the data content is larger than the threshold in the distance image DP in the immediately preceding scan, A composite image CP is generated. That is, the composite image generation unit 336 excludes a portion of the distance image DP in the latest scan that has a data content deviation larger than a threshold value, and the distance image DP in the immediately preceding scan also Exclude the site. Then, the composite image generation unit 336 includes the distance information in the portion including the distance information in the distance image DP based on the latest scan after the exclusion process and the distance image DP in the distance image DP based on the immediately preceding scan after the exclusion process. The part is combined with each other to generate a composite image CP. Note that if the portion where the deviation of the data content is larger than the threshold is larger than the predetermined range, the first scan may be performed again without performing the exclusion process.

  For example, as shown in FIG. 16, after a (N + 1) th (for example, the third) scan, and immediately after the (N + 2) th (for example, the fourth) scan, a certain work W (here, “work Wb”). )) And a change in the arrangement state of the work W has occurred, so that the distance image DP in the N + 2th scan (herein referred to as “distance image DPN2”) and the distance image DP in the N + 1th scan A composite image CP (here “composite image DPN1”) and a composite image CP (here “composite image DPN1”) and a distance image DP (herein referred to as “distance image DPN”) in the Nth scan (for example, the second scan) just before that. Let us consider a case where the deviation of the data content in the corresponding part Rb is larger than the threshold value (referred to as “CPN1”). In the example shown in FIG. 16, for convenience of explanation, the work W is not transferred by the robot 100 after the Nth scan and before the N + 1th scan, and after the N + 1th scan. In addition, the work W is not transferred by the robot 100 before the N + 2th scan. In this case, the part Rb in which the deviation of the data content is larger than the threshold is excluded from the distance image DPN2 in the N + 2th scan, and the part Rb in the distance image DPN1 in the N + 1th scan is also excluded. Is excluded. In FIG. 16, in the distance image DPN1 (herein referred to as “distance image DPN1 ′”) and the distance image DPN2 (herein referred to as “distance image DPN2 ′”) after the exclusion processing, a portion (exclusion) where data is missing. The portion corresponding to the portion Rb made is indicated by the symbol Pb. In the distance image DPN2 ′ based on the N + 2th scan after the exclusion process, the portions p1, p3, p5, p7, p9, p11, p13, and p15 including the distance information and the N + 1th scan after the exclusion process are included. Of the based distance image DPN1 ′, the parts p2, p4, p6, p8, p10, p12, p14, and p16 including distance information are combined to generate a combined image CP (herein referred to as “combined image CPN2”). . The composite image CPN2 includes a portion Pb in which data is lost due to portions Pb and Pb in which data is missing in the distance images DPN2 ′ and DPN1 ′ based on the N + 2th and N + 1th scans after the exclusion process. The part Pb is excluded from the above detection performed by the detection unit 340 using the composite image CP.

  Next, an example of the composite image CP generated by the composite image generation unit 336 each time each scan ends will be described with reference to FIGS.

  In the example shown in FIGS. 17 to 19, after the first scan is completed, a certain work is performed by the robot 100 based on the three-dimensional shape information detected by using the distance image DP1 in the first scan. W (here called “work W1”) is being transferred. In this case, after the second scan is completed, a portion corresponding to the transferred work W1 is excluded from the distance image DP1 obtained in the first scan. In FIG. 17, in the distance image DP1 after the exclusion process (herein referred to as “distance image DP1 ′”), a portion where data is missing (a portion corresponding to the excluded workpiece W1) is indicated by a symbol P1. Yes. Of the distance image DP2 in the second scan, portions p1, p3, p5, p7, p9, p11, p13, and p15 including distance information, and the distance image DP1 ′ based on the first scan after the exclusion process Of these, the portions p2, p4, p6, p8, p10, p12, p14, and p16 including the distance information are combined to generate a combined image CP (herein referred to as “combined image CP2”). The composite image CP2 includes a portion P11 in which data is lost due to a portion P1 in which data is lost in the distance image DP1 ′ based on the first scan after the exclusion process, and the portion P11 includes the composite image CP2 It is excluded from the above-mentioned detection performed by using.

  Thereafter, based on the three-dimensional shape information detected using the composite image CP2 generated based on the second scan, a certain workpiece W (herein referred to as “work W2”) is transferred by the robot 100. ing. In this case, after the third scan is completed, a portion corresponding to the transferred work W2 is excluded from the distance image DP2 in the second scan. In FIG. 17, in the distance image DP2 after the exclusion process (herein, referred to as “distance image DP2 ′”), a portion where data is missing (a portion corresponding to the excluded workpiece W2) is indicated by a symbol P2. Yes. Then, in the distance image DP3 in the third scan, the parts p2, p4, p6, p8, p10, p12, p14, p16 including the distance information, and the distance image DP2 ′ based on the second scan after the exclusion process Of these, the portions p1, p3, p5, p7, p9, p11, p13, and p15 including the distance information are combined to generate a combined image CP (herein referred to as “combined image CP3”). The composite image CP3 includes a portion P22 in which data is lost due to a portion P2 in which data is lost in the distance image DP2 ′ based on the second scan after the exclusion process, and the portion P22 includes the composite image CP3. It is excluded from the above-mentioned detection performed by using. In the composite image CP3, the data is restored to the portion P11 where the data is missing in the composite image CP2.

  Then, based on the three-dimensional shape information detected using the composite image CP3 generated based on the third scan, a certain work W (herein referred to as “work W3”) is transferred by the robot 100. ing. In addition, after the third scan and before the fourth scan, an arrangement shift occurs in a certain work W (here, referred to as “work W0”), and the arrangement state of the work W changes. The deviation of the data contents in the corresponding part R0 between the distance image DP4 in the fourth scan and the composite image CP3 generated based on the third scan is larger than the threshold value. In this case, after the fourth scan is completed, a portion R0 in which the data content deviation is larger than the threshold is removed from the distance image DP4 in the fourth scan. In addition, the portion corresponding to the transferred workpiece W3 and the portion R0 where the data content deviation is larger than the threshold are excluded from the distance image DP3 in the third scan. In FIG. 18, a portion (exclusion) in which data is missing in the distance image DP4 (herein referred to as “distance image DP4 ′”) and the distance image DP3 (herein referred to as “distance image DP3 ′”) after the exclusion process. The portion corresponding to the portion R0 made) is indicated by the symbol P0, and the portion where the data is missing (the portion corresponding to the excluded workpiece W3) is indicated by the symbol P3. In the distance image DP4 ′ based on the fourth scan after the exclusion process, the portions p1, p3, p5, p7, p9, p11, p13, and p15 including the distance information and the third scan after the exclusion process are performed. Of the distance image DP3 ′, the portions p2, p4, p6, p8, p10, p12, p14, and p16 including the distance information are synthesized to generate a synthesized image CP (herein, “synthesized image CP4”). . The composite image CP4 includes a portion P0 in which data is lost due to portions P0 and P0 in which data is lost in the distance images DP4 ′ and DP3 ′ based on the fourth and third scans after the exclusion process. There is a portion P33 in which data is lost due to a portion P3 in which data is lost in the distance image DP3 ′ based on the third scan after the exclusion process, and these portions P0 and P33 are executed using the composite image CP4. It is excluded from the above detection target. In the composite image CP4, the data is restored to the portion P22 where the data is missing in the composite image CP3.

  Thereafter, based on the three-dimensional shape information detected using the composite image CP4 generated based on the fourth scan, a certain workpiece W (herein referred to as “work W4”) is transferred by the robot 100. ing. In this case, after the fifth scan is completed, a portion corresponding to the transferred workpiece W4 is excluded from the distance image DP4 ′ based on the fourth scan after the exclusion process. In FIG. 18, in the distance image DP4 ′ after the exclusion process (herein referred to as “distance image DP4 ″”), a portion where data is missing (a portion corresponding to the excluded workpiece W4) is indicated by a symbol P4. ing. Of the distance image DP5 in the fifth scan, portions p2, p4, p6, p8, p10, p12, p14, and p16 including distance information, and a distance image DP4 ″ based on the fourth scan after the exclusion process. Of these, the portions p1, p3, p5, p7, p9, p11, p13, and p15 including the distance information are combined to generate a combined image CP (herein referred to as “synthesized image CP5”). The composite image CP5 includes portions P00 and P44 in which data is lost due to the portions P0 and P4 in which data is lost in the distance image DP4 ″ based on the fourth scan after the exclusion process, and these portions P00 and P44. Is excluded from the above-described detection performed using the composite image CP5 In the composite image CP5, the data is restored to the portion P33 where the data is missing in the composite image CP4. Of the portion P0 in which data is missing in the composite image CP4, the data returns to the portion resulting from the portion P0 in which the data is missing in the distance image DP3 ″ based on the third scan after the exclusion process, and the portion P44. It has become.

  Then, on the basis of the three-dimensional shape information detected using the composite image CP5 generated based on the fifth scan, one work W (herein referred to as “work W5”) is transferred by the robot 100. ing. In this case, after the sixth scan is completed, a portion corresponding to the transferred work W5 is excluded from the distance image DP5 in the fifth scan. In FIG. 19, in the distance image DP5 after the exclusion process (herein referred to as “distance image DP5 ′”), a portion where data is missing (a portion corresponding to the excluded workpiece W5) is indicated by a symbol P5. Yes. Then, among the distance image DP6 in the sixth scan, the parts p1, p3, p5, p7, p9, p11, p13, p15 including the distance information, and the distance image DP5 ′ based on the fifth scan after the exclusion process Of these, the portions p2, p4, p6, p8, p10, p12, p14, and p16 including the distance information are combined to generate a combined image CP (herein referred to as “synthesized image CP6”). The composite image CP6 includes a portion P55 in which data is lost due to a portion P5 in which data is lost in the distance image DP5 ′ based on the fifth scan after the exclusion process, and this portion P55 includes the composite image CP6. It is excluded from the above-mentioned detection performed by using. In the composite image CP6, the data is restored to the portions P00 and P44 where the data is missing in the composite image CP5.

  Next, an example of the control procedure of the object detection method executed by the sensor controller 330 of the sensor unit 300 will be described with reference to FIG.

  In FIG. 20, the process shown in this flow is started when, for example, the sensor unit 300 is turned on.

  First, in step S10, the sensor controller 330 resets the value of the variable C for counting the number of scans to 1.

  Thereafter, the process proceeds to step S <b> 20, and the sensor controller 330 controls the laser light source 321 by the light source control unit 341 to irradiate the laser slit light L.

  In step S <b> 30, the sensor controller 330 controls the motor 323 based on the rotation angle information of the rotary mirror 322 input from the angle detector 324 by the motor control unit 342 to rotate the rotary mirror 322. Thereby, the projection part of the laser slit light L received and reflected by the rotating mirror 322 onto the workpiece W or the like in the entire projection area TT is moved in the scanning direction A.

  Thereafter, the process proceeds to step S40, and the sensor controller 330 detects the number of scans by detecting the value of the variable C at this time. If the value of the variable C at this time is 1, that is, if the current scan is the first scan, the process proceeds to step S50.

  In step S50, the sensor controller 330 controls the camera 310 by the cooperation control unit 335, and in the entire projection area TT including the projection part that moves by the control in step S20 at a desired frame rate (imaging timing). The external appearance of the workpiece W or the like is continuously imaged 16 times, and the corresponding 16 imaging frames F1 to F16 are output. In other words, under the control of the cooperation control unit 335 in step S50, the camera 310 becomes the projection region T1, the projection region T2, the projection region T2, the projection region T2, and the projection region T3. At each of the timing,..., The timing to become the projection area T14, the timing to become the projection area T15, and the timing to become the projection area T16, the external appearance of the workpiece W and the like in the entire projection area TT including the projection part is imaged. Sixteen imaging frames F1 to F16 corresponding to these 16 consecutive imaging operations are output.

  In step S60, the sensor controller 330 uses the image acquisition unit 331 to acquire the 16 imaging frames F1 to F16 output from the camera 310 in step S50. The acquired 16 imaging frames F1 to F16 are stored in the image storage unit 332.

  Thereafter, the process proceeds to step S70, and the sensor controller 330 controls the distance image generation unit 333 by the cooperation control unit 335, and the 16 imaging frames F1 to F16 in the first scan stored in the image storage unit 332 are stored. Is used to generate one distance image DP1 corresponding to all of the 16 projection areas T1 to T16 included in the entire projection area TT. The method for generating the distance image DP1 during the first scan is as described above. The generated distance image DP1 is stored in the distance image storage unit 334 and is output to the detection unit 340.

  In step S80, the sensor controller 330 uses the detection unit 340 to acquire the distance image DP1 output from the distance image generation unit 333 in step S70. Then, the sensor controller 330 uses the detection unit 340 to detect the three-dimensional shape of each of the plurality of workpieces W in the stocker 20 using the acquired distance image DP1. A detection signal indicating a detection result by the detection unit 340 is output to the robot controller 200. As a result, the robot controller 200 acquires the detection signal output from the detection unit 340 in step S80 and holds one workpiece W based on the three-dimensional shape information represented by the acquired detection signal. The robot 100 is operated so as to be transferred. Thereafter, the process proceeds to step S170 described later.

  On the other hand, in step S40, if the value of the variable C at this time is N as described above, that is, if the current scan is the Nth scan, the process proceeds to step S90.

  In step S90, the sensor controller 330 controls the camera 310 by the cooperation control unit 335, and in the entire projection area TT including the projection part that moves by the control in step S20 at a desired frame rate (imaging timing). The appearance of the workpiece W or the like is continuously imaged eight times, and the corresponding eight imaging frames F1 to F8 are output. That is, under the control of the cooperation control unit 335 in step S90, the camera 310 becomes the projection region T1 when the projected portion of the moving laser slit light L becomes the projection region T1, the timing when the projection region T3, and the projection region T5. Within the entire projection area TT including the projection part, the timing, the timing to become the projection area T7, the timing to become the projection area T9, the timing to become the projection area T11, the timing to become the projection area T13, and the timing to become the projection area T15 The external appearance of the workpiece W or the like is imaged, and eight imaging frames F1 to F8 corresponding to these eight continuous imaging are output.

  Thereafter, the process proceeds to step S100, and the sensor controller 330 acquires the eight imaging frames F1 to F8 output from the camera 310 in step S90 by the image acquisition unit 331. The acquired eight imaging frames F1 to F8 are stored in the image storage unit 332.

  In step S <b> 110, the sensor controller 330 controls the distance image generation unit 333 by the cooperation control unit 335, and stores the eight imaging frames F <b> 1 to F <b> 8 in the Nth scan stored in the image storage unit 332. The one distance image DPN corresponding to the eight projection areas T1, T3, T5, T7, T9, T11, T13, and T15 out of the 16 projection areas T1 to T16 included in the total projection area TT. Is generated. The method for generating the distance image DPN at the N-th scan is as described above. The generated distance image DPN is stored in the distance image storage unit 334. Thereafter, the process proceeds to step S150 described later.

  On the other hand, in step S40, if the value of the variable C at this time is N + 1 described above, that is, if the current scan is the N + 1th scan, the process proceeds to step S120.

  In step S120, the sensor controller 330 controls the camera 310 by the cooperation control unit 335, and in the entire projection area TT including the projection part that moves by the control in step S20 at a desired frame rate (imaging timing). The appearance of the workpiece W or the like is continuously imaged eight times, and the corresponding eight imaging frames F1 to F8 are output. In other words, under the control of the cooperation control unit 335 in step S120, the camera 310 becomes the projection region T2, the projection region T4, the projection region T4, the projection region T4, and the projection region T6. Within the entire projection area TT including the projection part, the timing, the timing to become the projection area T8, the timing to become the projection area T10, the timing to become the projection area T12, the timing to become the projection area T14, and the timing to become the projection area T16 The external appearance of the workpiece W or the like is imaged, and eight imaging frames F1 to F8 corresponding to these eight continuous imaging are output.

  Thereafter, the process proceeds to step S130, and the sensor controller 330 acquires the eight imaging frames F1 to F8 output from the camera 310 in step S120 by the image acquisition unit 331. The acquired eight imaging frames F1 to F8 are stored in the image storage unit 332.

  In step S <b> 140, the sensor controller 330 controls the distance image generation unit 333 by the cooperation control unit 335, and stores the eight imaging frames F <b> 1 to F <b> 8 in the N + 1th scan stored in the image storage unit 332. The one distance image DPN1 corresponding to the eight projection areas T2, T4, T6, T8, T10, T12, T14, and T16 out of the 16 projection areas T1 to T16 included in the total projection area TT. Is generated. Note that the method of generating the distance image DPN1 during the N + 1th scan is as described above. The generated distance image DPN1 is stored in the distance image storage unit 334.

  Thereafter, the process proceeds to step S150, where the sensor controller 330 is a distance image based on a scan (latest scan) corresponding to the value of the variable C at this time stored in the distance image storage unit 334 by the composite image generation unit 336. DP and the distance image DP based on the immediately preceding scan are combined to generate a combined image CP. The method for generating the composite image CP is as described above. The generated composite image CP is stored in the composite image storage unit 337 and is output to the detection unit 340.

  In step S160, the sensor controller 330 acquires the composite image CP output from the composite image generation unit 336 in step S150 by the detection unit 340. Then, the sensor controller 330 uses the detection unit 340 to detect the three-dimensional shape of each of the plurality of workpieces W in the stocker 20 using the acquired composite image CP. A detection signal indicating a detection result by the detection unit 340 is output to the robot controller 200. Thereby, the robot controller 200 acquires the detection signal output from the detection unit 340 in step S160, and holds one workpiece W based on the three-dimensional shape information represented by the acquired detection signal. The robot 100 is operated so as to be transferred.

  Thereafter, the process proceeds to step S170, and the sensor controller 330 adds 1 to the value of the variable C, and then returns to step S20 to repeat the same procedure.

  Note that the processing shown in this flow is terminated when, for example, the power of the sensor unit 300 is turned off.

  As described above, in the sensor unit 300 of the present embodiment, the workpiece W is detected by the light cutting method. The laser slit light L emitted from the laser light source 321 is projected onto the workpiece W or the like, and the appearance of the workpiece W or the like including the projected portion of the laser slit light L is imaged by the camera 310. At this time, the projection part of the laser slit light L onto the workpiece W or the like is moved in the scanning direction A by rotating the rotary mirror 322. During this movement, the camera 310 continuously captures images at a desired frame rate, so that a plurality of imaging frames F each including the behavior of the projected part that moves every moment are output from the camera 310. Using the plurality of imaging frames F, the distance image generation unit 333 generates one distance image DP such as the workpiece W. Thereby, the three-dimensional shape of the workpiece | work W is detectable.

  In such a normal light cutting method, when imaging is performed by moving the projection part of the laser slit light L on the workpiece W or the like in the scanning direction A as described above, the end of the workpiece W or the like on the side opposite to the scanning direction is used. Between the time when the projected portion is generated in the portion and the projected portion is sequentially moved in the scanning direction A and further until the projected portion reaches the end on the scanning direction A side (in other words, substantially along the scanning direction A on the workpiece W or the like). The image must be taken evenly throughout the entire area). As a result, it takes a relatively long time to perform the continuous imaging, and it is difficult to improve the efficiency of the detection process.

  Therefore, in the present embodiment, the three-dimensional shape of the workpiece W is not detected using the plurality of imaging frames F in one scan that sequentially moves in the scanning direction A on the workpiece W or the like as described above. The three-dimensional shape of the workpiece W is detected using an image obtained by combining a plurality of imaging frames F in each of the scans. That is, the composite image generation unit 336 generates a composite image CP by combining the plurality of distance images DP generated by the distance image generation unit 333 based on a plurality of scans (different from each other in time). Then, the three-dimensional shape of the workpiece W is detected from the composite image CP.

  As described above, by adopting a technique for compositing images from a plurality of scans, the shape of the workpiece W can be detected by using the old imaging frame F in the already executed scan other than the imaging frame F in the latest scan. At this time, the time required for imaging can be set as a time for one scan for obtaining the latest imaging frame F. As a result, the detection process can be greatly shortened compared to the conventional method, and the processing efficiency can be improved. Alternatively, when it is possible to spend the same time as the conventional method, it is possible to extend an allowable time in one scan, and thus it is possible to use the camera 310 having a relatively low imaging speed. As a result, the cost can be reduced as compared with the conventional method. Alternatively, when it is not necessary to reduce the imaging speed while taking the same time as the conventional method, the resolution can be increased to the number of times of the plurality of scans.

  Further, in this embodiment, in particular, the projection part of the laser slit light L onto the work W or the like is moved in the scanning direction A by rotating the rotary mirror 322. Thus, by using the rotary mirror 322, the laser slit light L reflected toward the workpiece W or the like can be easily moved in the scanning direction A.

  In the present embodiment, in particular, based on the control of the cooperation control unit 335, the other than the eight imaging frames F1 to F8 by the eight consecutive imagings in the latest scan (second and subsequent scans) are executed immediately before that. Eight imaging frames F1 to F8 obtained by eight consecutive imaging operations in one completed scan are used to generate a composite image CP. As a result, the time required for imaging when detecting the shape of the workpiece W can be halved compared to the conventional method, the detection process can be greatly shortened, and the processing efficiency can be doubled. be able to. Alternatively, when the time equivalent to that of the conventional method may be taken, the allowable time in one scan can be doubled, and thus the camera 310 whose imaging speed is ½ slower than that of the conventional method is used. be able to. As a result, the cost can be reduced as compared with the conventional method. Alternatively, when it is not necessary to reduce the imaging speed while taking the same time as the conventional method, the resolution can be increased by a factor of two.

  However, in the first scan (first scan), there is no previous scan, so the above method is impossible. In the present embodiment, unlike the second and subsequent scans, the camera 310 sequentially captures 16 consecutive images in the entire projection area TT such as the workpiece W based on the control of the cooperation control unit 335. The 16 imaging frames F1 to F16 are used. Thereby, even in the first scan that cannot be shortened by thinning, one distance image DP can be reliably generated.

  In the present embodiment, in particular, the imaging timings of the N-th scan and the N + 1-th scan are shifted so as to be alternate with each other on the basis of the start of each scan. Accordingly, the eight imaging frames F1 to F8 in the Nth scan and the eight imaging frames F1 to F8 in the N + 1th scan can be positioned so as to complement each other. As a result, by combining them, an image for detecting the three-dimensional shape of the workpiece W can be obtained easily and reliably as in the conventional method.

  In the robot system 1 of the present embodiment, the plurality of workpieces W in the stocker 20 are sequentially held and transferred by the robot 100. In the present embodiment, during the transfer, the robot 100 operates according to the three-dimensional shape of each workpiece W detected by the sensor unit 300 based on the control of the robot controller 200 as described above. Thereby, the said several workpiece | work W can be transferred smoothly and reliably. Further, since the three-dimensional shape of each workpiece W is detected quickly and efficiently as described above, the transfer of the plurality of workpieces W by the robot 100 can be performed reliably and smoothly.

  In the present embodiment, in particular, when the robot 100 performs transfer of a certain workpiece W, the composite image generation unit 336 selects the transferred one of the distance images DP generated in the immediately preceding scan. A portion corresponding to the workpiece W is removed, and the removed image is combined with the distance image DP generated in the immediately following scan to generate a combined image CP. Thereby, the content of the part with low reliability can be excluded and the robot 100 can be controlled. As a result, it is possible to avoid a malfunction or meaningless operation of the robot 100 and perform a good work W transfer operation. In addition, the distance image DP including the uncertain contents as described above is then scanned again and the data reflecting the latest situation is overwritten at the same place, so that the highly reliable contents are again obtained. can do.

  In the present embodiment, in particular, when it is estimated that a change has occurred in the arrangement state of the workpiece W, specifically, the distance image DP generated in the latest scan and the composite image generated in the immediately preceding scan are generated. When the deviation of the data content in the corresponding part from the CP is larger than the predetermined threshold, the synthesized image generation unit 336 has the data content deviation of the distance image DP generated by the latest scan larger than the threshold. Exclude large parts. In addition, the part in the distance image DP generated in the immediately preceding scan is also excluded. Then, the synthesized image CP is generated by synthesizing the two distance images DP subjected to the exclusion process. As a result, similarly to the above, the robot 100 can be controlled by excluding the contents of the portion with low reliability. As a result, it is possible to avoid a malfunction or meaningless operation of the robot 100 and perform a good work W transfer operation.

  The embodiment is not limited to the above contents, and various modifications can be made without departing from the spirit and technical idea of the embodiment.

  For example, in the above-described embodiment, when the camera 310 performs the N-th scan, the projection part of the moving laser slit light L becomes the projection area T1, the projection area T3, and the projection area T5. , The timing to become the projection region T7, the timing to become the projection region T9, the timing to become the projection region T11, the timing to become the projection region T13, and the timing to become the projection region T15, respectively, within the entire projection region TT including the projection region. When the appearance of the workpiece W or the like is imaged and the N + 1-th scan is performed, the projected portion of the moving laser slit light L becomes the projection region T2, the timing of the projection region T4, the timing of the projection region T6, the projection region Timing for T8, timing for projection area T10, projection area Although the appearance of the workpieces W and the like in the entire projection area TT including the projection part is captured at each of the timing of T12, the timing of the projection area T14, and the timing of the projection area T16, the embodiment of the present disclosure Is not limited to this. That is, at the time of the N-th scan, the camera 310 has the projection region of the moving laser slit light L as the projection region T2, the timing as the projection region T4, the timing as the projection region T6, and the projection region T8. At each of the timing, the timing to become the projection region T10, the timing to become the projection region T12, the timing to become the projection region T14, and the timing to become the projection region T16, the appearance of the work W etc. in the entire projection region TT including the projection part When the image is captured and the (N + 1) th scan, the projected portion of the moving laser slit light L becomes the projection region T1, the timing of the projection region T3, the timing of the projection region T5, the timing of the projection region T7, the projection Timing to be the region T9, timing to be the projection region T11 , Timing of the projection region T13, at each timing when the projection region T15, the appearance such as a work W in the total projected area TT including the projection portion may be captured.

  In the above-described embodiment, in the second and subsequent scans, the camera 310 sequentially performs K / 2 times on a plurality of projection areas thinned out so that the total area becomes 1/2 of the total projection area TT. Continuous imaging is performed, the distance image generation unit 333 generates one distance image DP using the K / 2 imaging frames F, and the composite image generation unit 336 combines the two distance images DP to generate a composite image. CP was generated, that is, n = 2. However, n is not particularly limited as long as it is an integer equal to or greater than 2, and may be n = 3, n = 4, or n may be a larger integer. For example, when n = 3, in the second and subsequent scans, the camera 310 sequentially applies K / to a plurality of projection areas thinned out so that the total area becomes 1/3 of the total projection area TT. The distance image generation unit 333 generates one distance image DP using K / 3 imaging frames F, and the composite image generation unit 336 combines the three distance images DP. To generate a composite image CP.

  Further, in the above embodiment, the thinning scan as in the second and subsequent scans is not performed at the first scan, but the embodiment of the present disclosure is not limited to this, and the first scan is not performed. A thinning scan may be performed at the time of scanning.

  Moreover, in the said embodiment, although the sensor unit 300 was applied to the robot system 1, a sensor unit is applicable besides a robot system.

  Moreover, the arrows shown in FIG. 7 show an example of the signal flow, and do not limit the signal flow direction.

  Further, the flowchart shown in FIG. 20 is not limited to the procedure illustrated in the embodiment, and the procedure may be added / deleted or the order may be changed without departing from the spirit and technical idea.

  In addition to those already described above, the methods according to the above embodiments may be used in appropriate combination.

  In addition, although not illustrated one by one, the above-described embodiments and the like are implemented with various modifications within a range not departing from the gist thereof.

1 Robot system 20 Stocker (container)
100 robot 200 robot controller (controller)
300 Sensor unit (object detection device)
310 Camera 321 Laser light source 322 Rotating mirror (scanning unit)
333 Distance image generation unit 335 Cooperation control unit 336 Composite image generation unit CP Composite image DP Distance image (three-dimensional shape image)
F Imaging frame (image data)
L Laser slit light (slit laser light)
T1 to T16 Projection area TT Total projection area W Work (object)

Claims (9)

An object detection device for detecting an object to be detected,
A laser light source for irradiating slit-shaped laser light;
A scanning unit that moves a projection part of the laser light emitted from the laser light source onto the object in a predetermined scanning direction;
When the scanning unit moves the projection part of the laser beam on the object, the appearance of the object including the moving projection part is continuously imaged at a desired interval, and a plurality of corresponding image data is output. Camera to
A distance image generating unit that generates one three-dimensional shape image of the object using a plurality of the image data for each imaging step in which the camera performs a plurality of continuous imagings;
A plurality of the three-dimensional shape images generated by the distance image generation unit based on the plurality of imaging steps that are temporally different from each other are combined to generate a combined image for detecting the three-dimensional shape of the object. A composite image generation unit;
An object detection apparatus comprising: The scanning unit
The object detection apparatus according to claim 1, wherein the object detection apparatus is a rotating mirror that receives the laser light emitted from the laser light source and reflects the laser light toward the object. In the initial imaging process among the plurality of imaging processes, the camera sequentially performs continuous imaging of K times (K is a positive integer) for all projection regions to be projected of the object, and the distance image generation unit Using the K pieces of the image data to generate the one three-dimensional shape image corresponding to the entire projection area; and
In one or more thinning imaging steps executed after the initial imaging step among the plurality of imaging steps, the thinning is performed so that the total area is 1 / n (n is an integer of 2 or more) of the total projection region. In addition, the camera sequentially captures K / n times for a plurality of divided projection areas, and the distance image generation unit uses K / n pieces of the image data to correspond to each thinning imaging process. So as to generate three 3D shape images
The object detection apparatus according to claim 2, further comprising a cooperation control unit that controls the camera and the distance image generation unit in cooperation with each other. The plurality of thinning imaging steps include
Under the control of the cooperation control unit, the camera sequentially performs continuous imaging of K / 2 times for a plurality of first divided projection areas that are thinned and divided so that the total area becomes 1/2 of the total projection area, In addition, a first decimation imaging step in which the distance image generation unit generates one first three-dimensional shape image corresponding to the plurality of first divided projection areas by using K / 2 pieces of the image data;
A plurality of second divided projections that are thinned and divided so that the total area is ½ of the total projection area under the control of the cooperative control unit, and are arranged so as to respectively fill the plurality of first divided projection areas. The camera sequentially captures K / 2 times for the area, and the distance image generation unit uses K / 2 pieces of the image data to thereby provide one corresponding to the plurality of second divided projection areas. A second thinning imaging step of generating a second three-dimensional shape image;
Including
The composite image generation unit
The first 3D shape image generated by the distance image generation unit in the first decimation imaging step and the second 3D shape image generated by the distance image generation unit in the second decimation imaging step are combined. The object detection apparatus according to claim 3, wherein the composite image is generated. The cooperation control unit
The imaging timing of the camera in one step in the first decimation imaging step and the imaging timing of the camera in one step in the second decimation imaging step are alternate timings based on the start of each step. The object detection apparatus according to claim 4, wherein the camera is controlled so that A container containing a plurality of the objects;
A robot for sequentially holding and transferring the plurality of objects in the container;
The object detection device according to claim 4 or 5,
A controller for operating the robot based on a three-dimensional shape of each of the plurality of objects in the container detected by the object detection device;
A robot system comprising: In the object detection device, the tertiary by combining the second 3D shape image generated in the second thinning imaging step and the first 3D shape image generated in the immediately preceding first thinning imaging step. Based on the detection of the original shape, when the robot has transferred the corresponding object,
The composite image generation unit
The first three-dimensional shape image generated in the first thinning-out imaging step immediately after the second thinning-out imaging step and the second three-dimensional shape image generated in the second thinning-out imaging step are transferred. The robot system according to claim 6, wherein the synthesized image is generated by synthesizing an image excluding a portion corresponding to the object. Corresponding part of the first three-dimensional shape image generated in the first thinning imaging step and the synthetic image generated by the synthetic image generation unit based on the second thinning imaging step immediately before When the deviation of the data content in the vicinity of the part is larger than a predetermined threshold,
The composite image generation unit
Of the first three-dimensional shape image generated in the first decimation imaging step, an image excluding a portion where the deviation of the data content is larger than the threshold value and generated in the immediately preceding second decimation imaging step The composite image is generated by combining the second three-dimensional shape image with an image excluding a portion where the deviation of the data content is larger than the threshold value. Robot system. Irradiating slit-shaped laser light from a light source;
Moving the projected portion of the irradiated laser light onto the object in a predetermined scanning direction;
When moving the projection portion of the laser light, continuously imaging the appearance of the object including the moving projection portion at a desired interval, and outputting a plurality of corresponding image data;
Generating one three-dimensional shape image of the object using a plurality of the image data for each imaging step in which a plurality of continuous imaging is performed;
Combining a plurality of the three-dimensional shape images respectively generated based on the plurality of imaging steps temporally different from each other to detect a three-dimensional shape of the object;
An object detection method comprising:

Images