JP5634764B2 - MOBILE BODY CONTROL SYSTEM, PROGRAM, AND MOBILE BODY CONTROL METHOD - Google Patents

Description

  The present invention relates to a mobile control system, a program, and a mobile control method.

Patent Document 1 discloses a positioning device that positions a positioning object. The positioning device captures an image including a mark provided on the object installed on the table with a plurality of cameras while moving the table. The positioning device calculates a deviation amount between the mark position and the mark target position using each image, and rotates or translates the table until the deviation is eliminated. Patent Document 2 discloses a workpiece positioning device that specifies coordinates of a rotation center of a stage necessary for correcting a rotational deviation of a workpiece. The workpiece positioning apparatus measures the coordinates indicating the position of each mark from each image obtained by two imaging operations with two cameras. The workpiece positioning device calculates the rotation angle of the stage using the movement amount of each mark obtained from each coordinate and the distance between the marks inputted in advance, and uses the rotation angle and the respective coordinates of each mark. Calculate the coordinates of the rotation center of the stage. Patent Document 3 discloses an electronic component mounting apparatus that mounts chip-shaped electronic components at predetermined positions on an electronic circuit board. In the electronic component mounting apparatus, the suction nozzle is rotated based on the position data of the suction nozzle input in advance and the position data of the reference position of the electronic component obtained based on the image captured by the camera. Correct the posture of the part.
Patent Document 1 Japanese Patent No. 3531694 Patent Document 2 Japanese Patent Application Laid-Open No. 2006-49755 Japanese Patent Application Laid-Open No. 5-48295

  By the way, in the moving body control system that controls the moving body based on the images captured by the plurality of imaging units, if the accuracy of the correspondence relationship of the respective imaging coordinate systems determined for each imaging unit is poor, It takes time to move the object to the target position.

  Therefore, in a moving body control system that controls a moving body based on images captured by a plurality of imaging units, to improve the accuracy of the correspondence relationship of each imaging coordinate system defined for each imaging unit. Is desired.

  A mobile body control system according to an embodiment of the present invention includes an arbitrary first within a first imaging range of a first imaging unit that captures a first image including a first reference mark that is predetermined for the mobile body. When the moving body is rotated around the point, the first image acquisition unit that acquires the first image captured by the first imaging unit before and after the rotation, and the moving body is rotated around the first point. Second image acquisition for acquiring each of the second images captured by the second imaging unit that captures a second image including a second reference mark predetermined for the moving body before and after the rotation. And a first rotation center position corresponding to the first point in the first imaging coordinate system predetermined for the first imaging unit based on the first reference mark included in each of the first image and the first image. 1st rotation center position specific | specification part to specify, and each 2nd image A second rotation center position specifying unit that specifies a second rotation center position corresponding to the first point in the second imaging coordinate system predetermined for the second imaging unit based on each included second reference mark. And a coordinate system correspondence holding unit that holds a coordinate system correspondence indicating a correspondence between the first imaging coordinate system and the second imaging coordinate system based on the first rotation center position and the second rotation center position.

  In the mobile body control system, the first image acquisition unit may rotate the mobile body sequentially around each of a plurality of arbitrary first points in the first imaging range before and after each rotation. When the first image captured by the first imaging unit is acquired, and the second image acquisition unit rotates the moving body sequentially around each of a plurality of first points within the first imaging range. The second image captured by the second imaging unit before and after each rotation is acquired, and the first rotation center position specifying unit is included in each of the first images before and after the rotation. Based on the first reference mark, each first rotation center position corresponding to each first point is specified, and the second rotation center position specifying unit is included in each second image before and after rotation. Based on each second reference landmark The second rotation center position corresponding to each first point is specified, and the coordinate system correspondence holding unit holds the coordinate system correspondence based on the first rotation center position and the second rotation center position. May be.

  In the mobile body control system, the first image acquisition unit is configured to rotate the mobile body around the arbitrary second point in the second imaging range of the second imaging unit before rotating and after rotating. The first image captured by the imaging unit is acquired, and the second image acquisition unit is captured by the second imaging unit before and after the rotation when the moving body is rotated around the second point. Each of the second images is acquired, and the first rotation center position specifying unit determines the second point in the first imaging coordinate system based on the respective first reference marks included in the first images before and after the rotation. The second rotation center position specifying unit specifies the second imaging coordinate system based on the respective second reference marks included in the respective second images before and after the rotation. A second rotation center position corresponding to the second point at Target system correspondence holding unit may hold the first rotation center position and the coordinate system correspondence relationship based on the second rotation center position corresponding to the second point.

  In the mobile body control system, the first image acquisition unit may rotate the mobile body sequentially around each of a plurality of arbitrary second points within the second imaging range before and after each rotation. When the first image captured by the first imaging unit is acquired, and the second image acquisition unit rotates the moving body sequentially around each of a plurality of arbitrary second points within the second imaging range. The second image captured by the second imaging unit before and after each rotation is acquired, and the first rotation center position specifying unit is included in each of the first images before and after the rotation. Based on the first reference mark, each first rotation center position corresponding to each second point is specified, and the second rotation center position specifying unit is included in each second image before rotation and after rotation. Based on each second reference landmark The respective second rotation center positions corresponding to the respective second points are specified, and the coordinate system correspondence holding unit is provided at the respective first rotation center positions and the second rotation center positions corresponding to the respective second points. The corresponding coordinate system correspondence may be held.

  In the mobile body control system, by referring to the coordinate system correspondence, the first imaging of a first mark predetermined for an object held by the mobile body included in the first image captured by the first imaging unit. The first imaging coordinate system or the second imaging coordinate system of the second mark predetermined for the first position in the coordinate system or the second imaging coordinate system and the object included in the second image captured by the second imaging unit In the first imaging coordinate system or the second imaging coordinate system, a position specifying unit that specifies the second position in the first imaging coordinate system, a straight line connecting the first position and the second position in the second imaging coordinate system, and the first imaging coordinate system or the second imaging coordinate system are predetermined. Based on an angle formed by the target line, a predetermined point in the first imaging range of the first imaging unit or the second imaging range of the second imaging unit is centered to move the object to the target position. As mobile machine A, and a moving mechanism control unit for rotating the movable body through.

  In the moving body control system, the moving mechanism control unit moves the object to the target position by rotating the moving body through the moving mechanism around a point determined based on the first mark or the second mark. Also good.

  In the above moving body control system, the moving mechanism translates the moving body in two first direction moving mechanisms that translate the moving body in a predetermined first direction and a second direction different from the first direction. The movement mechanism control unit may rotate the moving body by controlling the two first direction movement mechanisms and the second direction movement mechanism, respectively.

  In the above moving body control system, when the moving body is translated by an arbitrary distance, the measured value and the second reference mark of the moving distance in the first imaging coordinate system of the first reference mark before and after the parallel movement are translated. A ratio acquisition unit that acquires a ratio of measured values of the movement distance in the second imaging coordinate system, a correction coefficient specifying unit that specifies a correction coefficient based on the ratio of the measured values, and a case where the object is moved to the target position, You may further provide the parallel displacement amount specific | specification part which specifies the parallel displacement amount of a moving body based on a correction coefficient.

  In the mobile body control system, when the mobile body is rotated a plurality of times around an arbitrary point, based on each first reference mark or second reference mark included in each first image or second image. Rotation that holds a rotation angle correspondence relationship that indicates a correspondence relationship between the rotation angle in the first imaging coordinate system or the second imaging coordinate system that is specified in this way and the rotation angle in the moving body coordinate system that is predetermined for the moving body An angle correspondence holding unit may be further provided.

  A moving body control system according to an embodiment of the present invention includes an imaging coordinate system predetermined for an imaging unit that captures an image including a reference mark predetermined for the moving body, and a moving body. A coordinate system correspondence holding unit for holding a coordinate system correspondence indicating a correspondence with a predetermined moving body coordinate system, and a coordinate value of an arbitrary point within or outside the imaging range in the imaging coordinate system. Specify the coordinate value in the moving object coordinate system corresponding to the specified coordinate value based on the coordinate system correspondence relationship and specify the coordinate value in the specified coordinate value, and rotate the moving object at any rotation angle around the specified coordinate value When the image is rotated and moved, the image acquisition unit that acquires the images captured by the imaging unit before and after the movement, and the rotation of the moving body based on the reference marks and the rotation angles included in the images, respectively. During ~ A rotation center position specifying unit that specifies the coordinate value in the imaging coordinate system for the position as the rotation center coordinate value, and a correspondence correction unit that corrects the coordinate system correspondence based on the coordinate value and the rotation center coordinate value in the moving object coordinate system With.

  In the above moving body control system, the image acquisition unit is imaged by the imaging unit before and after the movement when the moving body is rotated a plurality of times at an arbitrary rotation angle around the specified coordinate value. Each of the above images is acquired, and the rotation center position specifying unit calculates a coordinate value in the imaging coordinate system with respect to the rotation center position of the moving body based on each reference mark and rotation angle included in each image. May be specified.

  In the mobile body control system, the specified coordinate value specifying unit specifies a plurality of specified coordinate values, and the image acquisition unit is configured to correspond to each of the mobile body coordinate systems corresponding to the plurality of specified coordinate values based on the coordinate system correspondence. When the coordinate value is specified, and the moving body is rotated at an arbitrary rotation angle around the specified coordinate value, the image captured by the imaging unit is acquired before and after the movement, and rotated. The center position specifying unit sets the coordinate value in the imaging coordinate system for the rotation center position of the moving body as a rotation center coordinate value for each of a plurality of specified coordinate values based on the respective reference marks and rotation angles included in the respective images. The correspondence relationship correction unit may specify and correct the coordinate system correspondence relationship based on the specified coordinate values and the rotation center coordinate values.

  In the mobile body control system, the specified coordinate value specifying unit specifies a plurality of the specified coordinate values corresponding to the coordinate values arranged on the predetermined linear in the imaging coordinate system, and the correspondence correction unit A plurality of rotation center coordinate values specified for each specified coordinate value are corrected based on the linearly arranged coordinate values, and based on each specified coordinate value and each corrected rotation center coordinate value Thus, the coordinate system correspondence may be corrected.

  In the above moving body control system, when the moving body is rotated around the designated coordinate value, the image acquisition unit includes a first reference mark that is predetermined for the moving body before and after the rotation. A first image acquisition unit that acquires a first image captured by a first imaging unit that captures an image, and when the moving body is rotated around a specified coordinate value, the moving body is rotated before and after the rotation. A second image acquisition unit that acquires each of the second images captured by the second imaging unit that captures a second image including a second reference mark that is determined in advance, and the coordinate system correspondence holding unit is The correspondence relationship between the first imaging coordinate system of the first imaging unit and the second imaging coordinate system of the second imaging unit is maintained, and the rotation center position specifying unit includes the first reference mark and the second included in the first image. Identified separately based on each of the second reference landmarks included in the image. That based on Chi respective rotation center coordinates of the first image pickup image acquisition coordinate system and the second imaging coordinate system may identify a rotation center coordinate values of 1 in either one of the imaging coordinate system.

  In the above moving body control system, when the moving body is rotated around the designated coordinate value, the image acquisition unit includes a first reference mark that is predetermined for the moving body before and after the rotation. A first image acquisition unit that acquires a first image captured by a first imaging unit that captures an image, and when the moving body is rotated around a specified coordinate value, the moving body is rotated before and after the rotation. A second image acquisition unit that acquires each of the second images captured by the second imaging unit that captures the second image including the second reference mark determined in advance, and the coordinate system correspondence holding unit includes: The correspondence relationship between the first imaging coordinate system of the first imaging unit and the second imaging coordinate system of the second imaging unit is held, and the designated coordinate value specifying unit specifies an arbitrary point and a second point specified based on the first reference mark A perpendicular that passes through the midpoint of the line connecting the arbitrary points specified based on the reference landmark Any point of the upper may be identified as the designated coordinate value.

  In the mobile body control system, by referring to the coordinate system correspondence, the first imaging of a first mark predetermined for an object held by the mobile body included in the first image captured by the first imaging unit. The first imaging coordinate system or the second imaging coordinate system of the second mark predetermined for the first position in the coordinate system or the second imaging coordinate system and the object included in the second image captured by the second imaging unit In the first imaging coordinate system or the second imaging coordinate system, a position specifying unit that specifies the second position in the first imaging coordinate system, a straight line connecting the first position and the second position in the second imaging coordinate system, and the first imaging coordinate system or the second imaging coordinate system are predetermined. Based on the angle formed by the target line, a moving mechanism control that rotates the moving body via a moving mechanism around an arbitrary point within or outside the imaging range to move the object to the target position It may be provided with a door.

  A moving body control system according to an embodiment of the present invention includes an imaging coordinate system predetermined for an imaging unit that captures an image including a reference mark predetermined for the moving body, and a moving body. A coordinate system correspondence holding unit for holding a coordinate system correspondence indicating a correspondence with a predetermined moving body coordinate system, and a coordinate value of an arbitrary point within or outside the imaging range in the imaging coordinate system. Specify the coordinate value in the moving object coordinate system corresponding to the specified coordinate value based on the coordinate system correspondence relationship and specify the coordinate value in the specified coordinate value, and rotate the moving object at any rotation angle around the specified coordinate value When the image is rotated and moved, the image acquisition unit that acquires the images captured by the imaging unit before and after the movement, and the rotation of the moving body based on the reference marks and the rotation angles included in the images, respectively. During ~ A rotation center position specifying unit that specifies the coordinate value in the imaging coordinate system for the position as the rotation center coordinate value, and a correspondence correction unit that corrects the coordinate system correspondence based on the rotation center coordinate value and the specified coordinate value, The image acquisition unit acquires the image including the reference mark at any time point before and after the movement when the moving body moves by combining the rotational movement and the parallel movement at an arbitrary rotation angle.

  In the moving body control system, when the moving body is rotated a plurality of times around an arbitrary point, the rotation angle in the imaging coordinate system specified based on each reference mark included in each image, and movement You may further provide the rotation angle corresponding | compatible relationship holding part which hold | maintains the rotation angle corresponding relationship which shows a corresponding relationship with the rotation angle in a body coordinate system.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure which shows the structural example of the mobile body control system which concerns on one Embodiment. It is a figure which shows an example of the 1st image and 2nd image containing a 1st mark and a 2nd mark. It is a figure which shows the functional block of a mobile body control system. It is a flowchart which shows an example of the procedure in case a calibration part acquires coordinate system correspondence. It is a flowchart which shows an example of the procedure which aligns so that an alignment part may move an object to a target position. It is a figure which shows the functional block of the mobile body control system which concerns on other embodiment.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows a configuration example of a mobile control system 100 according to an embodiment. The moving body control system 100 includes a stage 10, a moving mechanism 30, a first imaging unit 40-1, a second imaging unit 40-2, and a moving mechanism control device 50.

  The stage 10 functions as a moving body that holds the object 20, and the object 20 is installed on the stage 10. The moving mechanism 30 moves the stage 10 in the horizontal direction, that is, the XY direction in FIG. 1, and the rotation direction, that is, the θ direction that is rotation about an arbitrary rotation axis in the Z direction in FIG.

  The first imaging unit 40-1 captures an image including one corner 20 a of the object 20 as an image including a predetermined first mark for the object 20 on the stage 10. An image captured by the first imaging unit 40-1 may be referred to as a “first image”. In the present embodiment, the vertex 20b of the corner 20a indicates the first mark.

  The second imaging unit 40-2 captures an image including the other corner 20c of the object 20 as an image including a second mark that is predetermined for the object 20 on the stage 10. An image captured by the second imaging unit 40-2 may be referred to as a “second image”. In the present embodiment, the vertex 20d of the corner 20c indicates the second mark.

  The movement mechanism controller 50 moves the stage 10 via the movement mechanism 30 to move the object 20 to a predetermined target position based on the first mark in the first image and the second mark in the second image. Control the movement of.

  Further, in the moving body control system 100 according to the present embodiment, the first imaging coordinate system predetermined for the first imaging unit 40-1 and the first predetermined for the second imaging unit 40-2. (2) The coordinate system correspondence indicating the correspondence with the imaging coordinate system is acquired with high accuracy. In the moving body control system 100 according to the present embodiment, by referring to the coordinate system correspondence relationship, the position of the first mark included in the first image and the position of the second mark included in the second image are set to the first imaging coordinates. The coordinate values are acquired as coordinate values in either one of the system or the second imaging coordinate system. Furthermore, the moving body control system 100 sets the object 20 around a point in the first imaging range of the first imaging unit 40-1 or the second imaging range of the second imaging unit 40-2 based on the respective coordinate values. Is rotated to move the object 20 to the target position.

  In the present embodiment, an example will be described in which the object 20 is moved by moving the stage 10 in the horizontal direction or the rotation direction. However, the movement of the object 20 does not necessarily have to be the movement of the stage 10. For example, the object 20 may be moved by adsorbing the object 20 installed on the stage 10 by an adsorption device installed above the stage 10 and moving the adsorption device in the horizontal direction or the rotation direction. That is, in this case, the suction device functions as a moving body.

  FIG. 2 shows an example of the first image and the second image including the first mark and the second mark captured by the first image capturing unit 40-1 and the second image capturing unit 40-2. Reference numeral 200 represents a first image, and reference numeral 210 represents a second image. A straight line connecting the vertex 20b as the first mark and the vertex 20d as the second mark is indicated by reference numeral 220. Further, the target position of the first mark is indicated by reference numeral 204, and the target position of the second mark is indicated by reference numeral 214. Further, a target line to be described later is indicated by reference numeral 230. An angle formed by the straight line 220 and the target line 230 is indicated by reference numeral 222. In the moving body control system 100 according to the present embodiment, the moving body control system 100 controls the stage 10 in order to move the object 20 to the object 20 ′ at the target position indicated by the one-dot broken line.

  FIG. 3 shows functional blocks of the mobile control system 100. The first imaging unit 40-1 provides the first image to the calibration unit 60 and the alignment unit 70 via the first image acquisition unit 42-1. Similarly, the second imaging unit 40-2 provides the second image to the calibration unit 60 and the alignment unit 70 via the second image acquisition unit 42-2.

  The moving mechanism 30 includes a first X-direction moving mechanism 32 and a second X-direction moving mechanism 34 that translate the stage 10 in the X direction, and a Y-direction moving mechanism 36 that translates the stage 10 in the Y direction. The movement mechanism 30 is a so-called UVW stage control mechanism that can move in a second direction different from the first direction and the first direction. The first X-direction movement mechanism 32, the second X-direction movement mechanism 34, and the Y-direction movement mechanism 36. Are respectively controlled to perform parallel movement in the X direction and Y direction of the stage 10 and rotational movement in the θ direction. In the present embodiment, as described above, the moving mechanism 30 will be described using the UVW stage control mechanism as an example. However, the moving mechanism 30 is, for example, a parallel moving mechanism that translates the stage 10 in the X direction and the Y direction, and a rotational movement that can move in the X direction and the Y direction by the parallel moving mechanism and rotate the stage 10 in the θ direction. A so-called XYθ stage control mechanism having a mechanism may be used.

  The movement mechanism control device 50 includes a first image acquisition unit 42-1, a second image acquisition unit 42-2, a calibration unit 60, a coordinate system correspondence holding unit 66, an alignment unit 70, a target position holding unit 78, and a movement. A mechanism control unit 80 is provided.

  The first image acquisition unit 42-1 acquires the first image captured by the first imaging unit 40-1, and provides the acquired first image to the calibration unit 60 and the alignment unit 70. The second image acquisition unit 42-2 acquires the second image captured by the second imaging unit 40-2, and provides the acquired second image to the calibration unit 60 and the alignment unit 70.

  The calibration unit 60 performs calibration based on the movement trajectories of the first reference mark and the second reference mark determined in advance on the stage 10 included in the first image and the second image. That is, the calibration unit 60 may be referred to as a positional relationship between the stage 10 and the first imaging unit 40-1 and the second imaging unit 40-2, that is, an orthogonal coordinate system of the stage 10 (“stage coordinate system”). ) And a coordinate system correspondence relationship indicating a correspondence relationship between the first imaging coordinate system and the second imaging coordinate system, which are orthogonal coordinate systems in the imaging ranges of the first imaging unit 40-1 and the second imaging unit 40-2. To do. Further, the calibration unit 60 includes a first imaging coordinate system and a second imaging coordinate at the rotation center when the stage 10 is rotated around an arbitrary first point included in the imaging range of the first imaging unit 40-1. A coordinate system correspondence indicating the correspondence with the system is acquired. Note that the first reference mark and the second reference mark may be points given in advance to the stage 10.

  The coordinate system correspondence holding unit 66 holds the coordinate system correspondence acquired by the calibration unit 60.

  The alignment unit 70 includes a position specifying unit 72, a parallel movement amount specifying unit 74, and a rotation amount specifying unit 76, and is included in the first mark and the second image that are predetermined for the object 20 included in the first image. Alignment is performed in order to move the object 20 to the target position based on a predetermined second mark for the object 20 to be set.

  The position specifying unit 72 specifies the xy coordinate value in the first imaging coordinate system of the first mark based on the first image. The position specifying unit 72 specifies the xy coordinate value in the second imaging coordinate system of the second mark based on the second image, and the xy coordinate value in the first imaging coordinate system of the xy coordinate value is associated with the coordinate system. Identify by referring to the relationship. The parallel movement amount specifying unit 74 specifies the parallel movement amounts in the X direction and the Y direction of the stage 10 for moving the object 20 to the target position. The rotation amount specifying unit 76 specifies the rotation amount in the θ direction of the stage 10 for moving the object 20 to the target position.

  FIG. 4 is a flowchart illustrating an example of a procedure when the calibration unit 60 acquires the coordinate system correspondence in the calibration. The calibration unit 60 acquires the first coordinate system correspondence between the first imaging coordinate system and the stage coordinate system based on the first reference mark included in the first image. In addition, the calibration unit 60 includes a first reference including the provisional first rotation center position of the first imaging coordinate system when the stage 10 is rotated when the moving mechanism 30 is at the initial setting position. Obtained based on the landmark (S100).

  More specifically, the calibration unit 60 first instructs the movement mechanism control unit 80 to move the stage 10 to the initial setting position. After the stage 10 moves to the initial setting position, the calibration unit 60 moves the stage 10 by a predetermined parallel movement amount in the X direction and the Y direction, and each time the first reference mark included in the first image is displayed. An xy coordinate value in the first imaging coordinate system is acquired. The calibration unit 60 associates the XY coordinate value of the stage coordinate position with the initially set position acquired each time and the xy coordinate value in the first imaging coordinate system, and holds the coordinate system correspondence as the first coordinate system correspondence. Held in the portion 66. Similarly, the calibration unit 60 may acquire the second coordinate system correspondence between the second imaging coordinate system and the stage coordinate system based on the second reference mark included in the second image. The coordinate value of the reference mark means the attitude of the entire reference mark represented in the imaging coordinate system, and in some cases, the coordinate position of the feature point of the reference mark.

  Further, the calibration unit 60 performs the first imaging based on the trajectory of the first reference mark included in each first image acquired before and after rotating the stage 10 at the initial setting position. The coordinate value of the rotation center position of the stage 10 in the coordinate system is acquired as the temporary first rotation center position.

  More specifically, the calibration unit 60 calculates the xy coordinate values in the plurality of first imaging coordinate systems of the plurality of first reference landmarks in the plurality of first images acquired every time Δθ rotates and moves to the first xy coordinates. Get as a value. The calibration unit 60 identifies the virtual arc based on the least square method based on the plurality of first xy coordinate values. The calibration unit 60 acquires the first xy coordinate value at the rotation center of the first imaging coordinate system as a temporary first rotation center position based on the first xy coordinate value corresponding to the center of the specified virtual arc. Similarly, the calibration unit 60 acquires the second xy coordinate value at the rotation center of the second imaging coordinate system as the provisional second rotation center position based on the plurality of second images acquired every time Δθ rotates. May be.

  Further, the calibration unit 60 acquires the xy coordinate values of the first imaging coordinate system and the second imaging coordinate system at the rotation center as the temporary first rotation center position and the temporary second rotation center position, respectively, according to the following procedure. Also good.

  The calibration unit 60 acquires a first image including a first reference mark. The calibration unit 60 acquires the xy coordinate value in the first imaging coordinate system of the first reference mark in the first image before rotation as the first xy coordinate value before rotation. Next, in order to rotate the stage 10 in a range where the first reference mark is included in the first image, that is, a range where the first reference mark is included in the first imaging range of the first imaging unit 40-1, the stage 10 is rotated. The moving mechanism control unit 80 is instructed to rotate by Δθ. After the stage 10 rotates Δθ, the calibration unit 60 again acquires a first image including the first reference mark.

  Next, the calibration unit 60 acquires the xy coordinate value in the first imaging coordinate system of the first reference mark in the first image after rotation as the first xy coordinate value after rotation. After acquiring the respective first xy coordinate values before and after the rotation of Δθ, the calibration unit 60 sets the straight line composed of the respective first xy coordinate values before and after the rotation of Δθ to the bottom side and the movement mechanism control unit 80. On the other hand, an isosceles triangle having an apex angle Δθ corresponding to the rotation amount of the stage 10 instructed is acquired. The calibration unit 60 acquires the xy coordinate value in the first imaging coordinate system of the vertex at the apex angle Δθ of the acquired isosceles triangle as the provisional first rotation center position in the rotation center of the first imaging coordinate system.

  Similarly, the calibration unit 60 sets the xy coordinate value in the second imaging coordinate system of the second reference mark in the second image before the rotation of Δθ and the second imaging of the second reference mark in the second image after the rotation of Δθ. The xy coordinate values in the coordinate system may be acquired as the second xy coordinate value before rotation and the second xy coordinate value after rotation. Further, the calibration unit 60 acquires an isosceles triangle having a base connecting a straight line connecting the second xy coordinate value before the rotation and the second xy coordinate value after the rotation, and an apex angle of Δθ, and the top of the acquired isosceles triangle. The xy coordinate value of the vertex at the angle Δθ in the second imaging coordinate system may be acquired as the provisional second rotation center position at the rotation center of the second imaging coordinate system.

  Subsequently, the calibration unit 60 specifies an arbitrary first point included in the first imaging range of the first imaging unit 40-1 based on the first imaging coordinate system correspondence and the temporary first rotation center position, The movement mechanism control unit 80 is instructed to rotate the stage 10 around the first point (S102).

  More specifically, the calibration unit 60 selects an xy coordinate value in an arbitrary first imaging coordinate system included in the first imaging range as the first xy coordinate value of the first point. Next, the calibration unit 60 acquires a difference coordinate value between the selected first xy coordinate value and the first xy coordinate value of the temporary first rotation center position. Further, the calibration unit 60 instructs the movement mechanism control unit 80 to rotate the stage 10 with the position moved by the acquired difference xy coordinate value from the first xy coordinate value of the temporary first rotation center position as the rotation center position. Instruct.

  Next, the calibration unit 60 calculates the xy coordinate value of the first point in the first imaging coordinate system based on the first reference mark included in the first image before and after the rotation around the first point as the first rotation center. Obtained as a position (S104). As described above, the calibration unit 60 may acquire the first rotation center position by a virtual arc based on the least square method. In addition, the calibration unit 60 may acquire the first rotation center position by using an isosceles triangle whose base is the first reference mark before and after the rotation.

  The calibration unit 60 sets the xy coordinate value in the second imaging coordinate system of the first point as the second rotation center position based on the second reference mark included in the second image before and after the rotation around the first point. Obtain (S106).

  Next, the calibration unit 60 associates the first rotation center position and the second rotation center position specified in Step S104 and Step S106 and holds them in the coordinate system correspondence holding unit 66 as a coordinate system correspondence (S108). .

  In addition, the calibration unit 60 determines whether or not a predetermined reference number can be acquired from the coordinate system correspondence relationship in which the first rotation center position and the second rotation center position are associated (S110). If the first imaging unit 40-1 and the second imaging unit 40-2 have an accurate orthogonal coordinate system as the first imaging coordinate system and the second imaging coordinate system, the reference number may be two or more. . For example, when the first imaging unit 40-1 or the second imaging unit 40-2 does not have an accurate orthogonal coordinate system due to the influence of lens distortion, the reference number may be three or more.

  In the above description, the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system corresponding to each of the first points when rotating around a plurality of arbitrary first points within the first imaging range is shown. The example which acquires the coordinate system corresponding relationship shown was demonstrated. However, coordinates indicating a correspondence relationship between the first imaging coordinate system and the second imaging coordinate system corresponding to each second point when the plurality of second points within the second imaging range are further rotated around the second imaging point. System correspondence may be further acquired. In this way, by acquiring the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system when the point within the first imaging range and the point within the second imaging range are each rotated around the center, more A coordinate system correspondence that accurately indicates the correspondence between the first imaging coordinate system and the second imaging coordinate system can be acquired.

  In the present embodiment, the alignment unit 70 moves the object 20 to the target position using the coordinate system correspondence indicating the correspondence between the first imaging coordinate system and the second imaging coordinate system.

  FIG. 5 shows an example of a procedure for performing alignment so that the alignment unit 70 moves the object 20 to the target position. The moving mechanism control device 50 holds in advance the target first xy coordinate value in the first imaging coordinate system of the first mark corresponding to the target position of the object 20 and the target second xy coordinate value in the second imaging coordinate system of the second mark. doing. For example, the moving mechanism control device 50 sets the first mark of the first mark on the basis of the first image and the second image that are accurately set in advance at the target position on the stage 10 and captured in that state. The target first xy coordinate value in the imaging coordinate system and the target second xy coordinate value in the second imaging coordinate system of the second mark are acquired in advance and held.

  When alignment is performed, the object 20 is first installed on the stage 10. Next, the alignment unit 70 acquires a first mark that is predetermined for the object 20 captured by the first imaging unit 40-1, that is, a first image including the corner 20a. The alignment unit 70 specifies the xy coordinate value in the first imaging coordinate system of the vertex 20b of the corner 20a as the first position based on the first image (S200). In addition, the alignment unit 70 acquires a second mark that is predetermined for the object 20 captured by the second imaging unit 40-2, that is, a second image including the corner 20c. The alignment unit 70 specifies the xy coordinate value in the second imaging coordinate system of the vertex 20d of the corner 20c based on the second image (S202). Furthermore, the alignment unit 70 refers to the coordinate system correspondence relationship indicating the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system, and thereby corresponds to the specified xy coordinate value in the second imaging coordinate system. The xy coordinate value in one imaging coordinate system is specified as the second position (S204).

  Subsequently, the alignment unit 70 acquires the difference between the target first xy coordinate value acquired in advance and the xy coordinate value corresponding to the first position specified in step 200, and translates the difference between the stages 10. Specify as quantity.

  Next, the alignment unit 70 performs first imaging corresponding to the straight line connecting the first position specified in step 200 and the second position specified in step 204, the target first xy coordinate value, and the target second xy coordinate value. An angle formed by the target line connecting the xy coordinate values of the coordinate system is acquired (S208). The alignment unit 70 specifies the acquired angle as the rotation amount of the stage (S210).

  After specifying the parallel movement amount and the rotation amount, the alignment unit 70 first instructs the movement mechanism control unit 80 to move the stage 10 in the X direction and the Y direction by the specified parallel movement amount. After the movement, the alignment unit 70 instructs the moving mechanism control unit 80 to rotate the stage 10 in the θ direction by the specified rotation amount around the first mark, that is, around the target first xy coordinate value. . As described above, the alignment unit 70 can move the stage 10 to the target position.

  In the above description, the example in which the alignment unit 70 moves the stage 10 to the target position by rotating the stage 10 around the target first xy coordinate value after moving the stage 10 in parallel has been described. However, the alignment unit 70 may move the stage 10 to the target position as follows.

  That is, the alignment unit 70 performs the first imaging corresponding to the straight line connecting the first position specified in step 200 and the second position specified in step 204, the target first xy coordinate value, and the target second xy coordinate value. An angle formed by a target line connecting the xy coordinate values of the coordinate system is acquired as a rotation amount. Next, the alignment unit 70 moves the XY coordinate value of the first imaging coordinate system of the first mark and the second mark when the movement is performed by the acquired rotation amount around an arbitrary point in the first imaging range. The xy coordinate value of the first imaging coordinate system is acquired. The alignment unit 70 acquires a difference xy coordinate value between the xy coordinate value of the first mark and the target first xy coordinate value in the acquired first imaging coordinate system. In addition, the alignment unit 70 acquires a difference xy coordinate value between the xy coordinate value of the second mark in the acquired first imaging coordinate system and the xy coordinate value of the first imaging coordinate system corresponding to the target second xy coordinate value. To do. The alignment unit 70 specifies the parallel movement amounts in the X direction and the Y direction for moving the stage 10 to the target position by taking the average value of the acquired difference xy coordinate values. Next, the alignment unit 70 moves the stage 10 by the acquired rotation amount. After the rotation, the alignment unit 70 moves the object 20 to the target position by moving the stage 10 by the further specified parallel movement amount.

  In the above description, the alignment unit 70 refers to the coordinate system correspondence relationship, and converts the xy coordinate value in the second imaging coordinate system into the xy coordinate value in the first imaging coordinate system, so that the object 20 is moved to the target position. An example in which the amount of parallel movement and the amount of rotation to be moved is specified has been described. However, the alignment unit 70 moves the object 20 to the target position by referring to the coordinate system correspondence and converting the xy coordinate value in the first imaging coordinate system into the xy coordinate value in the second imaging coordinate system. The amount of translation and the amount of rotation may be specified.

  As described above, the coordinate system correspondence indicating the correspondence between the stage coordinate system, the first imaging coordinate system, and the second imaging coordinate system by the method of rotating the stage 10 around an arbitrary point within the imaging range of the imaging unit. Can be specified with high accuracy. However, when higher accuracy is required, the following processing can be further performed for the purpose of improving the pitch accuracy between the plurality of imaging units.

  After the calibration with a certain degree of accuracy is completed by the above-described method, the calibration unit 60 moves each movement in the imaging coordinate system of each reference mark predetermined for the moving body when the moving body is translated. An actual measurement value of the distance is calculated, and a correction coefficient is calculated based on the ratio of the calculated actual measurement values. Furthermore, the calibration unit 60 specifies parallel movement amounts in the X direction and the Y direction of the stage 10 for moving the object 20 to the target position based on the correction coefficient. Here, the measured value of the moving distance is obtained by grasping the correspondence between the distance in the stage coordinate system (expressed in millimeters, for example) and the distance in each imaging coordinate system (expressed in pixels) with a certain accuracy or higher. It is assumed that the movement amount of the reference mark actually measured in each imaging coordinate system is assumed.

  Specifically, the calibration unit 60 includes the first reference mark within the first image captured by the first imaging unit 40-1 before and after the movement, with respect to the movement mechanism control unit 80, and the second Instructed to move the stage 10 in parallel in one direction (for example, the X direction of the stage coordinate system) by a predetermined distance within a range in which the second reference mark is included in the second image captured by the imaging unit 40-2. To do. The calibration unit 60 is specified based on the measured value of the moving distance of the first reference mark specified based on the first image and the second image when the stage 10 is translated by a predetermined distance. The correction coefficient is specified based on the ratio of the movement distance of the second reference mark to the actually measured value. Here, the calibration unit 60 uses the measured value of the movement distance of the first reference mark in the first imaging coordinate system as the reference measured distance, and the measured value of the movement distance of the second reference mark in the second imaging coordinate system as the reference measured value. Specify a factor to normalize for distance.

  For example, the movement of the stage 10 in the X direction in the stage coordinate system is 1.000 mm, the actual measurement value of the movement distance in the first imaging coordinate system is 1.000 mm, and the actual measurement value of the movement distance in the second imaging coordinate system is somehow If it is 1.001 mm due to the factor, the alignment unit 70 sets the correction coefficient related to the second imaging coordinate system to 1.001 (1.001 / 1.000), and obtains a value obtained by dividing the parallel movement amount by this coefficient. This is replaced with the corrected parallel movement amount related to the second imaging coordinate system. By this process, the measurement error of the parallel movement amount calculated by each of the plurality of imaging coordinate systems can be reduced.

  As described above, when improving the pitch accuracy between the plurality of imaging units, it is sufficient to measure the movement distance at least once (the number of imaging times is two before and after parallel movement for each imaging unit). However, the calibration unit 60 may change the starting point and end point of the parallel movement, measure the measured value of the moving distance twice or more, and specify the correction coefficient based on the average value of the measured values.

  In the above description, the first reference mark and the second reference mark on the stage 10 are used. However, the first mark and the second mark that are set in advance for the object arranged on the stage 10 are used as the first mark. It may be used as the first reference mark and the second reference mark.

  As described above, according to the present embodiment, the coordinate system correspondence relationship indicating the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system can be accurately acquired in advance in the calibration. Therefore, for example, in a moving body control system that controls a moving body based on images captured by a plurality of imaging units, the accuracy of the correspondence relationship of each imaging coordinate system determined for each imaging unit is poor. Thus, it is possible to prevent the movement of the object to the target position from taking time. For example, conventionally, in order to accurately maintain the accuracy of the correspondence between the image capturing units, it has been necessary to accurately install the image capturing units at predetermined positions. However, according to this embodiment, after installing each imaging unit, it is possible to systematically obtain the correspondence between the imaging units based on the images captured by the imaging units. Therefore, it is not necessary to accurately install each imaging unit at a predetermined position. Further, even if the installation position of any of the image capturing units is deviated, by performing the calibration as described above, it is possible to accurately acquire the correspondence relationship between the respective image capturing units.

  Further, for example, the radius of the arc drawn based on the trajectory of the first reference mark by the rotation around the point in the first imaging range is the trajectory of the first reference mark by the rotation around the point outside the first imaging range. Is smaller than the radius of the arc drawn based on Therefore, the first rotation center position specified based on the arc of rotation around the point in the first imaging range is specified based on the arc of rotation around the point outside the first imaging range. Less error than the center of rotation. Similarly, when the stage 10 is rotated around a point in the first imaging range, the lengths of the isosceles sides of the acquired isosceles triangle are relatively short and the apex angle is relatively large. On the other hand, when the stage 10 is rotated around a point outside the first imaging range, the lengths of the isosceles sides of the acquired isosceles triangle are relatively long and the apex angle is relatively small. Therefore, the error included in the first rotation center position acquired based on the isosceles triangle is rotated around the outside of the first imaging range when rotated around a point within the first imaging range. It is likely that the case is smaller.

  Therefore, as described above, for example, when the stage 10 is moved to the target position by rotating the stage 10 around the point in the first imaging range, the first xy coordinate value in the first imaging coordinate system of the rotation center is , Identified relatively accurately. Therefore, the parallel movement amount and rotation amount of the stage 10 necessary for moving the object 20 to the target position are specified relatively accurately. Therefore, it can be prevented that it takes time to move the object to the target position.

  Furthermore, conventionally, the correspondence relationship between the first imaging coordinate system and the stage coordinate system and the correspondence relationship between the second imaging coordinate system and the stage coordinate system have been individually maintained. However, when the correspondence between the stage coordinate system and the first imaging coordinate system and the correspondence between the stage coordinate system and the second imaging coordinate system each include an error, the first acquired via the stage coordinate system. The correspondence relationship between the first imaging coordinate system and the second imaging coordinate system includes many errors. Therefore, when the alignment unit 70 moves the object 20 to the target position based on the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system acquired via the stage coordinate system, the alignment unit 70 moves to the target position of the object 20. It takes time to move.

  On the other hand, in the present embodiment, the calibration unit 60 includes the first imaging coordinate system and the first imaging coordinate system corresponding to each first point when the stage 10 is rotated around a plurality of first points within the first imaging range. 2 Acquire a coordinate system correspondence indicating the correspondence with the imaging coordinate system. That is, the correspondence between the first imaging coordinate system and the second imaging coordinate system is acquired without going through the stage coordinate system. Therefore, according to this embodiment, compared with the case where the stage 10 is controlled based on the correspondence between the first imaging coordinate system and the second imaging coordinate system acquired via the stage coordinate system, Time required for movement to the target position can be shortened.

  In the above-described embodiment, the method for improving the calibration accuracy regarding an arbitrary point in the imaging range has been described. However, when only the method is used, the correspondence between the imaging coordinate system and the stage coordinate system regarding an arbitrary point specified outside the imaging range based on the imaging coordinate system is not considered. In the following, assuming a system configuration using a single imaging unit unless otherwise noted, calibration is intended to accurately grasp the correspondence between the imaging coordinate system and the stage coordinate system at an arbitrary point not limited within the imaging range. Is described (sometimes referred to as “spatial calibration”).

  FIG. 6 shows functional blocks of a mobile control system according to another embodiment. 6 is different from the functional block of the mobile control system shown in FIG. 3 in that the calibration unit 60 includes a designated coordinate value specifying unit 65 and a correspondence correction unit 67. The designated coordinate value specifying unit 65 is an arbitrary first imaging coordinate system within or outside the imaging range of the first imaging unit 40-1 as the designated coordinate value designated as the rotation center position when the stage 10 is rotated. Specifies the coordinate value at. The correspondence correction unit 67 rotates the stage 10 around the position corresponding to the designated coordinate value, thereby obtaining the coordinate system correspondence indicating the correspondence between the stage coordinate system and the first imaging coordinate system according to the procedure described later. to correct.

  The calibration unit 60 acquires in advance the coordinate system correspondence between the first imaging coordinate system and the stage coordinate system according to the procedure described in step S100 of FIG. The calibration unit 60 sets the posture of the stage 10 at the start of the spatial calibration process as the first posture. When the stage 10 is in the first posture, the first reference mark is present in the first image.

  Next, in the designated coordinate value specifying unit 65, the calibration unit 60 uses an arbitrary point in the first imaging coordinate system that is not limited to the first imaging range of the first imaging unit 40-1 as the designated coordinate value (Px, Py). Identify. The calibration unit 60 refers to the coordinate system correspondence and specifies the coordinate value (px, py) on the stage coordinate system corresponding to the designated coordinate value (Px, Py). The calibration unit 60 moves around the specified coordinate value (px, py) so as to rotate the stage 10 at an arbitrary rotation angle within a range where the first reference mark is within the first image before and after the rotational movement. It instructs the mechanism control unit 80. The posture of the stage 10 after rotating and moving in this way is set as the second posture.

  The calibration unit 60 specifies the positions of the first reference marks in the first posture and the second posture as coordinate values in the first imaging coordinate system. Further, the calibration unit 60 acquires the rotation center position calculated based on these coordinate values and the rotation angle as coordinate values (Qx, Qy) in the first imaging coordinate system. Alternatively, the calibration unit 60 places the stage 10 in three or more postures by the same rotational movement, and specifies a virtual arc based on the least square method based on three or more coordinate values of the first reference mark in the first image. Then, the coordinate value corresponding to the center of the specified virtual arc may be acquired as the coordinate value (Qx, Qy).

  When complete calibration is realized within the imaging range and outside the imaging range, the coordinate values (Px, Py) and the coordinate values (Qx, Qy) match. However, a mismatch usually occurs. Therefore, the calibration unit 60 specifies the corrected coordinate value (Rx, Ry) as the corrected specified coordinate value based on the coordinate value (Px, Py) and the coordinate value (Qx, Qy). In the present embodiment, the calibration unit 60 determines the coordinate value (Qx, Qy) as the corrected coordinate value (Rx, Ry) as it is, assuming that the reliability of the coordinate value (Qx, Qy) is high. Next, the calibration unit 60 corrects the coordinate system correspondence by associating the specified coordinate value (px, py) with the specified coordinate value after correction (Rx, Ry) in the correspondence correction unit 67.

  Alternatively, the calibration unit 60 assigns arbitrary weights to the coordinate values (Px, Py) and the coordinate values (Qx, Qy), or weighted coordinate values (Px, Py) and coordinate values (Qx , Qy), and the corrected coordinate value may be specified.

  Further, alternatively, when instructing the movement mechanism control unit 80 to rotate the stage 10 at an arbitrary rotation angle, the calibration unit 60 includes the first reference mark in the first image before and after the rotational movement. The coordinate value of the first reference mark in a plurality of postures may be acquired without limiting. However, when the first reference mark moves out of the imaging range, the reference mark cannot be picked up by the imaging unit. Therefore, the calibration unit 60 moves the stage 10 in order to keep the first reference mark within the imaging range. Move in parallel after rotational movement.

  The specific procedure is as follows. First, the calibration unit 60 specifies designated coordinate values (Px, Py) before correction. The calibration unit 60 pays attention to the coordinate values (M1x, M1y) of the first reference mark in the imaging coordinate system existing in the imaging range, and an arbitrary rotation angle around the designated coordinate values (Px, Py) before correction. The stage 10 is rotated. In this case, the first reference mark may once move out of the imaging range. Next, the calibration unit 60 uses the specified coordinate value (Px, Py) before correction and the above-mentioned arbitrary rotation angle as the coordinate position (M2x, M2y) as the position of the calculation destination of the first reference mark. Identify. Further, the calibration unit 60 calculates a difference value (M2x−M1x, M2y−M1y) in the xy direction between the coordinate values (M2x, M2y) and the coordinate values (M1x, M1y). Next, the calibration unit 60 translates the stage 10 by an amount corresponding to the difference value (M2x−M1x, M2y−M1y) in order to bring the first reference mark closer to the original position. It will be apparent to those skilled in the art that the rotational and translational movements described above can actually be performed as a single combined motion.

  Here, when complete calibration is realized inside and outside the imaging range, the coordinate values (M3x, M3y) in the first imaging coordinate system of the first reference mark after the parallel movement specified from the first image are It coincides with the coordinate values (M1x, M1y). However, a mismatch usually occurs. The cause of this mismatch is highly likely due to the fact that the designation accuracy of the designated coordinate values (Px, Py) before correction is not sufficient. That is, the shift amount of the coordinate values (M3x, M3y) with respect to the coordinate values (M1x, M1y) represents the shift amount from the coordinate value closer to the true rotation center position with respect to the designated coordinate values (Px, Py) before correction. There is a high possibility. Therefore, the calibration unit 60 adds the calculated difference value in the xy direction calculated above to the coordinate values (M2x, M2y), so that the stage 10 is in the posture after the rotation and before the parallel movement. In this case, the position of the first reference mark is acquired as coordinate values (M4x, M4y).

  The coordinate values (M1x, M1y) and coordinate values (M4x, M4y) obtained in this way are described above with respect to acquiring the rotation center position as the coordinate values (Qx, Qy) in the first imaging coordinate system. Can be replaced with “the position of the first reference mark in the first posture and the second posture”. With the same procedure, the position of the first reference mark corresponding to a plurality of rotational movements can be acquired. The subsequent procedure for specifying the corrected designated coordinate values (Rx, Ry) is as described above.

  Here, in order to bring the first reference mark closer to the original position, the step of moving the stage 10 in parallel moves the rotation angle relatively large and places the first reference mark within the imaging range. The purpose is to specify the corrected specified coordinate values (Rx, Ry) based on the virtual arc indicated by the movement locus of the mark. Therefore, the parallel movement amount may be determined based on the imaging range, and may not be the difference value (M2x−M1x, M2y−M1y) as in the above example. In addition, the calibration method including the parallel movement step described above improves the designation accuracy when an arbitrary point is designated in the image processing apparatus via the imaging unit. Therefore, it can be combined with various calibration methods described in this specification, or a part thereof can be replaced, or can be implemented alone, and each of such embodiments is included in the concept of the present invention.

  Similarly, the calibration unit 60 uses the designated coordinate value specifying unit 65 to set linear coordinate values arranged on a predetermined linear in the imaging coordinate system, for example, coordinate values corresponding to any of predetermined grid points. A plurality of designated coordinate values respectively corresponding to are specified, and a corrected designated coordinate value is specified for each designated coordinate value. Further, the calibration unit 60 further corrects the specified post-correction designated coordinate value to correspond to the linear coordinate value. That is, the calibration unit 60 further corrects the corrected specified coordinate value so that the corrected specified coordinate value is arranged on a predetermined linearity. The calibration unit 60 corrects the coordinate system correspondence to associate the corrected specified coordinate value after correction with the coordinate value of the stage coordinate system. Thereby, the space grasping accuracy in a wide range that is not limited within the imaging range can be improved.

  The distribution range and the number of specified coordinate values after correction specified by the execution of the spatial calibration can be arbitrarily determined according to the actual application. However, as the number of measurements in a wide range is increased, the improvement of the space grasp accuracy can be expected, and non-measurement points can be interpolated with high accuracy. When the array of each point of the correction designated coordinate value is non-linear, the distortion can be corrected linearly using a mathematical model of polynomial transformation. The number of coefficients of the polynomial transformation is represented by (n + 1) × (n + 2). Therefore, when a cubic polynomial is designated, the number of coefficients defining the transformation is 20. In this case, 20 or more specified coordinate values after correction are acquired. The calibration unit 60 may correct the coordinate system correspondence by associating 20 or more post-correction designated coordinate values in which the arrangement of each point is linearly corrected with the coordinate values of the stage coordinate system.

  As an advantage of improving the calibration accuracy outside the imaging range, it is possible to predict the posture of the stage 10 after the rotational movement with high accuracy even when any arbitrary position outside or inside the imaging range is specified as the rotation center during alignment. . Another advantage is that the position of an object existing outside the imaging range can be accurately identified based on information included in the image within the imaging range. For example, when a part of an object is visible in the imaging range, it is possible to measure at which position outside the imaging range the other part whose relative positional relationship with the part is known is present. . In other words, by being able to “see an invisible place”, there is a possibility that the required number of imaging units can be reduced even if the number of automated work steps for a single object increases.

  Further, in a system using a so-called UVW stage control mechanism or a mechanism similar thereto, there is a limitation due to a mechanical movement limit. However, the moving amount of each driving unit of the moving mechanism including a plurality of ball screws and a motor for moving the object from the installation position to the target position can have many combinations. In such a system, the center of rotation can be selected strategically according to the desired criteria. For example, each time an object is replaced, it is possible to move the object along a movement path corresponding to a combination of movement amounts that minimizes the cumulative wear degree of the drive unit of the movement mechanism or distributes the load. Become. Even in that case, the control accuracy is maintained. Even when the stage 10 is not returned to the basic posture every time the object is replaced in the production process, the control accuracy is maintained.

  When a system configuration using two imaging units is employed, the first image included in the first image is referred to by referring to the coordinate system correspondence relationship indicating the correspondence relationship between the first imaging coordinate system and the second imaging coordinate system. The coordinate values (Px, Py), the coordinate values (Qx, Qy), and the coordinate values (Rx, Ry) can be specified. Specifically, in the step of obtaining the coordinate values (Qx, Qy) described above, the calibration unit 60 moves the rotation center position (Q1x, Q1y) and the second reference mark based on the movement locus of the first reference mark. The rotation center position (Q2x, Q2y) based on the locus can be calculated, respectively, and the coordinate value (Qx, Qy) can be specified based on these coordinate values. For example, an average value of the coordinate values (Q1x, Q1y) and the coordinate values (Q2x, Q2y) can be used as the coordinate values (Qx, Qy).

  In the step of specifying an arbitrary point that is not limited within the imaging range as the designated coordinate values (Px, Py) before correction, the arbitrary point and the second image specified based on the first reference mark included in the first image An arbitrary point on a perpendicular line that passes through the midpoint of a straight line that connects to an arbitrary point specified based on the second reference mark included in may be specified as designated coordinate values (Px, Py) before correction. Thereby, the coordinate values (Qx, Qy) can be specified with higher accuracy using the two imaging ranges. As described above, the calculation accuracy of the rotation center is caused by the radius and the rotation angle of the arc. Furthermore, the same process may be repeated by moving the first reference mark and the second reference mark to different positions by moving the stage 10 with respect to the imaging unit. Thereby, many arbitrary points on said perpendicular can be obtained. That is, by using two imaging ranges, a large number of designated coordinate values after correction can be specified.

  The prior art has focused on improving the measurement accuracy of a specific rotation center point (generally, the control center in the basic attitude of the stage, and in the so-called XYθ stage control mechanism, the intersection of the X axis and the Y axis). On the other hand, in this embodiment, an image processing apparatus that processes an image captured by the imaging unit via the imaging unit positively gives a control position, and freely controls the moving body based on the control position. In other words, this embodiment is intended to provide flexibility in the control strategy.

  As described above, the stage 10 can be accurately controlled by specifying the corrected designated coordinate value. However, the correspondence between the rotation angle instructed to the stage 10 and the actual rotation angle has not been verified yet. Therefore, even if the stage 10 rotates around the control position that accurately corresponds to the designated coordinate value, there may be a difference between the rotation angle in the imaging coordinate system and the actual rotation angle of the stage 10. Such a difference becomes an obstacle to the final purpose of the system that the present embodiment of moving the object to a desired position is expected to be applied. Therefore, a calibration method (hereinafter sometimes referred to as “rotation angle calibration”) that aims to obtain accurate rotational movement according to the designated rotation angle will be described below.

  In starting the rotation angle calibration, the calibration unit 60 first instructs the moving mechanism control unit 80 to move the stage 10 to the initial setting position. After the stage 10 has moved to the initial setting position, the calibration unit 60 moves the stage 10 by an arbitrary angle in the θ direction, and each time acquires the xy coordinate value in the imaging coordinate system of the reference mark included in the image. The calibration unit 60 rotates the xy coordinate values of the stage in the stage coordinate system and the imaging coordinate system with respect to the initial setting position acquired each time, the rotation angle in the stage coordinate system, and the rotation angle in the imaging coordinate system, The rotation angle correspondence holding unit 69 holds the angle correspondence. The rotation angle in the stage coordinate system is a rotation angle recognized by the movement mechanism control unit 80. The rotation angle in the imaging coordinate system is based on the principle of the trigonometric function from the coordinate values of the two vertices constituting the base angle of the isosceles triangle virtually drawn by the rotation center point and the reference mark before and after the movement for each rotation movement. Can be calculated. In addition, the arbitrary rotation angles of each rotational movement do not need to be equal angles. The alignment unit 70 refers to the rotation angle correspondence relationship and specifies the rotation angle of the stage 10 for moving the stage 10 to the target position. That is, the alignment unit 70 specifies the rotation angle in the stage coordinate system corresponding to the rotation angle specified in the imaging coordinate system with reference to the rotation angle correspondence relationship.

  As the number of measurement points is increased by changing the rotation center position and the rotation angle, the accuracy of the rotation angle correspondence relationship is improved. For angles that are not measured, interpolation can be performed by any method. When angle calibration is performed with a large rotation angle with respect to the imaging range, the rotational movement and the parallel movement can be combined by the same method as described above as the process for placing the reference mark within the imaging range. Constraint conditions based on the imaging range can be avoided.

  In the present embodiment, the calibration method of the imaging coordinate system of the two imaging devices is mainly shown. However, the imaging coordinate systems of three or more imaging devices may be calibrated with each other in a multiplexed manner. For example, when three imaging devices A, B, and C are used, the configuration of this embodiment is applied only between AB and BC by applying between the imaging devices AB, BC, and AC. Rather, the accuracy of the entire system can be improved. Furthermore, according to such mutual and multiple calibration, it is as if a wide range of work areas that could not be handled by the prior art are combined by combining a plurality of generally sold imaging devices that are generally sold. It can be handled as if it were picked up by a single giant imaging device.

  The moving mechanism control device 50 according to the present embodiment may be configured by installing a program for performing various processes related to the calibration and alignment described above and causing the computer to execute the program. That is, by causing a computer to execute a program for performing various processes related to calibration and alignment, the first image acquisition unit 42-1, the second image acquisition unit 42-2, the calibration unit 60, and the coordinate system correspondence holding unit 66, the moving mechanism control device 50 may be configured by causing a computer to function as the alignment unit 70, the target position holding unit 78, and the moving mechanism control unit 80.

  The computer has various memories such as a CPU, ROM, RAM, and EEPROM (registered trademark), a communication bus, and an interface. The CPU reads and sequentially executes a processing program stored in the ROM as firmware, thereby controlling the moving mechanism. It functions as the device 50.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Stage 20 Object 30 Moving mechanism 32 1st X direction moving mechanism 34 2nd X direction moving mechanism 36 Y direction moving mechanism 40-1 1st imaging part 40-2 2nd imaging part 42-1 1st image acquisition part 42-2 1st 2 image acquisition unit 50 moving mechanism control device 60 calibration unit 66 coordinate system correspondence holding unit 70 alignment unit 72 position specifying unit 74 parallel movement amount specifying unit 76 rotation amount specifying unit 78 target position holding unit 80 moving mechanism control unit 100 movement Body control system

Claims (20)

When the moving body is rotated around an arbitrary first point in the first imaging range of the first imaging unit that captures a first image including a first reference mark that is predetermined for the moving body, A first image acquisition unit that acquires the first images captured by the first imaging unit before and after rotation;
When the movable body is rotated around the first point, the second imaging unit captures a second image including a second reference mark that is predetermined for the movable body before and after the rotation. A second image acquisition unit for acquiring each of the second images thus obtained;
A first rotation center position corresponding to the first point in the first imaging coordinate system predetermined for the first imaging unit based on each first reference mark included in each first image. A first rotation center position specifying unit for specifying
A second rotation center position corresponding to the first point in the second imaging coordinate system predetermined for the second imaging unit based on each second reference mark included in each second image. A second rotation center position specifying unit for specifying
A coordinate system correspondence relationship in which the first rotation center position and the second rotation center position are associated with each other and held as a coordinate system correspondence relationship indicating a correspondence relationship between the first imaging coordinate system and the second imaging coordinate system. A holding part;
A moving body control system comprising: The first image acquisition unit, when rotating the moving body sequentially around each of a plurality of arbitrary first points within the first imaging range, the first imaging before and after each rotation. Each of the first images captured by the unit,
The second image acquisition unit, when rotating the moving body sequentially around each of a plurality of arbitrary first points within the first imaging range, the second imaging before and after each rotation. Each of the second images captured by the unit,
The first rotation center position specifying unit is configured to set each first point corresponding to each first point based on each first reference mark included in each first image before and after rotation. Identify the center of rotation,
The second rotation center position specifying unit is configured to respectively correspond to the second points corresponding to the first points based on the second reference marks included in the second images before and after the rotation. Identify the center of rotation,
The moving body control system according to claim 1, wherein the coordinate system correspondence holding unit holds the coordinate system correspondence based on the first rotation center position and the second rotation center position. When the movable body is rotated around an arbitrary second point within the second imaging range of the second imaging unit, the first image acquisition unit is configured to rotate by the first imaging unit before and after the rotation. Acquiring each of the captured first images;
The second image acquisition unit acquires the second images captured by the second imaging unit before and after the rotation when the movable body is rotated around the second point,
The first rotation center position specifying unit corresponds to the second point in the first imaging coordinate system based on the first reference marks included in the first images before and after the rotation. Specifying the first rotation center position;
The second rotation center position specifying unit corresponds to the second point in the second imaging coordinate system based on the second reference marks included in the second images before and after the rotation. Specifying the second rotation center position;
The moving body control according to claim 1, wherein the coordinate system correspondence holding unit holds the coordinate system correspondence based on the first rotation center position and the second rotation center position corresponding to the second point. system. The first image acquisition unit, when rotating the moving body sequentially around each of a plurality of arbitrary second points within the second imaging range, the first imaging before and after each rotation. Each of the first images captured by the unit,
The second image acquisition unit is configured to rotate the movable body sequentially around each of a plurality of arbitrary second points within the second imaging range, and to rotate the second imaging before and after each rotation. Each of the second images captured by the unit,
The first rotation center position specifying unit is configured to display the first points corresponding to the second points based on the first reference marks included in the first images before and after the rotation, respectively. Identify the center of rotation,
The second rotation center position specifying unit is configured to correspond to the second points corresponding to the second points based on the second reference marks included in the second images before and after the rotation, respectively. Identify the center of rotation,
The movement according to claim 3, wherein the coordinate system correspondence holding unit holds the coordinate system correspondence based on the first rotation center position and the second rotation center position corresponding to each of the second points. Body control system. By referring to the coordinate system correspondence, the first imaging coordinate system of the first mark predetermined for the object held by the moving body included in the first image captured by the first imaging unit or The first imaging coordinate system or the second imaging of a second mark predetermined for the first position in the second imaging coordinate system and the object included in the second image captured by the second imaging unit. A position specifying unit for specifying the second position in the coordinate system;
A straight line connecting the first position and the second position in the first imaging coordinate system or the second imaging coordinate system, and a target line predetermined in the first imaging coordinate system or the second imaging coordinate system. Based on an angle formed, a moving mechanism around a predetermined point in the first imaging range of the first imaging unit or the second imaging range of the second imaging unit to move the object to a target position The moving body control system according to any one of claims 1 to 4, further comprising: a moving mechanism control unit that rotates the moving body via an axis. The said movement mechanism control part moves the said object to a target position by rotating the said moving body via the said movement mechanism centering | focusing on the point defined based on the said 1st mark or the said 2nd mark. 5. The moving body control system according to 5. The moving mechanism includes two first direction moving mechanisms that translate the moving body in a predetermined first direction, and a second direction that translates the moving body in a second direction different from the first direction. A moving mechanism,
The said moving mechanism control part is a moving body control system of Claim 5 or Claim 6 which rotates the said moving body by controlling the said two 1st direction moving mechanisms and the said 2nd direction moving mechanism, respectively. When the movable body is translated by an arbitrary distance, the measured value of the movement distance of the first reference mark before and after the parallel movement in the first imaging coordinate system and the second reference mark A ratio acquisition unit for acquiring a ratio of measured values of the movement distance in the two imaging coordinate system;
A correction coefficient specifying unit for specifying a correction coefficient based on the ratio of the actual measurement values;
8. The apparatus according to claim 5, further comprising: a parallel movement amount specifying unit that specifies a parallel movement amount of the moving body based on the correction coefficient when the object is moved to the target position. The moving body control system described. When the movable body is rotated a plurality of times around an arbitrary point, the moving object is specified based on the first reference mark or the second reference mark included in the first image or the second image. Rotation holding a rotation angle correspondence indicating a correspondence between a rotation angle in the first imaging coordinate system or the second imaging coordinate system and a rotation angle in a predetermined moving body coordinate system with respect to the moving body. The mobile body control system according to any one of claims 1 to 4, further comprising an angle correspondence holding unit.   A program for causing a computer to function as the mobile control system according to any one of claims 1 to 9. Rotating when the moving body is rotated around an arbitrary first point within the first imaging range of the first imaging unit that captures a first image including a first reference mark predetermined for the moving body. A first image acquisition stage for acquiring the first images captured by the first imaging unit before and after rotation, respectively;
When the movable body is rotated around the first point, the second imaging unit captures a second image including a second reference mark that is predetermined for the movable body before and after the rotation. A second image acquisition step of acquiring each of the second images performed;
A first rotation center position corresponding to the first point in the first imaging coordinate system predetermined for the first imaging unit based on each first reference mark included in each first image. A first rotation center position specifying step for specifying
A second rotation center position corresponding to the first point in the second imaging coordinate system predetermined for the second imaging unit based on each second reference mark included in each second image. A second rotation center position specifying step for specifying
Coordinate system correspondence relationship holding for holding the coordinate system correspondence relationship indicating a correspondence relationship between the first rotation center position and in correspondence with said second rotational center position of the first imaging coordinate system and the second imaging coordinate system Stages,
A moving body control method comprising: Correspondence between an imaging coordinate system predetermined for an imaging unit that captures an image including a reference mark predetermined for a moving object and a predetermined moving object coordinate system for the moving object A coordinate system correspondence holding unit that holds the coordinate system correspondence shown,
A designated coordinate value identifying unit that identifies a coordinate value of an arbitrary point within or outside the imaging range in the imaging coordinate system as a designated coordinate value;
When the coordinate value in the moving object coordinate system corresponding to the specified coordinate value is specified based on the coordinate system correspondence, and the moving object is rotated at an arbitrary rotation angle around the specified coordinate value In addition, an image acquisition unit that acquires the images captured by the imaging unit before and after the movement, and
A rotation center position specifying unit that specifies, as a rotation center coordinate value, a coordinate value in the imaging coordinate system with respect to the rotation center position of the moving body, based on the reference mark and the rotation angle included in each of the images;
A correspondence correction unit that corrects the coordinate system correspondence based on the coordinate value in the moving body coordinate system and the rotation center coordinate value;
Equipped with a,
The image acquisition unit
When the movable body is rotated around the designated coordinate value, a first imaging unit that captures a first image including a first reference mark that is predetermined for the movable body before and after the rotation. A first image acquisition unit for acquiring each of the captured first images;
When the movable body is rotated around the designated coordinate value, the second imaging unit captures a second image including a second reference mark that is predetermined for the movable body before and after the rotation. A second image acquisition unit for acquiring each of the second images thus obtained;
Have
The rotation center position specifying unit specifies a first rotation center coordinate value of a first imaging coordinate system of the first imaging unit based on each first reference mark included in each first image, and Identifying a second rotation center coordinate value of the second imaging coordinate system of the second imaging unit based on each second reference mark included in the second image of
The coordinate system correspondence holding unit is a coordinate indicating the correspondence between the first imaging coordinate system and the second imaging coordinate system by associating the first rotation center coordinate value with the second rotation center coordinate value. A moving body control system that maintains system relationships . When the moving body is rotated a plurality of times at an arbitrary rotation angle around the identified coordinate value, the image acquisition unit is configured to capture three or more images captured by the imaging unit before and after the movement. Acquire each image,
The rotation center position specifying unit specifies, as a rotation center coordinate value, a coordinate value in the imaging coordinate system with respect to the rotation center position of the moving body based on the reference mark and the rotation angle included in each image. The moving body control system according to claim 12. The specified coordinate value specifying unit specifies a plurality of the specified coordinate values,
The image acquisition unit identifies each coordinate value in the moving body coordinate system corresponding to the plurality of designated coordinate values based on the coordinate system correspondence relationship, and moves the movement around the identified coordinate values. When the body is rotated at an arbitrary rotation angle, the images captured by the imaging unit are acquired before and after the movement,
The rotation center position specifying unit determines a coordinate value in the imaging coordinate system for the rotation center position of the moving body based on the reference mark and the rotation angle included in each of the images. Each of these is specified as a rotation center coordinate value for each, and the correspondence correction unit corrects the coordinate system correspondence based on the specified coordinate value and the rotation center coordinate value. Item 14. A moving body control system according to Item 13. The specified coordinate value specifying unit specifies a plurality of specified coordinate values corresponding to coordinate values arranged on a predetermined linear in the imaging coordinate system,
The correspondence correction unit corrects a plurality of rotation center coordinate values specified for each of the plurality of designated coordinate values based on the coordinate values arranged on the linear, and corrects each of the specified coordinate values. The moving body control system according to claim 14, wherein the coordinate system correspondence is corrected based on each subsequent rotation center coordinate value. The rotation center position specifying unit includes a first imaging and imaging coordinate system separately specified based on each of the first reference mark included in the first image and the second reference mark included in the second image; The movement according to any one of claims 12 to 15, wherein one rotation center coordinate value in any one of the imaging coordinate systems is specified based on each rotation center coordinate value of the second imaging coordinate system. Body control system. The image acquisition unit
When the movable body is rotated around the designated coordinate value, a first imaging unit that captures a first image including a first reference mark that is predetermined for the movable body before and after the rotation. A first image acquisition unit for acquiring each of the captured first images;
When the movable body is rotated around the designated coordinate value, the second imaging unit captures a second image including a second reference mark that is predetermined for the movable body before and after the rotation. A second image acquisition unit for acquiring each of the second images thus obtained;
Have
The coordinate system correspondence holding unit holds a correspondence between the first imaging coordinate system of the first imaging unit and the second imaging coordinate system of the second imaging unit,
The specified coordinate value specifying unit specifies an arbitrary point on a perpendicular passing through a midpoint of a line segment connecting an arbitrary point specified based on the first reference mark and an arbitrary point specified based on the second reference mark. The mobile body control system according to any one of claims 12 to 15, which is specified as a value. By referring to the coordinate system correspondence, the first imaging coordinate system of the first mark predetermined for the object held by the moving body included in the first image captured by the first imaging unit or The first imaging coordinate system or the second imaging of a second mark predetermined for the first position in the second imaging coordinate system and the object included in the second image captured by the second imaging unit. A position specifying unit for specifying the second position in the coordinate system;
A straight line connecting the first position and the second position in the first imaging coordinate system or the second imaging coordinate system, and a target line predetermined in the first imaging coordinate system or the second imaging coordinate system. A moving mechanism control unit configured to rotate the moving body via a moving mechanism around an arbitrary point within or outside the imaging range in order to move the object to a target position based on an angle formed. The mobile body control system of Claim 16 or Claim 17. Correspondence between an imaging coordinate system predetermined for an imaging unit that captures an image including a reference mark predetermined for a moving object and a predetermined moving object coordinate system for the moving object A coordinate system correspondence holding unit that holds the coordinate system correspondence shown,
A designated coordinate value identifying unit that identifies a coordinate value of an arbitrary point within or outside the imaging range in the imaging coordinate system as a designated coordinate value;
When the coordinate value in the moving object coordinate system corresponding to the specified coordinate value is specified based on the coordinate system correspondence, and the moving object is rotated at an arbitrary rotation angle around the specified coordinate value In addition, an image acquisition unit that acquires the images captured by the imaging unit before and after the movement, and
A rotation center position specifying unit that specifies, as a rotation center coordinate value, a coordinate value in the imaging coordinate system with respect to the rotation center position of the moving body, based on the reference mark and the rotation angle included in each of the images;
A correspondence correction unit that corrects the coordinate system correspondence based on the rotation center coordinate value and the designated coordinate value;
Wherein the image acquiring unit, when moving the movable body is a combination of parallel movement and rotational movement by any rotation angle, also acquires the image including the said reference mark at any time before and after the movement ,
The image acquisition unit
When the movable body is rotated around the designated coordinate value, a first imaging unit that captures a first image including a first reference mark that is predetermined for the movable body before and after the rotation. A first image acquisition unit for acquiring each of the captured first images;
When the movable body is rotated around the designated coordinate value, the second imaging unit captures a second image including a second reference mark that is predetermined for the movable body before and after the rotation. A second image acquisition unit for acquiring each of the second images thus obtained;
Have
The rotation center position specifying unit specifies a first rotation center coordinate value of a first imaging coordinate system of the first imaging unit based on each first reference mark included in each first image, and Identifying a second rotation center coordinate value of the second imaging coordinate system of the second imaging unit based on each second reference mark included in the second image of
The coordinate system correspondence holding unit is a coordinate indicating the correspondence between the first imaging coordinate system and the second imaging coordinate system by associating the first rotation center coordinate value with the second rotation center coordinate value. A moving body control system that maintains system relationships . When the movable body is rotated a plurality of times around an arbitrary point, the rotation angle in the imaging coordinate system specified based on each reference mark included in each image, and the movable body coordinate system The mobile body control system according to any one of claims 12 to 19, further comprising a rotation angle correspondence holding unit that holds a rotation angle correspondence that indicates a correspondence with a rotation angle.

Images