JP2010203826A - Measuring method, measuring device, measurement control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a measuring method, a measuring device, and a measurement control program, capable of reducing complication on measurement of a three-dimensional position of a member. <P>SOLUTION: An object member placed on a plane is imaged by a camera to obtain an imaged image, and an image of a domain specified as a domain including the object member in the obtained imaged image is preserved in a memory as a first template. An in-plane direction position which is a position of a plane facing to the camera of the object member is detected based on matching between the first template and the imaged image, and out-of-plane direction position data in each two-dimensional array in the in-plane direction are obtained by a sensor, and an out-of-plane direction image for expressing the out-of-plane direction position data in each two-dimensional array by pixel values is generated based on the obtained out-of-plane direction position data. An aspect ratio of the generated out-of-plane direction image is corrected based on a prescribed ratio, and the position of the out-of-plane direction of the object member is detected based on matching between the corrected out-of-plane direction image and the first template. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for measuring a three-dimensional position of an object.

  Conventionally, as a technique for measuring the three-dimensional position of a member, measurement using a height sensor such as an optical cutting method is known. The measurement by the height sensor generates a height image showing the height in the out-of-plane direction in the in-plane direction based on the height data obtained by the height sensor for the target member as shown in FIG. To do. Specifically, as shown in FIG. 32, the height sensor obtains height data by imaging laser light irradiated to the target member with a line laser with a camera having a different viewing angle with respect to the target. At this time, as shown in FIG. 33, the position of the laser light in the image picked up by the camera differs depending on the height. This height difference makes it possible to measure the height of the target member. Next, for example, the position in the in-plane direction is detected by image processing such as template matching for the height image, and the height data in the region of interest is extracted from the detected position, thereby the position in the out-of-plane direction. Is detected.

  However, the measurement by the height sensor has a problem in that the positional accuracy in the in-plane direction is poor due to unevenness in the scanning speed of the height sensor with respect to the target member or the center of gravity calculation for several pixels. On the other hand, as a technique for measuring the three-dimensional position of a member by a height sensor, a technique for preparing a template for a height image for a member to be measured and searching for a region of interest from the height image is known. (For example, refer to Patent Document 1).

Japanese Patent Laid-Open No. 02-150704

  However, according to Patent Document 1, since a template is required for each of the position detection in the in-plane direction and the out-of-plane direction, the burden of creating the template increases, and the processing related to measurement becomes complicated. There is.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a measurement method, a measurement apparatus, and a measurement control program that can reduce the complexity of measuring the three-dimensional position of a member. And

  In order to solve the above-described problem, the measurement method includes imaging a target member placed on a plane with a camera, acquiring the captured image, and specifying the region including the target member in the acquired captured image. Is stored in a memory as a first template, and an in-plane direction position that is a position of the surface of the target member facing the camera is detected based on matching between the first template and the captured image, and a sensor To obtain out-of-plane direction position data in each of the two-dimensional arrays in the in-plane direction, and express the out-of-plane direction position data in each of the two-dimensional arrays by pixel values based on the obtained out-of-plane direction position data. An out-of-plane direction image is generated, the aspect ratio of the generated out-of-plane direction image is corrected based on a predetermined ratio, and the corrected out-of-plane direction image and the first template are corrected. Based on the matching of the detects the plane position of the target member.

  In addition, the measuring apparatus includes a camera that captures an object member placed on a plane, an out-of-plane direction sensor that detects a height with respect to the plane, and a first acquisition unit that acquires a captured image captured by the camera. A storage unit that stores an image of a region specified as a region including the target member in the captured image acquired by the first acquisition unit in a memory as a first template, and the first template and the captured image A surface in each of the two-dimensional arrays in the in-plane direction by the first detection unit that detects an in-plane direction position that is a position of the surface facing the camera of the target member based on the matching and the out-of-plane direction sensor A second acquisition unit for acquiring outward direction position data, and an out-of-plane direction in each of the two-dimensional arrays based on the out-of-plane direction position data acquired by the second acquisition unit An image generation unit that generates an out-of-plane direction image that expresses position data with pixel values, a correction unit that corrects an aspect ratio of the out-of-plane direction image generated by the image generation unit based on a predetermined ratio, and the correction A second detection unit that detects a position of the target member in the out-of-plane direction based on matching between the out-of-plane direction image corrected by the unit and the first template.

  In addition, the measurement control program includes a camera that images a target member that is a measurement target placed on a plane, and an out-of-plane direction sensor that detects an out-of-plane direction position that is a height of the target member with respect to the plane. A measurement control program for controlling a measurement apparatus having a first acquisition step for acquiring a captured image captured by the camera, and a region including the target member in the captured image acquired by the first acquisition step An in-plane direction that is a position of a surface of the target member facing the camera based on a storage step of storing an image of the region that has been stored in a memory as a first template and matching between the first template and the captured image A first detection step for detecting a position, and an out-of-plane direction in each of the two-dimensional arrays in the in-plane direction by the out-of-plane direction sensor A second acquisition step of acquiring position data, and an out-of-plane direction expressing the out-of-plane direction position data in each of the two-dimensional arrays by pixel values based on the out-of-plane direction position data acquired by the second acquisition step An image generation step for generating an image; a correction step for correcting an aspect ratio of the out-of-plane direction image generated by the image generation step based on a predetermined ratio; the out-of-plane direction image corrected by the correction step; Based on matching with the first template, the computer executes a second detection step of detecting the position of the target member in the out-of-plane direction.

  The complexity of measuring the three-dimensional position of the member can be reduced.

It is a figure which shows the structure of the measuring apparatus which concerns on this Embodiment. It is a figure which shows the hardware constitutions of a control apparatus. It is a figure which shows the function structure of a control apparatus. It is a flowchart which shows operation | movement of the control apparatus which concerns on the three-dimensional position measurement of an object member. It is a flowchart which shows operation | movement of a template image creation process. It is a figure which shows the picked-up image by an image sensor. It is a figure which shows ROI with respect to a target member. It is a figure which shows the template image of an object member. It is a flowchart which shows operation | movement of a correction ratio calculation process. It is a figure which shows ROI with respect to a ratio reference | standard device. It is a figure which shows a ratio reference | standard image. It is a figure which shows height data. It is a figure which shows the line direction and scanning direction of a laser beam. It is a figure which shows the height image before correction | amendment. It is a figure which shows the height image after correction | amendment by an estimated ratio. It is a figure which shows the matching of a height image and a ratio reference | standard image. It is a figure which shows the type | formula which calculates a correlation value. It is a figure which shows the distance of the horizontal direction of a ratio reference | standard device. It is a figure which shows the type | formula which calculates the distance L1 of a horizontal direction. It is a figure which shows the distance of the perpendicular direction of a ratio reference | standard device. It is a figure which shows the type | formula which calculates the distance L2 of a perpendicular direction. It is a flowchart which shows the operation | movement of an in-plane direction position detection process. It is a figure which shows the matching process by the template image with respect to the image by an image sensor. It is a flowchart which shows operation | movement of a member front-end | tip position detection process. It is a figure which shows ROI with respect to the front-end | tip of a target member. It is a figure which shows the type | formula which concerns on a Sobel filter. It is a figure which shows the edge and vertical line in the detection of the front-end | tip of a target member. It is a flowchart which shows the operation | movement of an out-of-plane direction position detection process. It is a figure which shows matching with a height image and a template image. It is a figure which shows a computer system. It is a figure which shows an object member. It is a figure which shows the outline | summary of a height sensor. It is a figure which shows the principle of a height sensor.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, the configuration of the measurement apparatus according to the present embodiment will be described. FIG. 1 is a diagram showing a configuration of a measuring apparatus according to the present embodiment.

  As shown in FIG. 1, a measuring apparatus 1 according to the present embodiment includes an image sensor 10, a height sensor 11, a linear motion stage 12, and a control device 13, and a three-dimensional position of a member placed on a plane. Measure. The image sensor 10 images a target member that is a measurement target from a direction perpendicular to a plane on which the target member is placed. In addition, the height sensor 11 measures the height (out-of-plane direction position) of the target member from the placed plane by an optical cutting method using a line laser and a camera. The height sensor 11 may measure the height of the target member by a method other than the light cutting method, and measures the height of the target member by scanning in the plane direction on which the target member is placed. It does n’t matter if you do. The linear motion stage 12 moves the height sensor 11 in order to scan the target member with a line laser in the height measurement of the target member by the height sensor 11. In addition, in the measurement of the three-dimensional position, the control device 13 executes control related to the control of the image sensor 10, the height sensor 11, and the linear motion stage 12, and the processing related to the three-dimensional position measurement. Note that the measuring apparatus 1 according to the present embodiment images and scans the ratio reference device (linear pattern) together with the target member in the three-dimensional position measurement. This ratio reference device is used as a reference for the ratio in the correction of the aspect ratio of the height image based on height data (out-of-plane direction position data) described later. Further, the ratio reference device has a characteristic shape so that the distance can be calculated in the horizontal direction and the vertical direction. In the present embodiment, this characteristic shape is a cross shape, and in this cross shape, straight lines in the horizontal direction and the vertical direction are the same distance from each other. The cross shape is expressed by a concave portion or a convex portion so that the shape can be read by the image sensor and the height sensor.

  Next, the hardware configuration and functional configuration of the control device will be described. FIG. 2 is a diagram illustrating a hardware configuration of the control device. FIG. 3 is a diagram illustrating a functional configuration of the control device.

  As illustrated in FIG. 2, the control device 13 includes a CPU (Central Processing Unit) 951 and a memory 952 as hardware. As shown in FIG. 3, the control device 13 functions as an image acquisition unit (first acquisition unit) 131, an in-plane image processing unit (image generation unit, first detection unit) 132, and an in-plane direction position detection unit. (First detection unit) 133, storage unit 134, height data acquisition unit (second acquisition unit) 135, height image generation unit (image generation unit) 136, out-of-plane image processing unit (correction unit, second detection unit) 137 and an out-of-plane direction position detection unit (second detection unit) 138. Note that these functions are realized by the CPU 951. The image acquisition unit 131 causes the image sensor 10 to image the target member and the ratio reference device, and acquires a captured image. The in-plane image processing unit 132 creates a template image (first template) based on the captured image, and performs matching processing using the template image. Further, the in-plane direction position detection unit 133 obtains the position of the target member in the in-plane direction by matching processing by the in-plane image processing unit 132. Further, the height data acquisition unit 135 causes the height sensor 11 to measure the height of the target member and the ratio reference device, and acquires height data indicating a two-dimensional height. In addition, the height image generation unit 136 generates a height image in which the height of the target member is indicated by luminance by converting the two-dimensional height in the height data into a luminance value. Further, the out-of-plane image processing unit 137 corrects the aspect ratio of the height image and performs matching processing using the template image. Further, the out-of-plane direction position detection unit 138 detects the height of the target member based on the corrected height image.

  Next, the operation of measuring the three-dimensional position of the target member by the measuring apparatus according to the present embodiment will be described. FIG. 4 is a flowchart illustrating the operation of the control device according to the three-dimensional position measurement of the target member. Details of the operation of each process shown in FIG. 4 will be described later.

  As shown in FIG. 4, first, the control device 13 executes a template image creation process for creating a template image related to three-dimensional position measurement (S101), and then uses the created template image to create a height image. A correction ratio calculation process for calculating a ratio for correcting the aspect ratio is executed (S102). Next, the control device 13 executes in-plane direction position detection processing for detecting the in-plane direction position of the target member (S103), and executes out-of-plane direction position detection processing for detecting the out-of-plane direction position of the target member. (S104), it is determined whether or not all the measurements related to the target member have been completed (S105).

  When all the measurements related to the target member are completed (S105, YES), the control device 13 ends the three-dimensional position measurement.

  On the other hand, when all the measurements related to the target member are not completed, that is, when there is another designated measurement (S105, NO), the control device 13 executes the in-plane direction position detection process again (S105). S103).

  Next, template image creation processing will be described. FIG. 5 is a flowchart showing the operation of the template image creation process. FIG. 6 is a diagram illustrating an image captured by the image sensor. FIG. 7 is a diagram showing the ROI for the target member. FIG. 8 is a diagram showing a template image of the target member. Hereinafter, the operation shown in FIG. 5 will be described with reference to FIGS.

  As illustrated in FIG. 6, the image acquisition unit 131 of the control device 13 causes the image sensor 10 to image the target member and the ratio reference device, and acquires the captured image (S201, first acquisition step). Next, the in-plane image processing unit 132 converts the image in the ROI (Region of Interest) set so as to surround the target member as shown in FIG. 7 as a template image as shown in FIG. The data is stored in the memory 952 (S202, storage step), and the template image creation process is terminated. Note that the ROI set for the captured image is set manually.

  Next, the correction ratio calculation process will be described. FIG. 9 is a flowchart showing the operation of the correction ratio calculation process. FIG. 10 is a diagram showing an ROI for the ratio reference unit. FIG. 11 is a diagram showing a ratio reference image. FIG. 12 is a diagram showing height data. FIG. 13 is a diagram showing the line direction of the laser beam and the scanning direction. FIG. 14 is a diagram showing a height image before correction. FIG. 15 is a diagram illustrating a height image after correction based on the estimated ratio. FIG. 16 is a diagram showing matching between the height image and the ratio reference image. FIG. 17 is a diagram illustrating an equation for calculating a correlation value. Moreover, FIG. 18 is a figure which shows the distance of the horizontal direction of a ratio reference | standard device. FIG. 19 is a diagram illustrating an equation for calculating the horizontal distance L1. FIG. 20 is a diagram illustrating the distance in the vertical direction of the ratio reference device. FIG. 21 is a diagram showing an equation for calculating the distance L2 in the vertical direction. Hereinafter, the operation shown in FIG. 9 will be described with reference to FIGS.

  First, as shown in FIG. 10, the in-plane image processing unit 132 of the control device 13 converts an image in the ROI that is set to surround the ratio reference unit in the captured image into a ratio reference image as shown in FIG. 11. It is stored in the memory 952 as (second template) (S301, storage step). Next, the height data acquisition unit 135 causes the height sensor 11 to measure the height of the target member and the ratio reference device, and in the two-dimensional array corresponding to the in-plane direction of the target member, each of the arrays is the target member. Height data indicating the height is acquired (S302, second acquisition step). As shown in FIG. 12, this height data is data indicating the height in each of the two-dimensional arrays on the plane on which the target member is placed.

Next, the height image generation unit 136 generates a height image based on the height data (S303, image generation step). Here, the height image will be described. As shown in FIG. 13, the height image has a laser beam line direction of m, a scanning direction of n, a height at a certain point of H (m, n), a minimum height value of Hmin, When the maximum value is Hmax, the pixel value I (m, n) at a certain point is obtained by the following equation.
I (m, n) = 255 × (H (m, n) −Hmin) / (Hmax−Hmin)

  By performing linear interpolation as shown in this equation, a height image is generated from the height data. According to this equation, the height data is converted into a black and white image with 256 gradations. That is, the height data is converted into an image indicated by one parameter (pixel value) in which the out-of-plane direction position in the two-dimensional array in the in-plane direction is indicated by 0 to 255.

  When the height image is generated by the height image generation unit 136, the out-of-plane image processing unit 137 corrects the aspect ratio of the height image as shown in FIG. 14 using the estimated ratio cEST (predetermined ratio). (S304, correction step). Specifically, assuming that the number of vertical pixels is m and the number of horizontal pixels is n, n that is the scanning direction is multiplied by the estimated ratio cEST.

  Next, the out-of-plane image processing unit 137 detects the ratio reference device by matching processing for scanning the ratio reference image on the corrected height image as shown in FIG. 15 (S305, correction step). Here, the matching process will be described. As shown in FIG. 16, this matching processing is performed by comparing M with I and M from I when an image to be scanned (ratio reference image) is M and an image to be detected (height image) is I. M is detected. Specifically, in the matching process, when the I pixel is I (i, j) and the M pixel is M (i, j), the correlation value k (a, b) is obtained by the equation shown in FIG. The position of M is determined such that the correlation value becomes maximum in the image I. Here, a and b indicate the scanning amount of the image M, and a correlation value of M with respect to the image I at each position of the image I is obtained by shifting a and b by one pixel.

  Note that the out-of-plane image processing unit 137 can also change the estimated ratio cEST so that the correlation value of M with respect to the image I becomes the highest. Hereinafter, optimization of the estimation ratio will be described. First, the out-of-plane image processing unit 137 sets the estimation ratio in a plurality of stages such as 0.9, 0.8, 0.7,..., And corrects the height image with each estimation ratio. Next, the out-of-plane image processing unit 137 performs the above-described matching processing on the height images corrected by the respective estimation ratios, and sets the estimation ratio of the height image having the highest correlation value as the optimum estimation ratio. To do. When a certain estimated ratio (for example, 0.6) becomes the optimum ratio, the estimated ratio may be set in more detailed steps such as 0.51, 0.52, 0.53,. In this case, the out-of-plane image processing unit 137 repeats the correction of the ratio of the image I and the matching process with the image M so that the correlation value becomes high. In this way, by obtaining a further optimal estimation ratio, it is possible to correct the height image so that the accuracy of the aspect ratio becomes higher.

  Next, as shown in FIG. 18, the out-of-plane image processing unit 137 detects the left and right tip points C and D of the cross-shaped horizontal straight line in the ratio reference device on the height image after correction (S306). ). Specifically, the out-of-plane image processing unit 137 calculates a luminance difference between two adjacent pixels along a horizontal line of the cross shape, and compares it with a predetermined threshold value Ith to thereby calculate a cross shape. The positions C (imaxC, jmaxC) and D (imaxD, jmaxD) of the left and right tip points are detected. Here, the out-of-plane image processing unit 137 has a line L1 connecting the points C and D when the coordinates of the intersection of the horizontal lines and the vertical lines of the cross shape are (x, y) and the inclination of the line is a1. Is defined as y = a1 · x. Here, x represents the scanning direction of the height sensor, and y represents the line direction of the laser. In this case, the tip point is a point where | I (x, a1 · x) −I (x + 1, a1 · x + 1) |> Ith. Next, the out-of-plane image processing unit 137 calculates a linear distance L1 between the points C and D (S307). Specifically, the distance L1 is obtained by the equation shown in FIG.

  Next, as shown in FIG. 20, the out-of-plane image processing unit 137 detects the left and right tip points E and F of the cross-shaped vertical straight line in the ratio reference device on the height image after correction (S308). ). Specifically, the out-of-plane image processing unit 137 calculates the luminance difference between two adjacent pixels along the vertical line of the cross like the points C and D, and calculates a predetermined threshold value Ith. , The positions E (imaxE, jmaxE) and F (imaxF, jmaxF) of the left and right tip points of the cross shape are detected. Here, the out-of-plane image processing unit 137 defines the straight line L2 as y = a2 · x, where a2 is the slope of the straight line L2 connecting the points E and F. In this case, the tip point is a point where | I (x, a2 · x) −I (x + 1, a2 · x + 1) |> Ith. Next, the out-of-plane image processing unit 137 calculates a linear distance L2 between the points E and F (S309). Specifically, the distance L2 is obtained by the equation shown in FIG.

  When the horizontal and vertical distances of the cross are obtained on the corrected height image, the out-of-plane image processing unit 137 calculates a correction ratio c based on these distances (S310). The correction ratio c is obtained by c = cEST · L1 / L2 since the aspect ratio of the height image is equivalent to the image captured by the image sensor 10 when L1 and L2 match.

  Next, the in-plane direction position detection process will be described. FIG. 22 is a flowchart showing the operation of the in-plane direction position detection process. FIG. 23 is a diagram illustrating matching processing using a template image with respect to an image by the image sensor.

  As shown in FIG. 22, first, the image acquisition unit 131 causes the image sensor 10 to image the target member and the ratio reference device, and acquires the captured image (S401). Next, the in-plane image processing unit 132 performs matching processing between the template image and the captured image (S402, first detection step).

  Next, the in-plane direction position detection unit 133 performs a member tip position detection process for detecting the tip position of the target member (S403, first detection step), and the target member performing the three-dimensional position measurement is 1 It is determined whether the member is a member (S404). The member tip position detection process will be described later.

  When the target member is the first member (S404, YES), the in-plane direction position detection unit 133 registers the tip position of the target member detected by the member tip position detection process as the reference position (S405, No. 1). 1 detection step). When it is not necessary to detect the tip position of the target member and the member tip position detection process is not performed, the in-plane direction position detection unit 133 has the highest correlation value in the matching process between the template image and the captured image. The position of the center of gravity of the high template image is set as the position of the target member in the captured image.

  On the other hand, when the target member is not the first member (NO in S404), the in-plane direction position detection unit 133 makes the relative position from the reference position of the tip position of the target member detected by the member tip position detection processing. The position is calculated (S406). In this way, when there are a plurality of target members, by calculating the relative positional relationship, for example, it is possible to inspect and adjust a component made up of a plurality of members.

  Next, the member tip position detection process will be described. FIG. 24 is a flowchart showing the operation of the member tip position detection process. FIG. 25 is a diagram showing an ROI with respect to the tip of the target member. FIG. 26 is a diagram illustrating an expression relating to the Sobel filter. FIG. 27 is a diagram showing edges and vertical lines in detection of the tip of the target member. The operation shown in FIG. 24 will be described below with reference to FIGS.

First, the in-plane direction position detection unit 133 extracts an image in Rdst, which is an ROI set in a portion including the tip of the target member (S501), and uses the Sobel filter for the extracted image. The edge of the member is extracted (S502). Specifically, the in-plane direction position detection unit 133 represents the product f _A of a matrix composed of nine pixels in the vertical and horizontal directions of the pixel of interest (i, j) and the matrix shown in Equation 1 in FIG. Each product f _B with the matrix is calculated. Further, as shown in Expression 3, the in-plane direction position detection unit 133 uses the square of the square sum of f _A and f _B as a pixel value to detect an edge in the image.

  Next, as shown in FIG. 27, the in-plane direction position detection unit 133 scans the detected edge with a straight line in the vertical direction (S503), and obtains a point sequence of the intersection of this straight line and the upper edge. (S504). Next, the in-plane direction position detection unit 133 applies the least square method to the acquired point sequence and calculates an upper straight line equation (S505). Further, the in-plane direction position detection unit 133 acquires a point sequence of intersection points between the straight line in the vertical direction and the lower edge in the same manner as the upper edge (S506), and the least square method is performed on the acquired point sequence. Is applied to calculate the equation of the lower straight line (S507).

  Next, the in-plane direction position detection unit 133 calculates a straight line passing through the center of the two straight lines calculated based on the point sequence as a central line (S508), and calculates the intersection of the central line and the edge of the target member. It detects as a member tip position (S509).

  Thus, by detecting the tip position of the target member, for example, the tip position can be associated with the height image to obtain the height of the tip position. In detecting the tip position, the edge is scanned with a straight line in the vertical direction. For example, when the edge direction is close to the vertical direction, the edge is detected with a straight line in the horizontal direction. That is, in detecting the tip position, the straight line scanned by the edge is the horizontal direction or the vertical direction depending on the direction of the edge.

  Next, the out-of-plane direction position detection process will be described. FIG. 28 is a flowchart showing the operation of the out-of-plane direction position detection process. FIG. 29 is a diagram showing matching between a height image and a template image.

  First, the height data acquisition unit 135 causes the height sensor 11 to scan the target member and the ratio reference device to acquire height data (S601, second acquisition step). Next, the height image generation unit 136 generates a height image based on the height data (S602, generation step).

  Next, the out-of-plane image processing unit 137 corrects the aspect ratio of the height image using the correction ratio c (S603, correction step). As shown in FIG. Matching processing is performed using the template image, and the region of the target member in the height image is detected (S604, second detection step).

  Next, the out-of-plane direction position detection unit 138 detects a position in the out-of-plane direction from the height data corresponding to the target member in the height image (S605, second detection step). Specifically, the out-of-plane direction position detection unit 138 calculates the average value of the data of the height of the region having the highest correlation value with the template image and calculates the average value. The position in the out-of-plane direction of the target member.

  As described above, the measuring apparatus 1 according to the present embodiment detects the in-plane direction position of the target member based on the captured image captured by the image sensor 10 and the out-of-plane direction position as the height sensor 11. Is detected based on the height data scanned by. Further, according to the present embodiment, since the matching process is performed using the same template image for the image by the image sensor 10 and the height image by the height sensor 11, the complexity associated with the three-dimensional position measurement is complicated. Can be reduced. Moreover, the accuracy of the correction of the aspect ratio of the height image can be improved by using the ratio reference device as a reference of the aspect ratio as an imaging and scanning object together with the target member.

  The present invention can be applied to the following computer system. FIG. 30 is a diagram illustrating an example of a computer system to which the present invention is applied. A computer system 13 as a control device shown in FIG. 30 is used to input various information to a main body 901 incorporating a CPU, a disk drive, and the like, a display 902 that displays an image according to instructions from the main body 901, and the computer system 13. A keyboard 903, a mouse 904 for designating an arbitrary position on the display screen 902a of the display 902, and a communication device 905 for accessing an external database or the like and downloading a program or the like stored in another computer system. The communication device 905 may be a network communication card, a modem, or the like.

  Moreover, the program which performs each step mentioned above in the computer system which comprises a control apparatus as mentioned above can be provided as a measurement control program. By storing this program in a recording medium that can be read by the computer system, the program can be executed by the computer system constituting the control device. A program for executing the above steps is stored in a portable recording medium such as a disk 910 or downloaded from a recording medium 906 of another computer system by the communication device 905. Further, a measurement control program (measurement software) that causes the computer system 13 to have at least a measurement function is input to the computer system 13 and compiled. This program causes the computer system 13 to operate as a control device having a measurement function. Further, this program may be stored in a computer-readable recording medium such as a disk 910, for example. Here, as a recording medium readable by the computer system 13, portable storage such as an internal storage device such as a ROM or a RAM, a disk 910, a flexible disk, a DVD disk, a magneto-optical disk, an IC card, etc. It includes a medium, a database holding a computer program, or other computer systems and the database, and various recording media accessible by a computer system connected via communication means such as a communication device 905.

  DESCRIPTION OF SYMBOLS 1 Measuring apparatus, 10 Image sensor, 11 Height sensor, 12 Linear motion stage, 13 Control apparatus, 131 Image acquisition part, 132 In-plane image processing part, 133 In-plane direction position detection part, 134 Storage part, 135 Height data Acquisition unit, 136 height image generation unit, 137 out-of-plane image processing unit, 138 out-of-plane direction position detection unit, 951 CPU, 952 memory.

Claims (5)

The target member placed on the plane is imaged with a camera,
Acquire the captured image,
In the acquired captured image, an image of an area designated as an area including the target member is stored in a memory as a first template,
Based on the matching between the first template and the captured image, an in-plane direction position that is a position of the surface of the target member facing the camera is detected;
Out-of-plane direction position data in each of the two-dimensional arrays in the in-plane direction is acquired by a sensor,
Based on the acquired out-of-plane direction position data, generate an out-of-plane direction image that expresses out-of-plane direction position data in each of the two-dimensional arrays by pixel values,
Correct the aspect ratio of the generated out-of-plane image based on a predetermined ratio,
A measurement method comprising: detecting a position of the target member in an out-of-plane direction based on matching between the corrected out-of-plane image and the first template. The camera captures a linear pattern having a linear shape in the horizontal and vertical directions in the planar direction together with the target member,
The linear pattern is scanned together with the target member by the out-of-plane direction sensor,
An image of an area designated as an area including the linear pattern on the captured image is stored in the memory as a second template,
The straight line pattern in the out-of-plane direction image is detected based on matching between the out-of-plane direction image whose aspect ratio is changed by the predetermined ratio and the second template, and the ratio based on the distance of the straight line shape of the straight line pattern The measurement method according to claim 1, wherein an aspect ratio of the out-of-plane image is corrected. The detection of the in-plane direction position is performed by detecting an edge of a captured image in a region designated as a region including the tip of the target member in a region on the captured image having the highest correlation value in matching with the first template. And detecting parallel edges among the detected edges as two straight lines, and detecting an intersection of the center line and the edge that are the same distance from the two straight lines as an in-plane direction position of the target member, The measuring method according to claim 1 or 2. A camera for imaging a target member placed on a plane;
An out-of-plane direction sensor that detects a height relative to the plane;
A first acquisition unit that acquires a captured image captured by the camera;
A storage unit that stores an image of a region designated as a region including the target member in the captured image acquired by the first acquisition unit in a memory as a first template;
A first detection unit that detects an in-plane direction position that is a position of a surface of the target member facing the camera based on matching between the first template and the captured image;
A second acquisition unit that acquires out-of-plane direction position data in each of the two-dimensional arrays in the in-plane direction by the out-of-plane direction sensor;
Based on the out-of-plane direction position data acquired by the second acquisition unit, an image generation unit that generates an out-of-plane direction image that expresses out-of-plane direction position data in each of the two-dimensional arrays by pixel values;
A correction unit that corrects the aspect ratio of the out-of-plane image generated by the image generation unit based on a predetermined ratio;
A measurement apparatus comprising: a second detection unit configured to detect an out-of-plane direction position of the target member based on matching between the out-of-plane image corrected by the correction unit and the first template. Measurement for controlling a measurement apparatus having a camera that images a target member that is a measurement target placed on a plane, and an out-of-plane direction sensor that detects an out-of-plane direction position that is a height of the target member with respect to the plane. A control program,
A first acquisition step of acquiring a captured image captured by the camera;
A storage step of storing an image of a region designated as a region including the target member in the captured image acquired by the first acquisition step in a memory as a first template;
A first detection step of detecting an in-plane direction position that is a position of a surface of the target member facing the camera based on matching between the first template and the captured image;
A second acquisition step of acquiring out-of-plane direction position data in each of the two-dimensional arrays in the in-plane direction by the out-of-plane direction sensor;
Based on the out-of-plane direction position data acquired by the second acquisition step, an image generation step of generating an out-of-plane direction image that expresses out-of-plane direction position data in each of the two-dimensional arrays by pixel values;
A correction step of correcting the aspect ratio of the out-of-plane image generated by the image generation step based on a predetermined ratio;
A measurement control program that causes a computer to execute a second detection step of detecting a position of the target member in the out-of-plane direction based on matching between the out-of-plane image corrected in the correction step and the first template.

Images