JP2014041074A - Image processing apparatus and inspection apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that detects a position of a subject and the amount of rotation at a high speed through simple processing, from image data obtained by imaging the subject, and an inspection apparatus including the image processing apparatus.SOLUTION: An image processing apparatus includes: a distance image generation part 207 that generates distance image data from image data obtained by imaging a subject; a background/subject region separation part 208 that separates the distance image data into a background region and a subject region; and a rotation amount calculation part 209 that calculates a position of the subject and the amount of rotation from the subject region in the distance image data.

Description

  The present invention relates to an image processing apparatus that processes image data obtained by imaging a test object to detect the position and rotation amount of the test object, and an inspection apparatus having a processing function of the image processing apparatus About.

  Conventionally, there is a known method for detecting the position (positional deviation) and rotation amount (rotational deviation) of an object using Hough transform, template matching, or the like for image data obtained by imaging the object. ing. In addition, a method (ICP method) for detecting the position (positional deviation) and rotation amount (rotational deviation) of a test object in the same manner using distance image data is also known (for example, Non-Patent Document 1). However, since the conventional method uses the entire screen of image data and distance data as the processing target area, there is a problem that the calculation processing amount, calculation time, hardware used, and the like increase. Also, the ICP method has a very complicated algorithm.

  In Patent Document 1, in the position detection method using template matching, the calculation target region of template matching is limited according to the density of feature points in the image data, so that the position of the subject can be detected at high speed. Although techniques for detecting misalignment and rotational misalignment are described, feature point extraction needs to cover the entire screen of image data, and there is a limit to speeding up the processing.

  The present invention relates to an image processing apparatus that detects the position and rotation amount of a test object at high speed by simple processing from image data obtained by imaging the test object as compared with the prior art, and the image An object of the present invention is to provide an inspection apparatus having a processing function of a processing apparatus.

  The image processing apparatus of the present invention includes distance image generation means for generating distance image data from image data obtained by photographing a test object, and a background for separating the distance image data into a background area and a test object area. A test object region separating unit and a position / rotation amount calculating unit for calculating the position / rotation amount of the test object from the test object region of the distance image data.

  In addition, the inspection apparatus of the present invention includes a photographing unit that photographs a test object from a plurality of directions and acquires a plurality of image data having different parallaxes, and a distance image generation unit that generates distance image data from the plurality of image data. A background and test object region separating means for separating the distance image data into a background region and a test object region, and calculating the position and rotation amount of the test object from the test object region of the distance image data Position and rotation amount calculating means, and comparing the calculated position and rotation amount of the test object with a predetermined threshold value to determine the position shift and rotation shift of the test object. And determining means.

  According to the present invention, it is possible to detect the position and rotation amount of a test object at high speed by simple processing using distance image data.

It is the figure which showed the installation state of the to-be-tested object and camera of the inspection apparatus containing the image processing apparatus which concerns on one Embodiment of this invention. It is a system configuration figure of the inspection device concerning this embodiment. It is a figure which shows the example of the random pattern by a pattern projection part. FIG. 3 is a detailed block diagram illustrating a configuration example of a processing unit in FIG. 2. It is a figure which shows the specific example of background distance image data. It is a figure which shows the specific example of distance image data and background / test object area | region separation. It is a figure which shows another specific example of distance image data and a background and test object area | region separation. It is a figure showing the coordinate of the distance image data in the case of FIG. It is a figure showing the coordinate of the distance image data in the case of FIG. It is a figure showing the state transition of the control part of FIG. It is a figure which shows the operation | movement flow of the processing unit in each state of FIG. It is a figure which shows the flow of the whole operation | movement of the test | inspection apparatus which concerns on this embodiment. It is a figure which shows the display screen of the personal computer in a background photography state. It is a figure which shows the display screen of the personal computer in a threshold value adjustment state. It is a figure which shows the display screen of the personal computer at the time of threshold value adjustment setting. It is a figure which shows the display screen of the personal computer in an operation state. It is a figure which shows one Example of an integrated stereo camera. It is a figure which shows another Example of an integrated stereo camera.

  Embodiments of the present invention will be described below with reference to the drawings. In the embodiment, the object to be imaged is captured by a stereo camera including two cameras as imaging means, two image data having different viewpoints are acquired for the object, and distance image data is obtained from the two image data. Is generated, and the distance image data is separated into a background area and a test object area, and the position and rotation amount of the test object are calculated by calculating the center of gravity and moment of the test object area.

  FIG. 1 is a diagram illustrating an example of an installation state of a test object and a camera of an inspection apparatus (including an image processing apparatus) according to the present embodiment. In FIG. 1, a camera A101 and a camera B102, which are image pickup means, are fixed to a camera fixing jig 100 at a constant interval and so that their optical axes are parallel to each other. The camera A101 and camera B102 constitute a stereo camera, the camera A101 corresponds to the left eye of the stereo camera, and the camera B102 corresponds to the right eye. Further, the pattern projection unit 103 is fixed to the camera fixing jig 100. The camera fixing jig 100 is fixed to the support base 110. The camera fixing jig 100 attached to the support base 110 and the installation surface 112 of the support base 110 are configured to be parallel. As a result, the camera A101, the camera B102, and the pattern projection unit 103 are installed vertically downward. The test object 10 is placed on the installation surface 112 of the support base 110 at a position facing the camera A 101, the camera B 102, and the pattern projection unit 103. Here, the shape of the test object is a rectangular parallelepiped.

  FIG. 2 is an overall system configuration diagram of the inspection apparatus according to the present embodiment. Cameras A101 and B102 fixed to the camera fixing jig 100 take images of the background of the installation surface 112 and the test object 10, respectively, convert the optical image into an analog electric signal, and further convert it into a digital signal. Output data. For example, the image data consists of 8 bits / pixel and takes 0 to 255 gradations (luminance values).

  The pattern projection unit 103 irradiates the test object 10 and its surroundings with a random pattern so that distance calculation using a stereo image by a camera A101 and a camera B102 described later can be stably performed. As shown in FIG. 3, the pattern projection unit 103 irradiates a random pattern to a region wider than the stereo camera ranging region of the camera A 101 and the camera B 102. Note that the pattern projection unit 103 does not have to be used when ranging calculation can be performed stably, such as by shooting a subject with a pattern or the like to perform ranging calculation.

  The processing unit 200 controls the photographing of the camera A101 and the camera B102, takes in the image data output from the camera A101 and the camera B102, calculates the position and rotation amount of the test object 10 from the image data, It is a processing substrate that performs various processes for determining misalignment and rotational misalignment, mainly image processing.

  A personal computer (PC) 300 and an external communication unit 400 are connected to the processing unit 200.

  The personal computer 300 designates parameters for processing in the processing unit 200 and other instructions to the processing unit 200 and receives image data, calculation results, determination results, and the like from the processing unit 200. indicate. That is, the personal computer 300 functions as user interface means. Note that an operation unit such as a touch panel may be used instead of the personal computer 300, and the processing unit 200 may be configured integrally.

  The external communication unit 400 is generally connected to a PLC (Programmable Logic Controller). For example, the external communication unit 400 can give a shooting trigger to the processing unit 200, monitor the operation state of the processing unit 200, and receive a determination result from the processing unit 200.

  FIG. 4 is a block diagram illustrating a detailed configuration of the processing unit 200 according to the present embodiment. The processing unit 200 includes a camera A interface (I / F) 201, a camera BI / F 202, a PCI / F 203, an external I / F 204, a memory 205, an image conversion unit 206, a distance image generation unit 207, a background / test object separation. Module 208, position / rotation amount calculation unit 209, position shift / rotation shift determination unit 210, and control unit 211. These modules are connected by a bus 212. Here, the image conversion unit 206, the distance image generation unit 207, the background / test object separation unit 208, and the position / rotation amount detection unit 209 function as an image processing apparatus according to the present invention.

  The control unit 211 controls the operation of each module. The control unit 211 also exchanges image data and other data with each module, and stores / retrieves image data and other data to and from the memory 205. The memory 205 stores image data, parameters, user setting values, and the like used in each module.

  The camera AI / F 201 is an interface with the camera A 101, transmits a shooting instruction, setting data, and the like from the control unit 211 to the camera A 101, and captures image data output from the camera A 101 and stores it in the memory 205. The camera BI / F 202 is an interface with the camera B102. Similarly, the camera BI / F 202 transmits shooting instructions, setting data, and the like from the control unit 211 to the camera B102, and captures and stores the image data output from the camera B102 in the memory 205. . Here, the setting data includes automatic exposure, white balance, image signal mode (grayscale / YUV color / raw image, etc.) and the like. The image signal mode is gray scale. Therefore, monochrome luminance image data is output from the camera A101 and the camera B102.

  The PCI / F 203 is an interface with the personal computer (PC) 300, receives a user setting value (positional deviation / rotational deviation determination threshold) set in the personal computer 300, and stores it in the memory 205. The PCI / F 203 transmits image data, position / rotation calculation results, determination results, and the like to the personal computer 300.

  The external I / F 204 is an interface with the external communication unit 400, receives a shooting trigger from the external communication unit 400, and outputs an operation state of the processing unit 200 and a determination result to the external communication unit 400.

  The image conversion unit 206 takes in the image data captured by the camera A 101 (hereinafter referred to as luminance image data A) and the image data captured by the camera B 102 (hereinafter referred to as luminance image data B) from the memory 205, and performs image distortion. Correct. Further, the image conversion unit 206 parallelizes the distortion-corrected luminance image data A with the distortion-corrected luminance image data B (parallelization of the stereo image), and the parallelized luminance image data A ′ and the luminance Image data B ′ is stored in the memory 205. For example, as a distortion correction method, “A flexible new technique for camera calibration”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11): 1330-1334, 2000, a parallelization method is Fusiello A, Trucco E and Verri A: A compact algorithm for rectification of stereo pairs. The method described in Machine Vision and Applications: 16-22, 2000 may be used.

  Note that the parallelization process can be omitted by fixing the camera A101 and the camera B102 to the camera fixing jig 100 in a state where the camera A101 and the camera B102 are parallelized in advance.

  The distance image generation unit 207 takes in the parallel luminance image data A ′ and luminance image data B ′ from the memory 205, performs block matching using the two luminance image data A ′ and B ′, and performs parallax. Image data C is generated. The parallax image data C is generated for each pixel. Then, distance image data D is created from the parallax image data C. For each parallax value p of the parallax image data C, the distance image data D is for each pixel, with the focal lengths of the cameras A101 and B102 being F1, and the base length (distance between the optical axes of the cameras A101 and B102) being B1. Further, the distance value d is obtained by using the formula d = Fl * Bl / p. The distance image generation unit 207 stores the generated distance image data D in the memory 205.

  Note that when there are three or more cameras, a plurality of distance image data can be obtained. Therefore, it is necessary to merge each distance image data into one distance image data. For this purpose, a conventionally known method may be used (for example, Non-Patent Documents 2 to 4). Furthermore, if the distance image data can be generated, the stereo camera method may not be used. For example, the distance image data may be generated using a light cutting method or a pattern projection method.

  In the present embodiment, first, luminance image data A ′ and luminance image obtained by the camera A 101 and the camera B 102 photographing the installation surface 112 (background) in a state where the test object 10 is not installed on the installation surface 112. Distance image data (hereinafter referred to as background distance image data D1) is generated from the data B ′ and stored in the memory 205. Next, when the test object 10 is installed on the installation surface 112, the luminance image data A ′ and the luminance image data B obtained by the camera A101 and the camera B102 photographing the test object 10 and the installation surface 112 are displayed. The distance image data (hereinafter referred to as distance image data D2) is generated from 'and stored in the memory 205.

  The background / test object separation unit 208 takes the background distance image data D1 and the distance image data D2 from the memory 205, compares the distance values of the two, and separates the distance image data D2 into a background area and a test object area. To do. Specifically, the background / test object separation unit 208 compares the distance value of the background distance image data D1 and the distance value of the distance image data D2 for each pixel, and the absolute value of the difference between the distance values is predetermined. Pixels that are equal to or greater than the threshold value are separated from the test object region, and other pixels are separated from the background region.

  FIG. 5 to FIG. 7 are diagrams for explaining the background / test object region separation processing in the background / test object separation unit 208. Here, it is assumed that the distances from the camera A101 and the camera B102 to the installation surface 112 are 100 mm. Further, the test object 10 is a rectangular parallelepiped having a height of 20 mm.

  FIG. 5A shows luminance image data B ′ obtained by the camera B102 when the test object 10 is not installed on the installation surface 112. FIG. That is, the luminance image data B ′ is a blank sheet. The same applies to the luminance image data A ′ of the camera A101 (not shown). Actually, the luminance image data reflects a random pattern by the pattern projection unit 103, which is omitted for simplification of the drawing. The same applies to FIGS. 6A and 7A.

  FIG. 5B shows a background distance generated from the luminance image data A ′ and the luminance image data B ′ by the cameras A 101 and B 102 when the test object 10 of FIG. 5A is not installed on the installation surface 112. The image data D1 is shown. Here, each 升 represents a pixel, and the numerical value in 升 represents a distance value (mm).

  FIG. 6A is an example of luminance image data B ′ obtained by the camera B102 when the test object 10 is installed on the installation surface 112. FIG. The same applies to the luminance image data A ′ of the camera A101 (not shown). FIG. 6B shows distance image data D1 generated from the luminance image data A ′ and the luminance image data B ′ by the camera A 101 and the camera B 102 in the case of FIG. 6A.

  Similarly, FIG. 7A is another example of the luminance image data B ′ obtained by the camera B102 when the test object 10 is installed on the installation surface 112. FIG. The same applies to the luminance image data A ′ of the camera A101 (not shown). FIG. 7B shows the distance image data D1 generated from the luminance image data A ′ and the luminance image data B ′ by the camera A 101 and the camera B 102 in the case of FIG. 7A.

  For each pixel, the background / test object separation unit 20 determines the distance between the distance value of the background distance image data D1 (FIG. 5 (b)) and the distance image data D2 (FIG. 6 (b), FIG. 7 (b)). By comparing the values, pixels in the distance image data D2 whose absolute value of the difference between the distance values is equal to or larger than a predetermined threshold are set as the test object region, and other pixels are set as the background region. Here, the threshold value is 10. As a result, the distance image data D2 in FIG. 6B is separated into the test object region and the background region as shown in FIG. 6C. Similarly, the distance image data D2 in FIG. 7B is separated into a test object region and a background region as shown in FIG. 7C. In FIG. 6 (c) and FIG. 7 (c), the shaded portion represents the specimen region.

  In the present embodiment, the distance image data is separated into the background area and the test object area using the background distance image data and the distance image data. However, for each pixel of the distance image data, the distance value is a predetermined value. The pixels below the threshold may be separated from the test object region, and the other pixels may be separated from the background region. For example, in the case of FIG. 6B and FIG. 7B, by setting the threshold value to 85, separation can be performed as shown in FIG. 6C and FIG. 7C.

The position / rotation amount calculation unit 209 calculates moments M ₀₀ , M ₁₀ , M ₀₁ , M ₁₁ , M ₂₀ , M ₀₂ for the test object region of the distance image data D 2 separated by the background / test object separation unit 208. The position X and Y of the test object 10 and the rotation amount Θ are calculated using the moment.

Here, the following formula (1) is used to calculate the moments M ₀₀ , M ₁₀ , M ₀₁ , M ₁₁ , M ₂₀ , M ₀₂ .

x and y are the coordinates of the distance image data D2. src (x, y) is an image array that represents 1 when the coordinate (x, y) is the test object region, and 0 when the coordinate (x, y) is the background region.

From the moments M ₀₀ , M ₁₀ , M ₀₁ , M ₁₁ , M ₂₀ , M ₀₂ , the position (center of gravity position) X, Y and the rotation amount Θ of the test object 10 are calculated as follows.

When μ ₂₀ = μ ₀₂ , the rotation amount Θ is 0.

FIG. 8 shows the coordinates of the distance image data D2 in the case of FIG. 6 (when the test object is arranged horizontally). In FIG. 8, the shaded portion is the test object region, and the non-shaded portion is the background region. From equation (1), in FIG. 8, M ₀₀ = 30, M ₁₀ = 105, M ₀₁ = 120, M ₁₁ = 420, M ₂₀ = 455, M ₀₂ = 540. Thereby, the positions X and Y of the test object 10 are calculated as X≈3.5, Y≈4.0, and the rotation amount Θ is calculated as Θ = 0.

FIG. 9 shows the coordinates of the distance image data D2 in the case of FIG. 7 (when the test object is tilted). In FIG. 9, the shaded portion is the test object region, and the non-shaded portion is the background region. From equation (1), in FIG. 9, M ₀₀ = 28, M ₁₀ = 129, M ₀₁ = 118, M ₁₁ = 556, M ₂₀ = 706, M ₀₂ = 562. Thus, the positions X and Y of the test object 10 are calculated as X≈4.6, Y≈4.2, and the rotation amount Θ is calculated as Θ≈27.6.

In equation (1), src (x, y) = 0 in the background area of the distance image data. Therefore, in the calculation of the moments M ₀₀ , M ₁₀ , M ₀₁ , M ₁₁ , M ₂₀ , M ₀₂ , the coordinates of the test object region of the distance image data (shaded portions in FIGS. 8 and 9) are expressed as ). Thereby, the calculation processing amount and the calculation time are shortened.

  The position deviation / rotation deviation determination unit 210 determines a predetermined threshold (minX, maxX, minY, maxY, minΘ) for the position X, Y and the rotation amount Θ of the test object 10 calculated by the position / rotation amount calculation unit 209. , MaxΘ), the position / rotational deviation of the test object 10 is determined. That is, it is determined whether or not the test object is placed normally. Specifically, if minX ≦ X ≦ maxX, minY ≦ Y ≦ maxY, and minΘ ≦ Θ ≦ maxΘ, the determination result is 1 (OK), otherwise 0 (NG).

  The determination result in the positional deviation / rotation deviation unit 210 is output via the control unit 211 to the PLC connected from the PCI / F 203 to the personal computer 300 or from the external I / F 204 to the external communication unit 400.

  The controller 211 operates differently depending on the state. FIG. 10 shows the transition state of the control unit 211. In the initial state, it is in the background photographing state 1001. When the background shooting registration is completed, the state transitions to the threshold adjustment state 1002. When the determination threshold setting is completed, the operation state 1003 is transitioned to. Note that when initialization (initialization of a distance image and a determination threshold) is performed, the state moves to a shooting state 1001.

  FIG. 11 is a diagram illustrating an operation flow of the processing unit 200 in each state of the control unit 211.

  FIG. 11A is an operation flow when a trigger is input when the control unit 211 is in the background shooting state 1001. In the background photographing state 1001, the test object 10 is not yet installed on the installation surface 112.

  The control unit 211 issues a shooting instruction in synchronization with the cameras A101 and B102 (step 1101). Then, the image data (luminance image data A) of the background image captured by the camera A101 is transferred to the image conversion unit 206, and the image data (luminance image data B) of the background image captured by the camera B102 is also converted into the image conversion unit. 206. Then, the image conversion unit 206 generates luminance image data A ′ and luminance image data B ′ that have been made parallel by pixel distortion correction (step 1102). The luminance image data A ′ and the luminance image data B ′ are transferred to the distance image generation unit 207. The distance image generation unit 207 generates the parallax image data C using the luminance image data A ′ and the luminance image data B ′, and generates the distance image data D from the parallax image data C (step 1103). Here, background distance image data D1 is generated. The control unit 211 stores the background distance image data D1 in the memory 205 and transfers the luminance image data B ′ (background luminance image) to the personal computer 300 (step 1104). Note that the control unit 211 may transfer the luminance image data A ′ to the personal computer 300. In short, it is only necessary to know that it is a background image for the user.

  FIG. 7B is an operation flow when a trigger is input when the control unit 211 is in the threshold adjustment state 1002. In the threshold adjustment state 1002, the test object (model product) 10 to be subjected to the positional deviation / rotational deviation determination is correctly installed on the installation surface 112.

  The control unit 211 issues a shooting instruction in synchronization with the cameras A101 and B102 (step 1101). Then, image data (luminance image data A) of the test object image captured by the camera A101 is transferred to the image conversion unit 206, and image data (luminance image data B) of the test object image also captured by the camera B102. Is transferred to the image conversion unit 206. Then, the image conversion unit 206 generates luminance image data A ′ and luminance image data B ′ that have been made parallel by pixel distortion correction (step 1102). The luminance image data A ′ and the luminance image data B ′ are transferred to the distance image generation unit 207. The distance image generation unit 207 generates the parallax image data C using the luminance image data A ′ and the luminance image data B ′, and generates the distance image data D from the parallax image data C (step 1103). Here, distance image data D2 is generated. The distance image data D2 is transferred to the background / test object separation unit 208. Further, the background distance image data D1 stored in the memory 205 is also transferred to the background / test object separation unit 208. The background / test object separation unit 208 compares the background distance image data D1 and the distance image data D2, and separates the distance image data D2 into a background area and a test object area (step 1105). This separation result is transferred to the position / rotation amount calculation unit 209. The position / rotation amount calculation unit 209 calculates the moment of the object area of the distance image data D2, and calculates the position X, Y of the object (model product) 10 and the rotation amount Θ from this moment (step) 1106). The control unit 211 transfers the luminance image data B ′, information about the positions X and Y, and the rotation amount Θ to the personal computer 300 (step 1107). Also in this case, the luminance image data A ′ may be transferred to the personal computer 300 as the luminance image data (test object luminance image). In short, it is only necessary for the user to know the position / rotation state of the test object image.

  The personal computer 300 displays the transferred luminance image data B ′, position X, Y, and rotation amount Θ information on the display unit. Based on this, the user sets and inputs determination threshold values (minX, maxX, minY, maxY, minΘ, maxΘ). The personal computer 300 transfers the setting threshold value input to the processing unit 200, and the control unit 211 of the processing unit 200 holds the transferred determination threshold value in the memory 205.

  FIG. 11C is an operation flow when a trigger is input when the control unit 211 is in the operation state 1003. In the operation state 1003, the actual test object 10 to be inspected is installed on the installation surface 112 in an arbitrary manner.

  The control unit 211 issues a shooting instruction in synchronization with the cameras A101 and B102 (step 1101). Then, image data (luminance image data A) of the test object image captured by the camera A101 is transferred to the image conversion unit 206, and image data (luminance image data B) of the test object image also captured by the camera B102. Is transferred to the image conversion unit 206. Then, the image conversion unit 206 generates luminance image data A ′ and luminance image data B ′ that have been made parallel by pixel distortion correction (step 1102). The luminance image data A ′ and the luminance image data B ′ are transferred to the distance image generation unit 207. The distance image generation unit 207 generates the parallax image data C using the luminance image data A ′ and the luminance image data B ′, and generates the distance image data D from the parallax image data C (step 1103). Here, distance image data D2 is generated. The distance image data D2 is transferred to the background / test object separation unit 208. Further, the background distance image data D1 stored in the memory 205 is also transferred to the background / test object separation unit 208. The background / test object separation unit 208 compares the background distance image data D1 and the distance image data D2, and separates the distance image data D2 into a background area and a test object area (step 1105). This separation result is transferred to the position / rotation amount calculation unit 209. The position / rotation amount calculation unit 209 calculates the moment of the test object region of the distance image data D2, and calculates the position X, Y and the rotation amount Θ of the test object 10 from this moment (step 1106). The calculated data of the positions X and Y and the rotation amount Θ are transferred to the positional deviation / rotational deviation determination unit 210. The determination threshold values (minX, maxX, minY, maxY, minΘ, maxΘ) stored in the memory 205 are also transferred to the positional deviation / rotational deviation determination unit 210. The positional deviation / rotational deviation determination unit 210 compares the calculated positions X and Y and the rotation amount Θ with the determination threshold value to determine the positional deviation / rotational deviation of the test object 10 (step 1108). Then, the determination result (OK / NG) is notified to the control unit 211. The control unit 211 transfers the luminance image data B ′ as a result of the determination to the PC 300 or the external communication unit 400 (step 1109).

  The specific operation of the inspection apparatus including the image processing apparatus according to this embodiment will be described below.

  FIG. 12 is a diagram showing a flow of the overall operation of the present inspection apparatus together with user operations. In FIG. 12, shaded parts indicate user operations. FIG. 13 is a diagram showing the transition of the display screen of the personal computer 300.

  Assume that the camera A101, the camera B102, and the pattern projection unit 103 are installed as shown in FIG. As described above, the pattern projection unit 103 may be omitted. Initially, the test object 10 is not installed.

  The user activates a dedicated application for determining the displacement / rotation deviation of the test object on the personal computer 300 (step 2001). At this time, the display screen of the personal computer 300 is as shown in FIG. Further, the control unit 211 of the processing unit 200 is in the background photographing state 1001 in the initial state (FIG. 10).

  Next, the user presses the background shooting button 502 (step 2002). Then, a trigger is input from the PC 300 to the control unit 211 through the PCI / F 203. As a result, the processing unit 200 executes the processing operation shown in FIG. 11A (step 2003) and outputs luminance image data (background luminance image) B ′ to the personal computer 300. At this time, the luminance image data B 'is displayed in the captured image display area 501 of the display screen of the personal computer 300, but nothing is displayed because the test object is not installed. That is, the display screen is the same as in FIG. In practice, the pattern (FIG. 3) by the pattern projection unit 103 is displayed, but is omitted here for the sake of simplicity. After transferring the luminance image data B ′, the control unit 211 transitions to the threshold adjustment state 1002 (FIG. 10).

  Next, the user installs the test object (model product) 10 for which the positional deviation / rotational deviation is to be determined immediately below the camera A101 and the camera B102 (step 2004). At that time, the test object 10 is installed at a position included in the field of view of both the camera A101 and the camera B102. Moreover, the test object 10 is installed so that there is no position shift and rotation shift.

  The user presses the shooting button 503 (step 2005). Then, a trigger is input from the PC 300 to the control unit 211 through the PCI / F 203. As a result, the processing unit 200 executes the processing operation shown in FIG. 11B (step 2006), and outputs luminance image data (test object image) B ′ and position / rotation amount information to the personal computer 300. At this time, the display screen of the personal computer 300 is as shown in FIG. That is, the image of the test object 10 is displayed in the projection image display area 501. The calculated positions X and Y and the rotation amount Θ are displayed on a position bar (X) 505, a position bar (Y) 506, and a rotation bar 507. Here, position X = 0, position Y = 0, and rotation amount Θ = 0.

  The user operates the position bar (X) 505, the position bar (Y) 506, and the rotation bar 507 using a mouse or the like, and uses the X positional deviation allowable amount (minX, maxX) and the Y positional deviation allowable amount as determination thresholds. (MinY, maxY) and allowable rotational deviation (minΘ, maxΘ) are set (step 2007). At this time, the display screen of the personal computer 300 is as shown in FIG.

  When the setting of the determination threshold is completed, the user presses the adjustment completion button 504 (step 2008). Then, determination threshold values (minX, maxX, minY, maxY, minΘ, maxΘ) set by the user are notified from the personal computer 300 to the processing unit 200 through the PCI / F 203. The control unit 211 holds the determination threshold value in the memory 205 and transitions to the operation state 1003 (FIG. 10).

  Thereafter, the test object 10 to be actually inspected is installed, and when the user presses the photographing button 503, a trigger is input from the PC 300 to the control unit 211 through the PCI / F 203. As a result, the processing unit 200 executes the processing operation shown in FIG. 11C (step 2009), and outputs the luminance image data (test object image) B ′ and the determination result to the personal computer 300. FIG. 13-D shows an example of the display screen of the personal computer 300 at this time. This indicates that the positions X and Y of the test object 10 are within the allowable range, but the determination result is “NG” because the rotation amount Θ exceeds the allowable range.

  In addition, by inputting a shooting trigger from the PLC connected to the external communication unit 400, the processing unit 200 similarly executes the processing operation shown in FIG. 11C, and passes through the external I / F 204 and the external communication unit 400. The determination result can be obtained by PLC.

  Next, another embodiment of the stereo camera will be described. In FIG. 1, the camera fixing jig 100 is a stereo camera with the camera A101 and the camera B102 spaced apart from each other. However, the present invention is not limited to such a configuration.

  FIG. 14 is a diagram illustrating a configuration example of an integrated stereo camera in which two cameras that can be used in place of the camera A 101 and the camera B 102 are integrated. This is a cross-sectional schematic diagram of a configuration in which a subject is present in the direction of the arrow and the subject is imaged by an integrated stereo camera. Reference numeral 601 denotes a lens array. The lens array is composed of two surfaces, a surface on the object side and a surface on the image surface side, and a plurality of lenses are arrayed in the surface. FIG. 14 shows a double-sided lens array in which lens surfaces are provided on both the object side and the image side. Reference numeral 6001a denotes a lens provided on the object-side surface, and reference numeral 601b denotes a lens provided on the image-side surface. 601a and 601b are combined to form an image of the object on the image surface. Reference numeral 602 denotes a light-shielding partition wall (referred to as a light-shielding wall) for preventing crosstalk (crosstalk) of light rays between adjacent lens sets in the lens array, and is opaque to imaging light such as metal or resin. Made of material. The image side surface of the lens array 601 and the light shielding wall are bonded. Reference numeral 603 denotes an aperture array in which a circular hole is provided in a plate-like member corresponding to each lens set, and acts as a lens diaphragm. The aperture array 603 and the lens array 601 are bonded to each other through a protrusion 607 provided on a plane portion of the surface on the subject side of the lens array 601. Reference numeral 604 denotes a CMOS sensor (imaging device) that captures an image of a subject captured by each lens set in the lens array, and is mounted on a substrate 605. Reference numeral 606 denotes a housing, which is bonded to the subject-side surface of the lens array 601 to hold the lens array 601, the light shielding wall 602, and the aperture array 603, and is bonded to the substrate 605.

  When the integrated stereo camera having the configuration shown in FIG. 14 is used, an image having parallax corresponding to the lens array 601 is captured by the CMOS sensor 604. According to the configuration of FIG. 10, the stereo camera can be downsized although the distance measurement accuracy is low.

  FIG. 15 is a diagram showing another configuration example of an integrated stereo camera in which two cameras are integrated. The basic configuration is the same as that shown in FIG. 14, and the difference is that the CMOS sensor (imaging device) is composed of two pieces 604a and 604b. According to the configuration of FIG. 15, the number of sensors increases, so that the cost increases, but there is an advantage that the ranging accuracy is improved.

DESCRIPTION OF SYMBOLS 10 Test object 101,102 Camera 103 Pattern projection part 200 Processing unit 201,202 Camera I / F
203 PCI / F
204 External I / F
205 memory 206 image conversion unit 207 distance image generation unit 208 background / test object separation unit 209 position / rotation amount calculation unit 210 position deviation / rotation deviation determination unit 211 control unit 300 personal computer 400 external communication unit

JP 2011-107878 A

P. J. Besl and N. D. McKay, A method for registration of 3-D shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14 (2) 1992, pp. 239-256. Greg Turk and Marc Levoy. Zippered polygonmeshes from range images. In Proc. SIGGRAPH '94, pp. 311-318. ACM, 1994. Curless, B., Levoy, M., A Volumetric Method for Building Complex Models from Range Images, In Proc. SIGGRAPH'96, ACM, 1996, pp. 303-312. M. Wheeler, Y. Sato, and K. Ikeuchi, Consensussurfaces for modeling 3D objects from multiplerange images, In Proc. IEEE International Conference on Computer Vision, pp. 917-923, Jan, 1997.

Claims (6)

Distance image generating means for generating distance image data from image data obtained by imaging the test object;
A background / test object region separating means for separating the distance image data into a background region and a test object region;
Position / rotation amount calculating means for calculating the position and rotation amount of the test object from the test object region of the distance image data;
An image processing apparatus comprising: The distance image means generates distance image data from background distance image data and image data obtained by photographing the subject from the background image data obtained by photographing in the absence of the subject,
The background / test object region separating means compares the background distance image data and the distance image data, and separates the distance image data into a background region and a test object region,
The image processing apparatus according to claim 1. The position / rotation amount calculating means calculates a moment of the test object region of the distance image data, and calculates a position / rotation amount of the test object from the moment.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Imaging means for imaging a test object from a plurality of directions and acquiring a plurality of image data having different parallaxes;
Distance image generation means for generating distance image data from the plurality of image data;
A background / test object region separating means for separating the distance image data into a background region and a test object region;
Position / rotation amount calculating means for calculating the position and rotation amount of the test object from the test object region of the distance image data;
A positional deviation / rotational deviation determining means for comparing the calculated position and rotation amount of the test object with a predetermined threshold value to determine a position error and a rotation error of the test object,
An inspection apparatus comprising: The imaging means is approximately provided by each of a lens array provided with a plurality of lenses arranged in an array, and a plurality of lenses provided on the image plane side of the lens array. An image pickup device for picking up a reduced image of the object to be imaged,
The inspection apparatus according to claim 4.   The inspection apparatus according to claim 4, wherein the imaging unit further includes a pattern projection unit that irradiates a random pattern over a wider area than the imaging region of the test object.