JP2011191170A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a work for confirming the relation between an actual object and an imaging means and a work for comparing the object with an image processing result, to thereby readily carry out the operation without the use of a monitor. <P>SOLUTION: In an imaging section 1, a light-projecting section 11 includes a LCD 12, and is disposed as to be coaxial with a CCD 10, so that an image displayed on a LCD 12 can be projected toward a visual field of the CCD 10. In a processing section 2, a defect D in a workpiece W0 is detected, by using the image generated by the CCD 10 to generate a projection image 50, by setting an image data brighter than an ambience to an area RP corresponding to an area R, including the defect, provided to the projecting section 11, and projected. A range detected as the defect by image processing is defined by a marking pattern M by the projected image in the area RP among a surface of the workpiece W0 with regard to the projection. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus used for the purpose of measuring a plurality of articles whose sizes and shapes are unified, such as a production site, and inspecting articles based on the measurement results.

  As this type of image processing apparatus, one that performs two-dimensional measurement processing using an image generated by one camera, and one that performs three-dimensional measurement using stereo images generated by a plurality of cameras There is. In any type of image processing apparatus, it is necessary to match the range of the visual field of the camera and the direction of the optical axis so that the object can be correctly imaged before use.

  In a conventional camera adjustment process, an image generated by the camera is displayed on a monitor, and the position and orientation of the camera are adjusted so that an optimal imaging state is obtained while comparing the monitor display with the actual object. .

  Further, even when the adjustment is completed and the apparatus is used, the processing result is appropriately displayed on the monitor so as to facilitate the user. As a conventional example of this display, for example, in Patent Document 1, three-dimensional measurement is performed on a specific point or region in a stereo image, and the measurement result is associated with the entire image of the measurement object to create a three-dimensional image. It is described that it displays automatically.

  In the invention described in Patent Document 2, an inspector who conducts a visual inspection is allowed to wear a head mounted display provided with a CCD and a sensor for detecting a line of sight. Further, in Patent Document 2, an image generated by a CCD is displayed on a display as a visual image, and a defect is detected by processing an image when the gaze sensor detects that the gaze of the inspector is fixed. It is described that a frame image that detects and indicates the position of the defect is superimposed on the visual image and displayed.

JP 2009-53147 A JP 2009-150866 A

  When the processing result is displayed on the monitor as in Patent Document 1, the user on site confirms the suitability of the processing result by comparing the actual object with the display. However, when the installation location of the monitor is far from the location where imaging is performed, the confirmation becomes difficult and there is a risk of hindering the work.

  According to the invention described in Patent Document 2, the user can easily compare the image of the target object with the image processing result, but can only compare the images to the end, and can compare the actual object with the image processing result. It will not be verified.

  In addition, in the invention described in Patent Document 2, the user needs to wear a special device (head mounted display). However, as in Patent Document 2, the user is restricted to the site. This is the case where it is assumed that It is difficult to apply this method to applications that do not require the user to constantly look at the object to be processed, such as automatic inspection for products flowing through the production line and three-dimensional measurement processing in a picking system.

  The present invention pays attention to the above-mentioned problems, and does not use a monitor to perform an operation of confirming a relationship between an actual object and an imaging unit or an operation of comparing an actual object and an image processing result. It is an object to be able to easily perform it at the site where the

  According to the present invention, an image pickup unit having a two-dimensional image pickup device and a two-dimensional image generated by the image pickup device (hereinafter referred to as “captured image”) are input from the image pickup unit, and predetermined using the input image. The present invention is applied to an image processing apparatus having image processing means for executing image processing and measurement processing.

  An image processing apparatus to which the present invention is applied includes a display element for displaying a two-dimensional digital image, an image projection unit that projects an image displayed on the display element toward a field of view of an imaging unit, And a projection control means for providing the image projection means with a digital image including a two-dimensional pattern in which image data having different brightness or color is set to the image projection means. In addition, the imaging unit and the image projecting unit are configured so that the imaging target range of the imaging element in the imaging plane of the image projected from the image projecting unit is included in the range in which the image from the image projecting unit is projected. The relationship is adjusted and a two-dimensional pattern is projected into the imaging target area.

  According to the above configuration, when the surface of the object is substantially aligned with the imaging plane of the image projected from the image projecting means, a clear image can be projected onto the surface of the object. It becomes possible. In addition, since a two-dimensional pattern in which image data different in brightness or color from the surroundings is set within the imaging target range of the imaging device is projected, among the surfaces of the objects included in the range imaged by the imaging device, It is possible to clearly indicate the location where the two-dimensional pattern is projected by the projection image of the pattern. Furthermore, since the imaging target range of the imaging device is included in the range in which the projected image is formed, the above two-dimensional pattern can be projected at any position in the imaging target range.

  Therefore, it is possible to perform a projection process such as projecting a pattern representing a relationship with the field of view of the imaging unit or a pattern indicating a part detected by the process on the captured image onto an actual object. This eliminates the need to confirm the captured image by the display on the monitor.

  In a preferred embodiment of the image processing apparatus, the projection control means determines a pixel for setting the image data of the two-dimensional pattern based on a processing result of the image processing means, and outputs an image projected on the image projection means. Generate. In this way, for example, when a part having a predetermined feature is detected by image processing, projection processing is performed by setting image data with high brightness to the pixel on the display element side corresponding to the detection range. As a result, the range detected by the image processing on the actual surface of the object can be clearly indicated by a bright projection pattern. In addition, even if the position, shape, and size of the detected part are variously changed, it is possible to easily cope with it.

  An image processing apparatus according to another preferred embodiment includes an image registration unit for registering an image including the two-dimensional pattern, and reads one of the images registered in the image registration unit as a display element. By displaying, the projection processing of the image is executed.

  According to the above-described embodiment, for example, by registering an image including a two-dimensional pattern representing the center position of the field of view of the imaging unit and the measurement target range in the image registration unit, the position and optical axis of the imaging unit can be determined. When performing the matching operation, it is possible to perform the operation without confirming the display on the monitor by projecting the registered image. Also, if a plurality of images are registered and the user selects an image necessary for the work from among them, or an image created by the user according to his / her own use is registered, the convenience is further improved. be able to.

  In the image processing apparatus according to another preferred embodiment, the image pickup unit and the image projection unit form a coaxial optical system and are accommodated in the same casing. Further, the image pickup means and the image are projected so that image data from different pixels of the display element is projected to each point corresponding to each pixel on the image pickup element side in the imaging plane of the image projected from the image projection means. The relationship with the projection means is adjusted.

  According to the above configuration, the optical axes of the imaging unit and the image projecting unit are aligned on the same axis, and the correspondence between the imaging element and the display element is clarified. Therefore, it is possible to easily specify a pixel to be projected onto a location to be clearly specified within the imaging target range and set a projection image.

  In the image processing apparatus according to another preferred embodiment, the image projecting unit is formed separately from the image capturing unit, and corresponds to each pixel on the image sensor side in the imaging plane of the image projected from the image projecting unit. The image sensor is arranged in a state in which the relationship with the imaging unit is adjusted so that image data from different pixels of the display element is projected to each point. The image processing unit executes a three-dimensional measurement process using the image input from the imaging unit. Further, the projection control means perspectively converts the three-dimensional information based on the result of the three-dimensional measurement processing of the image processing means to the display surface of the display element, and uses the perspective conversion result to convert the two-dimensional pattern to the pixel representing the three-dimensional measurement result. An image in which the image data is set is generated, and the image is projected by displaying the image on a display element.

  In this embodiment, it is desirable to configure the imaging means as a stereo camera using a plurality of two-dimensional cameras and perform three-dimensional measurement using the stereo cameras.

  According to the configuration of the above embodiment, for example, based on the three-dimensional information acquired by the three-dimensional measurement process, a three-dimensional pattern indicating the contour shape of the measurement target part is generated, and this is perspective-transformed on the surface of the display element. By doing so, it is possible to obtain a two-dimensional pattern that represents the contour shape of the measurement target site as viewed from the display element. Accordingly, if a projection image is generated by projecting the two-dimensional pattern representing the contour shape into a pattern in which image data brighter than the surroundings is set, the user can project the projected contour pattern and the contour line of the actual object. It is possible to recognize whether or not the result of the three-dimensional measurement is appropriate.

  According to the present invention, the relationship between the field of view of the imaging means and the object is confirmed by the two-dimensional pattern projected within the imaging object range of the actual object, and the result of the image processing and the actual object state Therefore, the user can easily perform the confirmation work without looking at the display on the monitor. In addition, since a two-dimensional pattern can be projected to any location within the imaging target range of the imaging device, and the shape and size of the pattern can be freely set, a highly convenient apparatus is provided. be able to. In addition, since the user does not need to wear a visual recognition tool and only needs to visually recognize an actual object as necessary, it can be used for image processing for various purposes.

It is a figure which shows the structural example of the image processing apparatus to which this invention is applied. It is a figure which matches and shows the structure of the optical system of an imaging part, and the functional block diagram of a process part. It is a figure which matches and shows the relationship of the optical system with respect to the defect of a workpiece | work, and the process sequence which a process part performs. It is a figure which shows the relationship between the optical system of each image pick-up part and a workpiece | work in the case of aligning the viewpoint of two image pick-up parts, and the process sequence which a process part performs. It is a figure which matches and shows the relationship between the optical system and workpiece | work of each imaging part in the case of aligning the measurement object area | region of two imaging parts, and the process procedure which a process part performs. It is a figure which shows the structure of the image processing apparatus of the type by which the imaging part and the projection part were made into a different body. FIG. 7 is a diagram illustrating a relationship between an optical system and a workpiece of the image processing apparatus in FIG. 6 and a processing procedure executed by a processing unit.

FIG. 1 shows an example of use of an image processing apparatus to which the present invention is applied.
The image processing apparatus according to this embodiment has defects such as scratches and dirt visually recognized on the surface of a finished product or a part of a part of the product (shown as “work W0” in this embodiment) at the production site. The image pickup unit 1 is disposed above the workpiece W0 to be inspected, and the processing unit 2 includes a computer. The processing unit 2 is connected to a monitor and a setting operation unit (keyboard, mouse, etc.) as necessary (the same applies to other embodiments).

  The processing unit 2 determines the loading of the workpiece W0 to the imaging position based on a detection signal from a workpiece detection sensor (not shown), and drives the imaging unit 1 accordingly. Thus, when an imaging process is performed and a captured image is input to the processing unit 2, the processing unit 2 executes image processing for defect detection on the captured image. This image processing includes processing for extracting defect features, such as binarization and edge extraction, and processing for measuring the extracted features.

  Further, the processing unit 2 determines the presence / absence of a defect based on the result of the image processing, outputs the determination result to an external device (not shown), and generates an inspection result image to be described later.

  Furthermore, in the image processing apparatus according to this embodiment, when a defect is detected, the imaging unit 1 is placed at a corresponding portion of the actual workpiece W0 so that a worker on site can easily check the detected portion and the state of the defect. The light with high brightness is projected and the location is clearly indicated.

FIG. 2 shows the configuration of the optical system of the imaging unit 1 of the image processing apparatus and the functional block diagram of the processing unit 2 in association with each other. FIG. 2 and the next FIG. 3 show a workpiece W0 having a defect D on its surface.
As shown in FIG. 2, the imaging unit 1 is provided with a coaxial optical system including a lens 14, a CCD 10, a projection unit 11, and a half mirror 15. In addition, although not shown, the imaging unit 1 includes a circuit for driving the CCD 10 and the projection unit 11, an output circuit for an image signal, and the like.

  The projection unit 11 includes a liquid crystal panel (LCD) 13 and a light source 14 for backlight. The LCD 12 is arranged at the focal position of the lens 14 as in the CCD 10. In this embodiment, the LCD 12 and the CCD 10 have the same vertical and horizontal sizes (number of pixels) and the same resolution.

  The surface of the workpiece W0 is irradiated with light from external illumination (not shown). In FIG. 2, it is assumed that the surface of the workpiece W0 is aligned with the focal position of the lens 14, and the optical path of the reflected light from one point on the surface is indicated by a one-dot chain line. As indicated by the alternate long and short dash line, light incident on the lens 14 from the workpiece W0 is guided to the CCD 10 via the half mirror 16, and an image representing the surface state of the workpiece W is generated by the CCD 10 receiving the light.

  In the projection unit 11, when a two-dimensional image is displayed on the LCD 12 under backlight illumination from the light source 13, the light from each pixel in the image passes through the half mirror 15 and the lens 14 and the field of view of the lens 14. Flooded within range. As a result, the image displayed on the LCD 12 is projected onto the surface of the workpiece W0.

  In the processing unit 2, functions of an imaging processing unit 21, an inspection execution unit 22, a projection image generation unit 23, and a projection operation control unit 24 are set. The imaging processing unit 21 drives the CCD 10 and inputs the generated captured image. The inspection execution unit 22 receives the above-described captured image supplied from the imaging processing unit 21 and executes image processing for defect detection and processing for determining the presence or absence of a defect. Further, the process of outputting the discrimination result to the external device and the process of generating the test result image are also executed by the test execution unit 22.

  The projection image generation unit 23 is activated in response to the inspection execution unit 22 detecting the defect D and generating an inspection result image, and generates a projection image to be displayed on the LCD 12. The projection operation control unit 24 outputs the projection image generated by the projection image generation unit 23 to the projection unit 11 and performs the above-described projection processing.

  In this embodiment, the CCD 10 and the LCD 12 have the same number of pixels and the same resolution and are arranged at the focal point of the lens 14, so that the image of the CCD 10 is captured on the image formation plane of the image projected from the LCD 12. The target range and the projection range of the image from the LCD 12 are completely the same. That is, between the CCD 10 and the LCD 12, the pixels at the same coordinates correspond to each other. Therefore, when the surface of the workpiece is aligned with the in-focus position of the lens 14, the same coordinate (on the LCD 12 side) with respect to an arbitrary point on the workpiece W0 that appears at the coordinates (x, y) in the captured image. Image data can be projected from x, y).

  In this embodiment, by utilizing the characteristics of the optical system described above, a pattern image (hereinafter referred to as a “marking pattern”) having a high luminance color at a location where a defect is detected by image processing in the actual workpiece W0. Is projected). By this projection, an on-site worker can visually recognize the brightly illuminated part of the actual work as a part where a defect is detected, so that the suitability of the inspection result and the degree of the defect can be easily confirmed. .

  FIG. 3 shows an optical path for the defect D of the workpiece W0 for the optical system of the imaging unit 1, and shows a flowchart of a schematic procedure of processing performed in the processing unit 2 in association with an example of an image generated during the processing. . Hereinafter, processing executed in the processing unit 2 and the imaging unit 1 will be described with reference to FIG.

  When the workpiece W0 to be inspected is carried in, the processing unit 2 drives the CCD 10 by the imaging processing unit 21, inputs the generated image (step A), and performs image processing (measurement processing or (Including discrimination processing) is executed (step B). Further, when the defect D is detected, the inspection execution unit 22 generates an inspection result image 40 indicating the detection result.

  The inspection result image 40 shown in FIG. 3 is based on a captured image, and a rectangular region R is set in a range surrounding the defect D1 in the image. Hereinafter, this region R is referred to as “detection region R”.

Based on the characteristics of the optical system described above, the projection image generation unit 23 sets a color region RP (for example, a red region) with a high luminance in the same range as the detection region R in the inspection result image 40, and sets other pixels. A projection image 50 in which a dark color (for example, black) is set is generated (step C).
The projection operation control unit 24 gives the projection image 50 to the projection unit 11 to execute a projection process (step D). Thereby, light from the color set in the region RP is projected to the work W0 from each pixel in the region RP in the projection image 50, but almost light is emitted from the pixels set with other dark colors. Is not flooded. Therefore, the marking pattern M based on the projection image of the region RP is projected on the range corresponding to the detection region R of the actual workpiece W0, and this range is clearly indicated. The projection range of the marking pattern M corresponds to a portion that appears in the detection region R in the inspection result image 50. Therefore, if an actual defect D is included in the marking pattern M, the inspection result is correct.

  According to the above projection processing, the position and size of the marking pattern M can be freely changed based on the result of the image processing, so that the position, size, shape, etc. of the detected defect are determined by the individual workpiece W0. It is possible to clearly indicate the defect detection range on the surface of the actual work W0 without any problem. Therefore, even if a monitor is not provided near the site where imaging is performed, the operator can confirm the inspection result without any trouble.

  In the above embodiment, the size and resolution of the CCD 10 and the LCD 12 are unified to simplify the description. However, even if they are different, the image formation plane of the projection image from the LCD 12 is used. The imaging target range of the CCD 10 is included in the imaging range of the projection image, and image data from different pixels of the LCD 12 is projected to each point corresponding to each pixel of the CCD 10 in the imaging target range. If the relationship is established, the same projection processing as described above can be performed based on the correspondence relationship of the pixels between the CCD 10 and the LCD 12.

  Further, the above-described projection processing is not limited to the purpose of showing the defect detection result. For example, in a process performed manually, it is also possible to image a workpiece that has been carried in, detect a work target part by image processing, and project a marking pattern onto the detected target part. According to such processing, the worker can easily recognize the work target part and perform the work efficiently.

  Furthermore, since the image processing apparatus having the above configuration can generate and project an image including various two-dimensional patterns, the imaging unit 1 is used before full-scale use, as shown in FIGS. Can be used to adjust the field of view and the direction of the optical axis.

  4 and 5, the workpiece W1 is different by using two imaging units 1 having the same configuration as the embodiment of FIGS. 1 to 3 for the workpiece W1 larger than the example of FIGS. Take an image from the viewpoint. 4 and 5, the configuration of the imaging unit 1 is indicated by the same numerals as those shown in FIGS. 2 and 3 with a and b added thereto, and processing executed by the processing unit 2 for each imaging unit 1 a and 1 b. The flowchart showing the procedure is associated with the projection image.

  Although not shown in FIGS. 4 and 5, the imaging units 1 a and 1 b are connected to the same processing unit 2, and the processing unit 2 uses captured images input from the imaging units 1 a and 1 b. Three-dimensional measurement processing or two-dimensional image processing with different contents for each image is executed. In the processing unit 2, the functions of the imaging processing unit 21, the projection image generating unit 23, and the projection operation control unit 24 shown in FIG. 2 are set for each of the imaging units 1a and 1b.

  The configuration of the apparatus is not limited to the above. For example, each imaging unit 1a, 1b is connected to a separate processing unit 2, and processing is performed while communicating between the processing units 2 and 2, or each processing unit It is also possible to output the processing results from 2 and 2 to the host device and to secondary-process each processing result by the host device.

  In the embodiment of FIG. 4, a projection function is used to align the center point of the field of view of each imaging unit 1a, 1b with one point on the surface of the workpiece W1.

  More specifically, in this embodiment, projection images 51a and 51b including cross-shaped patterns Pa and Pb are generated for each of the imaging units 1a and 1b (steps C1 and C2), and these projection images are displayed. Projection processing is performed by giving the projection units 12a and 12b (D1, D2). The central part (cross cross part) of each pattern Pa, Pb corresponds to the pixel at the central part of the CCDs 10a, 10b, and any of the projection images 51a, 51b has a pixel other than the constituent pixels of the patterns Pa, Pb. A dark color is set as the background color. The shapes and sizes of the patterns Pa and Pb in the projection images 51a and 51b are the same, but the colors are different. For example, the pattern Pa of the image 51a is set to red, and the pattern Pb of the image 52b is set to blue.

  According to the above, by projecting the projection images 51a and 51b from the projection units 11a and 11b of the imaging units 1a and 1b, a cross formed by the projection images of the patterns Pa and Pb on the surface of the workpiece W. Mold marking patterns Ma and Mb are projected. Since these marking patterns Ma and Mb represent the center of the field of view of the imaging units 1a and 1b, respectively, the operator can adjust the center of each marking pattern Ma and Mb at the target position. The viewpoints of the imaging units 1a and 1b can be adjusted by adjusting the positions and orientations of the imaging units 1a and 1b.

The embodiment of FIG. 5 uses a projection function for the purpose of aligning the measurement ranges by the imaging units 1a and 1b with respect to the workpiece W1 similar to FIG.
Also in this embodiment, the processing unit 2 executes the same steps C1, D1, and C2, D2 as those in the example of FIG. 4 for each of the imaging units 1a and 1b. In this example, the processing unit 2 is set to a captured image. Projection images 52a and 52b in which rectangular patterns Sa and Sb with high-brightness colors are set in a range corresponding to the measurement target region are generated. Further, in this embodiment, similarly to the example of FIG. 4, different colors are set for the rectangular patterns Sa and Sb of the projection images 52a and 52b.

  According to the above processing, the projection images 52a and 52b are projected from the projection units 11a and 11b, so that the projection images Ta and Tb of the rectangular patterns Sa and Sb are measured by the imaging units 1a and 1b. It can project on the workpiece | work W1 as a marking pattern which shows an object area | region. Thereby, the operator can set an optimal measurement state by adjusting the direction of the optical axis of the imaging unit so that the overlapping range of the marking patterns Ta and Tb of each color is as wide as possible.

  In the embodiment of FIG. 5, the image data generated by the pixels on the outer periphery of the CCDs 10a and 10b is excluded from the measurement target in consideration of the influence of noise and the like, and accordingly, the projection images 52a and 52b. However, although dark areas are set around the rectangular patterns Sa and Sb, when it is desired to clearly indicate the entire range corresponding to the field of view of the imaging units 1a and 1b, all the pixels of the projection images 52a and 52b are set. Projection may be performed by setting image data with high luminance.

  Next, in the embodiments of FIGS. 4 and 5, the image for projection registered in advance is read out and projected without being based on the image processing for the captured image, but in these embodiments as well, the image is captured during the projection processing. The generated captured image may be processed, and the state of each projection image may be changed based on the processing result.

  For example, the embodiment of FIG. 5 will be described. A range in which the marking patterns Ta and Tb are overlapped from the captured image by performing imaging with either of the CCDs 10a and 10b in a state where the projection images 52a and 52b are projected. When the area of the overlapping range exceeds a predetermined reference value, a process for unifying the colors of the patterns Sa and Sb can be performed. In this way, even a worker unfamiliar with the setting work can measure the relationship between the imaging units 1a and 1b by performing the alignment work until the colors of the marking patterns Ta and Tb projected onto the work W1 are unified. It becomes possible to set to an optimal state.

  In the example of FIGS. 4 and 5, the projection function is used for the purpose of aligning the viewpoints and measurement target areas of the two imaging units 1a and 1b, but a single imaging unit as in the first embodiment. Even when 1 is used, the same projection processing can be performed to assist the work of aligning the position of the viewpoint of the imaging unit 1 with respect to the workpiece and the optical axis direction.

  In the above description, FIGS. 4 and 5 have been described as different embodiments. However, a plurality of types of projection images including four types of projection images 51a, 51b, 52a, and 52b applied to these embodiments. May be registered in the processing unit 2 and a projection image corresponding to the purpose may be selected from these projection images and used for the projection processing. Furthermore, if an image created by image creation software by a user can be registered as a projection image, the degree of freedom increases and the image can be used for purposes other than the alignment of the imaging unit 1. For example, when a work is manually placed at a measurement target position, a marking pattern can be projected onto a specific location of the work for the purpose of notifying the operator that imaging and image processing have been completed.

  Next, FIG. 6 shows a configuration example of an image processing apparatus of a type in which a projection unit and an imaging unit are separated. In FIG. 6, the imaging unit is indicated as 100, the projection unit as 110, and the processing unit as 200. The imaging unit 100 includes only members related to the imaging function, such as the CCD 101 and the lens 102, and the projection unit 110 includes an LCD 111, a light source 112, and a projection lens 113. In this embodiment, a cubic work W2 is shown as an example of the measurement object.

  Although not shown in FIG. 6, in this embodiment, a plurality of imaging units including the imaging unit 100 in the figure are provided (hereinafter, the imaging unit 100 in the figure is applied to all imaging units). .) The processing unit 200 executes a three-dimensional measurement process using the captured image input from each imaging unit 100, and includes an imaging processing unit 201, a three-dimensional measurement processing unit 202, a projection image generation unit 203, and a projection operation control. Functions of the unit 204 and the calibration processing unit 205 are set.

  The imaging processing unit 201 drives the CCD 101 of each imaging unit 100 at the same timing, and inputs a captured image from each CCD 101. The three-dimensional measurement processing unit 202 executes a three-dimensional measurement process on the workpiece W2 using these captured images. The projection image generation unit 203 generates a projection image based on the result of the three-dimensional measurement process. The projection operation control unit 204 receives the projection image from the projection image generation unit 203 and outputs the projection image to the LCD 111 of the projection unit 110, thereby projecting the image toward the field of view of the imaging unit 100.

  Prior to the measurement process, the calibration processing unit 205 operates in accordance with an instruction from the operator who performs the calibration work, and executes a calibration calculation for obtaining a parameter for three-dimensional measurement for each of the plurality of imaging units 100. The parameter derived by is registered in the internal memory. Further, the calibration processing unit 205 performs a calibration calculation on the LCD 111 of the projection unit 110 to obtain parameters necessary for specifying pixels corresponding to the three-dimensional coordinates obtained by the three-dimensional measurement. Perform the operation.

Here, the calibration process for the LCD 110 will be briefly described.
In the processing unit 200 of this embodiment, a projection image indicating a calibration pattern having a configuration in which a plurality of feature points are arranged according to a specific rule is registered. The operator places a screen at the position to be imaged after calibration for each imaging unit 100 is completed, and instructs the processing unit 2 to read out the projection pattern projection image and output it to the projection unit 110. Perform the operation. In response to this operation, the projection image generation unit 203 and the projection operation control unit 204 are activated to execute the above-described projection image reading and projection processing. At this time, the operator adjusts the position and orientation of the projection unit 110 so that a sufficient number of feature points appear clearly in the field of view of each imaging unit 100.

  When the above adjustment is completed, the operator performs an operation to instruct the processing unit 200 to execute the calibration calculation. By this operation, processing by the cooperation of the imaging processing unit 201, the three-dimensional measurement processing unit 202, and the calibration processing unit 205 is started. First, three-dimensional measurement using an image from each imaging unit 100 is performed. The three-dimensional coordinates of a plurality of feature points that are commonly included in the field of view of the imaging unit 100 are measured. The calibration processing unit 205 associates these three-dimensional coordinates with the coordinates of the feature points in the original projection image, and uses these to relate the measured values of the three-dimensional coordinates and the two-dimensional coordinates on the LCD 111. Deriving a parameter representing. The derived parameters are registered in the internal memory.

  FIG. 7 is a flowchart showing an outline of processing executed in the image processing apparatus in which the registration processing is completed, a relationship of the optical system of the imaging unit 100 with respect to the workpiece W2, and a procedure of processing executed in the processing unit 200. Indicated by

  In the processing unit 200 of this embodiment, first, the imaging processing unit 201 inputs a captured image from each imaging unit 100 (step A3) and executes a three-dimensional measurement process by the three-dimensional measurement processing unit 202 (step B3). . In the example of FIG. 7, it is assumed that the three-dimensional coordinates of the edge portion of the workpiece W2 are measured by the three-dimensional measurement, and an image 43 obtained by combining the measurement result with a perspective image is combined with the captured image. Create as. When a monitor is connected to the processing unit 200, the measurement result image 43 is displayed on the monitor.

  The fluoroscopic image generation unit 203 acquires the three-dimensional coordinates obtained by the above processing, and perspective-transforms these on the display surface of the LCD 111 based on the parameters registered by the calibration processing. Further, a contour pattern EP is set along the projection point of each three-dimensional coordinate found by this perspective transformation, and a projection image 53 in which the contour pattern EP is expressed by a color with high luminance is generated (step C3).

  The projection operation control unit 204 gives the projection image 53 generated by the above processing to the projection unit 110 to execute the projection processing. Thereby, the projected image ME by the contour pattern EP is projected onto the workpiece W2 as a marking pattern indicating the three-dimensional measurement result.

  The contour pattern EP in the projection image 53 generated by the above processing indicates a state in which each point measured by the three-dimensional measurement is observed from the position of the LCD 111 of the projection unit 110. Therefore, when the result of the three-dimensional measurement is correct, the marking pattern ME generated by the projection of the contour pattern EP should be in a state along the actual contour of the workpiece W2. Therefore, the operator can confirm the accuracy of the three-dimensional measurement by comparing the contour line indicated by the marking pattern generated by the projection of the pattern EP with the contour line of the actual workpiece.

1,100 Imaging unit 2,200 Processing unit 10,101 CCD
11,110 Projection unit 12,111 LCD
23, 203 Projected image generation unit 24, 204 Projection operation control unit 22 Inspection execution unit 202 Three-dimensional measurement processing unit 50, 51a, 51b, 52a, 52b, 53 Projection image M, Ma, Mb, Ta, Tb, ME Marking Pattern W0, W1, W2 Workpiece

Claims (5)

An image pickup means having a two-dimensional image pickup device, and an image processing means for inputting a two-dimensional image generated by the image pickup device from the image pickup means and executing predetermined image processing and measurement processing using the input image. In the image processing apparatus provided,
An image projecting unit that includes a display element for displaying a two-dimensional digital image, and projects the image displayed on the display element toward the field of view of the imaging unit;
Projection control means for providing the image projection means with a digital image including a two-dimensional pattern in which image data different in brightness or color from the surroundings is set, and executing projection processing of the image;
An imaging unit and an image projecting unit such that an imaging target range of the imaging element in an imaging plane of an image projected from the image projecting unit is included in a range in which an image from the image projecting unit is projected And the two-dimensional pattern is projected into the imaging target range.
An image processing apparatus.   The said projection control means determines the pixel which sets the image data of the said two-dimensional pattern based on the process result of the said image processing means, The image projected on the said image projection means is produced | generated according to Claim 1. Image processing device.   The projection control unit includes an image registration unit for registering an image including the two-dimensional pattern, and reads one of the images registered in the image registration unit and displays the image on the display element. The image processing apparatus according to claim 1, wherein the image projection processing is executed. The image pickup unit and the image projection unit form a coaxial optical system and are housed in the same casing, and are located on the image pickup element side in the imaging plane of an image projected from the image projection unit. The relationship between the imaging unit and the image projecting unit is adjusted so that image data from different pixels of the display element is projected to each point corresponding to each pixel.
The image processing apparatus according to claim 1. The image projecting unit is formed separately from the image capturing unit, and is displayed at each point corresponding to each pixel on the image sensor side in an image formation plane of an image projected from the image projecting unit. Deployed with the relationship to the imaging means adjusted so that image data from different pixels of the element is projected,
The image processing means executes a three-dimensional measurement process using the image input from the imaging means,
The projection control means perspectively converts the three-dimensional information based on the result of the three-dimensional measurement process of the image processing means to the display surface of the display element, and uses the result of the perspective conversion to convert the pixel into the pixel representing the three-dimensional measurement result. The image processing apparatus according to claim 1, wherein an image in which the image data of the two-dimensional pattern is set is generated, and the image is projected by displaying the image on the display element.

Images