JP2009058359A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily set the sequence of processing formed by combining three-dimensional measurement combining three or more cameras into a plurality of combinations and two-dimensional measurement processing. <P>SOLUTION: Individual camera codes 0/1/2/3 are set for four cameras, respectively, processing items showing the change of respective processing object images of two-dimensional image processing and three-dimensional measurement (change of two-dimensional camera, and change of three-dimensional camera) are individually set, these processing items are received together with the input of the camera code of the camera corresponding to the image after changing, and are incorporated into the sequence. When the sequence is executed, management information by respective camera codes is set individually in two-dimensional image processing and three-dimensional measurement, and processing of updating the management information based on the camera code in association with this processing item is executed as "change of two-dimensional camera" and "change of three-dimensional camera". <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention is capable of executing both two-dimensional image processing and three-dimensional measurement, and has an image processing function that freely sets the content of processing to be executed and the execution order (sequence) of the processing according to the processing purpose. Relates to the device.

  The applicant has recently developed a general-purpose image processing apparatus capable of performing both two-dimensional image processing and three-dimensional measurement. In this device, a plurality of types of processing items and programs necessary for executing the processing indicated by these processing items are registered, and the user selects the necessary processing items and sets the execution order, A series of sequences from image input to output of processing results can be assembled, and processing according to the sequences can be executed (see Patent Document 1).

JP 2007-26423 A

  Three-dimensional measurement is generally performed using two cameras (stereo cameras) whose positional relationship is known, but depending on the situation in the field and the purpose of processing, a combination of stereo cameras can be used. There is a desire to set multiple sets.

  For example, when the object is large and a plurality of measurement target parts are set, the processing can be speeded up by arranging two cameras for each of the plurality of measurement target parts and driving the cameras all at once. . In addition, when it is desired to perform measurement by separately detecting parallax in two different directions for one measurement target part, a set of stereo cameras is used for each direction of parallax detection target using at least three cameras. Need to be deployed.

  However, when a plurality of combinations of cameras for three-dimensional measurement are set, and two-dimensional image processing is also executed on the combination, it is considerably complicated for the user to set a processing sequence.

  This invention pays attention to the above points, and even when three or more cameras are introduced and a process combining three-dimensional measurement and two-dimensional measurement processing in which these cameras are combined in a plurality of ways, the user can perform a sequence. The purpose of this is to make it possible to set easily.

  The image processing apparatus according to the present invention, like the apparatus disclosed in the above-mentioned Patent Document 1, performs two-dimensional image processing using an image generated by one camera and images generated by two cameras. The three-dimensional measurement used was selected by accepting an operation for selecting a processing item, a memory storing a predetermined number of processing items and a program necessary for executing the processing indicated by these processing items, respectively. Sequence assembly means for assembling a sequence according to each processing item, and sequence execution means for executing a program corresponding to the processing item included in the assembled sequence according to the sequence.

  Furthermore, in the image processing apparatus according to the present invention, the camera codes of three or more cameras each identified by a unique camera code and the two cameras used for three-dimensional measurement are derived by calibration for these cameras. There is also provided a registration means capable of registering registration information associated with the three-dimensional measurement parameters for a plurality of combinations of cameras. Further, as processing items that can be incorporated into the sequence, a processing item representing processing for switching a processing target image for two-dimensional image processing and a processing item representing processing for switching a processing target image for three-dimensional measurement are individually provided.

  The sequence assembling means receives and selects a selection operation for each processing item representing processing for switching processing target images together with an operation for selecting camera codes of one or two cameras corresponding to the processing target images after switching. The corresponding processing item is registered in association with the selected camera code.

  For the two-dimensional image processing and the three-dimensional measurement, the sequence execution unit sets management information based on the camera code of the camera corresponding to the processing target image as information for specifying the processing target image. When a processing item representing processing for switching a processing target image for 2D image processing is read, management information for 2D image processing is updated with one camera code associated with the processing item, and 3D measurement processing is performed. When a processing item representing processing for switching the processing target image is read, the management information for three-dimensional measurement is updated with two camera codes associated with the processing item. Further, when executing the measurement process by the process item for 3D measurement in the sequence, the 3D corresponding to these codes from the registration means by the two camera codes constituting the management information for 3D measurement at that time A parameter for measurement is read, and an operation using the read parameter is executed.

  According to the image processing apparatus having the above-described configuration, three-dimensional measurement is performed for each combination by setting a plurality of combinations of two cameras in advance, specifying the relationship between the cameras for each combination, and executing calibration. It becomes possible to execute. The assembly of the sequence is performed according to an operation for selecting a processing item to be executed, an operation for selecting a processing item representing switching of the processing target image, and an operation for selecting the camera code of the camera corresponding to the processing target image after switching. Therefore, the operation is not complicated, and the general user can easily understand the operation method.

  In addition, the processing target image for the two-dimensional image processing and the processing target image for the three-dimensional measurement are individually managed by the camera code of the camera corresponding to each image, and the switching of the processing target image is also performed in two dimensions, three, and three. Since the processing is performed separately for each dimension, selection or change of the processing target image can be performed easily and without error. In addition, although each management information may be comprised only with a camera code, you may include information other than this not only in this but a camera code.

  In a preferred aspect of the image processing apparatus, the sequence execution means includes the camera code of one specific camera as management information for two-dimensional image processing and the camera of two specific cameras as management information for three-dimensional measurement. After initial setting of each code, in response to reading a processing item representing processing for switching the processing target image, the management information corresponding to the processing item is rewritten by the camera code associated with the read processing item. .

  According to the above aspect, both the management information for two-dimensional image processing and the management information for three-dimensional measurement are initially set in advance, so that the processing corresponding to the initial setting is performed first. Thus, it is possible to reduce the number of times the processing target image is switched.

  In a further preferred aspect, the processing item for two-dimensional image processing includes at least one processing item representing processing for converting the processing target image into a new image. The sequence execution means initializes an image generated by each camera as a processing target image specified by the camera code of each camera, and executes a processing item involving image conversion for two-dimensional image processing. The processing target image specified by the camera code included in the management information for two-dimensional image processing at the time of execution is updated to the image after the conversion processing.

  According to the above aspect, for example, when two-dimensional image processing with image conversion, such as processing for correcting a positional deviation of an object, processing for removing noise, or the like is performed on an image generated by a certain camera, If the same camera code included in the management information at the time of this process is set in the management information, the process for the converted image can be executed without distinguishing between two dimensions and three dimensions. it can.

  Therefore, even when the result of the pre-processing performed before the two-dimensional full-scale processing is reflected in the three-dimensional measurement, the user does not need to perform a special setting operation and can prevent the operation from being complicated. . In addition, for two-dimensional image processing, if a plurality of types of processing items accompanied by the above-described conversion processing are set, and a plurality of types of processing items for three-dimensional measurement are set, processing of various contents can be performed by combining these processing items. Can be executed.

  In the image processing apparatus according to a more preferable aspect, the processing item for three-dimensional measurement uses one of the two processing target images as a reference image, a process of detecting a measurement target portion from the reference image, and the other image is detected from the reference image. The contents of the process for specifying the part corresponding to the measurement target part and the process for calculating the coordinates representing the spatial position using the coordinates of the measurement target part in each image are respectively represented.

  According to the above aspect, since a series of processing contents of the three-dimensional measurement is collected into one processing item, even when three-dimensional measurement is performed for each combination of a plurality of cameras, the selection of the processing item for switching the processing target image is performed. By selecting an operation item for 3D measurement, a sequence can be assembled, and the number of users can be greatly reduced.

  According to the image processing apparatus having the above-described configuration, it is possible to easily and without error set a series of processing sequences including three-dimensional measurement using various combinations of stereo cameras and various two-dimensional image processing. Can greatly enhance the versatility and convenience.

FIG. 1 shows a configuration of an imaging unit of an image processing apparatus to which the present invention is applied, and an example of imaging processing.
The image processing apparatus according to this embodiment measures a predetermined part of the vehicle body (for example, an edge portion of a bonnet frame) for a workpiece W (vehicle in the illustrated example) assembled at a factory and detects defects such as scratches and distortions. It is for checking whether there is any.

  Furthermore, in this embodiment, in order to improve the inspection efficiency for a large workpiece W such as a vehicle, two locations of the workpiece W are simultaneously imaged by the two imaging units 1A and 1B. Hereinafter, the imaging unit 1A is referred to as “first imaging unit 1A”, and the imaging unit 1B is referred to as “second imaging unit 1B”.

  Each of the imaging units 1A and 1B has a configuration in which two cameras are incorporated in the casings 15A and 15B. These cameras are assigned identification codes of 0, 1, 2, and 3 for identification at the time of sequence setting described later (hereinafter, these codes are referred to as “camera codes”). In FIGS. 1 and 2 and the following description, based on these camera codes, the respective cameras are indicated by the symbols C0, C1, C2, and C3, respectively, but when the four cameras C0 to C3 are collectively referred to as “ It is called “Camera C”.

Two cameras (C0 and C1, or C2 and C3) incorporated in the same imaging unit are respectively arranged so that their fields of view overlap each other for three-dimensional measurement.
One camera C0, C2 of each imaging unit 1A, 1B is installed with the optical axis oriented in the vertical direction, and cameras C2, C3 are installed with the optical axis oriented obliquely downward. Accordingly, the images generated by the cameras C0 and C2 indicate the state of the inspected site viewed from the front (hereinafter, these images are referred to as “front-view images”), and are generated by the cameras C1 and C3. The image shows a state in which the part to be inspected is viewed obliquely from above.

  In this embodiment, based on the camera installation described above, measurement is performed by various two-dimensional image processing using the front-view images of the cameras C0 and C2, and the images generated by the cameras C0 and C1 and the cameras C2 and C3 are used. The three-dimensional measurement is executed by combining the images generated by the above. The contents and order of the processes to be executed can be variously set by an operation for selecting a pre-registered process item.

FIG. 2 is a block diagram showing the overall configuration of the image processing apparatus.
This image processing apparatus has a configuration in which each imaging unit 1A, 1B, monitor 11, operation unit 12 and the like are connected to a main body unit 10 in which a computer is incorporated.

  In the main body 10, image input units 100, 101, 102, 103 corresponding to the cameras C0, C1, C2, and C3, a camera driving unit 104, an image memory 105, a control processing unit 106, a memory 107, and an output unit are provided. 108, an I / O port 109, and the like are incorporated.

  The control processing unit 106 includes a CPU, ROM, RAM, and the like (not shown). The memory 107 is a non-volatile memory such as a flash disk or a hard disk, and stores programs necessary for the operation of the control processing unit 106 (programs for controlling the operation of the apparatus, programs necessary for executing various processing items, and the like). The Further, in the memory 107, a processing item table 201 and a sequence storage unit 202, which will be described later, a storage unit for parameters for 3D measurement (used for calculation for calculating 3D coordinates), and the like are set. The

  Prior to this processing, the three-dimensional measurement parameters are derived by performing individual calibration on the imaging units 1A and 1B, and are registered in association with the camera codes of the two cameras to be calibrated. . Furthermore, in this embodiment, prior to this processing, the contents of the processing item table 201 are displayed on the monitor 11 and a setting operation by the operation unit 12 is accepted, thereby a sequence in which processing items to be executed are arranged in a predetermined order. It is set and stored in the memory 107.

FIG. 3 shows a partial configuration of the processing item table.
The processing item table of this embodiment is classified according to the purpose of processing such as “image input / image switching”, “two-dimensional preprocessing”, “two-dimensional measurement”, “three-dimensional measurement”, and a plurality of processing items for each classification. Is registered.

  "Image input / image switching" is literally for inputting and switching images used for processing. There are three types of processing items: "image input", "two-dimensional camera switching" and "three-dimensional camera switching". Is set.

“Image input” means a process of driving the camera C and capturing a generated image. In this process, as a general rule, four cameras C are driven all at once. However, by selecting a process item and inputting a specific camera code, one to three cameras C can be operated.
For example, if only the cameras C0 and C1 of the first imaging unit 1A are desired to perform imaging, “image input” may be selected together with the input of the camera codes “0” and “1”.

“Two-dimensional camera switching” indicates processing for switching a processing target image for two-dimensional image processing, and “three-dimensional camera switching” indicates processing for switching a processing target image for three-dimensional measurement. These processing items are also selected together with the input of the camera code of the camera C corresponding to the processing target image after switching.
Note that the expression “camera switch” is used for the names of these processing items because the process of specifying the image to be processed is performed using a camera code, so “camera switch” rather than “image switch” is used. However, this is because it is easy for the user to grasp and it is considered that specification errors are unlikely to occur.

  Next, in the “two-dimensional preprocessing”, processing items for setting the processing target image to an image suitable for full-scale measurement, such as displacement of a measurement target portion in the image and removal of noise, are set. In these processing items, a processing target image is converted to generate a new image. For example, in “edge position correction” and “model position correction”, a feature part (edge point, pattern corresponding to the model, etc.) is extracted from the processing target image, and the part comes to a pre-registered reference position. Then, an image in which the position of each pixel is corrected is generated. “Filtering” converts image data of each pixel of the processing target image by an operation such as Fourier transform.

  “Return to previous image” is a process of returning the converted image to the previous image. This processing item is mainly selected when it is desired to return the processing target image to the original image for another measurement after the image is converted by another processing item and some measurement processing is executed. This processing item is also originally performed by conversion opposite to the conversion performed previously. However, in this embodiment, the image before conversion is left in the image memory 105, and this is read as appropriate. It is designed to efficiently handle “Return to Image”.

  In the “two-dimensional measurement”, processing items for full-scale measurement by two-dimensional image processing are set. “Tint edge position” indicates a process of specifying x and y coordinates representing the position of an edge by performing binarization and projection processing on the processing target image. “Correlation search” indicates a process of detecting a region corresponding to a model by performing a correlation operation on a model image registered in advance. The “ECM search” is a method for checking an area corresponding to a model by matching a distribution pattern of angle information (applicant calls “edge code”) indicating a density gradient direction in pixel units with a model pattern. The process to detect is shown. Further, in “correlation search” and “ECM search”, a specific measurement method (for example, area calculation, center of gravity coordinate calculation, etc.) for the detected region can be set using a menu (not shown).

  In “3D measurement”, a plurality of processing items related to the calculation of 3D coordinates are set. In any of the processing items, a process for detecting a measurement target part from the front view image by the camera C0 or C2, and a part corresponding to the measurement target part in the front view image is detected with respect to the image by the other camera C1 or C3. Each process of a process and the process which calculates the three-dimensional coordinate by performing the calculation using the coordinate in each image about the measurement object site | part detected by two images is shown. The three types of processing items “3D edge position”, “3D search”, and “3D ECM search” shown here are methods according to “Dark edge position”, “Correlation search”, and “ECM search” of 2D measurement, respectively. Then, the measurement target part is detected from the front view image.

FIG. 4 shows a system configuration related to the setting and execution of the above sequence.
In the figure, 201 is the processing item table shown in FIG. In addition to the processing item table 201, the system of this example includes a sequence storage unit 202, a program storage unit 203, a designation operation reception unit 204, a sequence assembly unit 205, a sequence execution unit 206, a processing result output unit 207, and the like. Among these, the processing item table 201, the sequence storage unit 202, and the program storage unit 203 are stored as files in the memory. The other configuration is a function set in the control processing unit by a program in the memory.

  The designation operation accepting unit 204 displays the contents of the process item table 201 on the monitor 11 and accepts a designation operation from the operation unit 12 regarding the selection of process items and the execution order of the selected process items. When “2D camera switching” or “3D camera switching” is selected, an operation for selecting a processing target image after switching by a camera code is accepted.

  The sequence assembling unit 205 receives the designation content received by the above operation from the designation operation accepting unit 204 and assembles a sequence according to the designation. A specific data structure representing this sequence is obtained by arranging code information representing designated processing items in a designated order. Further, “image input”, “2D camera switching”, and “3D camera switching” in the sequence are respectively associated with camera codes.

  The sequence execution unit 206 reads the sequence stored in the sequence assembly unit 205, and sequentially reads and executes the programs corresponding to the processing items constituting the sequence from the program storage unit 203. The processing result output unit 207 outputs the processing result finally obtained by executing this sequence.

Furthermore, the sequence execution unit 206 of this embodiment individually sets management information based on the camera code corresponding to the processing target image for each of the two-dimensional image processing and the three-dimensional measurement processing, and sets each processing item based on the management information. The image to be processed is specified.
For example, if the management information of the two-dimensional image processing is [0] and the management information of the three-dimensional image processing is [2] and [3], it belongs to “two-dimensional preprocessing” and “two-dimensional measurement”. When executing the process item, an image input from the camera C0 or a converted image corresponding to the image is set as a processing target. On the other hand, when processing items belonging to “three-dimensional measurement” are executed, images input from the cameras C2 and C3 or converted images corresponding to these images are processed. In the calculation for calculating the three-dimensional coordinates, parameters associated with the two camera codes constituting the management information are read from the memory 107, and the calculation using these parameters is executed.

  In this embodiment, management information for two-dimensional image processing is [0] (camera code of the front-view camera C0 of the imaging unit 1A), and management information for three-dimensional measurement is [0] [1] (imaging). (Camera codes of the cameras C0 and C1 of the section 1A) are respectively initialized, and the processing shown in the sequence is started. For “2D camera switching” and “3D camera switching” in the sequence, the management information is updated by the camera code associated with each processing item. Thereby, the processing target image specified by the management information is switched.

FIG. 5 shows a specific example of a sequence set based on the processing item table 201 and a change in management information while this sequence is executed.
In this sequence, two-dimensional measurement processing is performed using images generated by the front-view cameras C0 and C3, and three-dimensional measurement is performed for each of the imaging units 1A and 1B.

The “sequence number” in the figure indicates the order of processing items in the sequence. Hereinafter, the step of the process executed by the process item having the sequence number n is referred to as “step n”.
“2D management information” in the figure is management information for 2D image processing, and is represented by the camera code of one camera. The “three-dimensional management information” is management information for three-dimensional measurement, and is represented by camera codes of two cameras (limited to combinations in which parameters for three-dimensional measurement are set). The sequence execution unit 206 executes the sequence and updates the management information.

  In the first step 1 of this sequence, the cameras C0 to C3 are simultaneously driven, and four generated images are input. When this processing is executed, [0] is set as an initial value in the two-dimensional management information, and [0] and [1] are set as initial values in the three-dimensional management information. Although not shown here, the images inputted from the cameras C0 to C3 and stored in the image memory 105 are set in a state where they can be read by the camera codes [0] to [3] of the cameras, respectively. Is done.

In step 2, “model position correction” of “two-dimensional preprocessing” is executed. Since this processing item belongs to the two-dimensional image processing, the processing target image is specified based on the two-dimensional management information. Therefore, in step 2, the image corresponding to the camera code 0 (the input image from the camera C0 at this time) is set as the processing target image.
In “model position correction”, conversion processing for correcting the processing target image is executed. When the processing is completed, the camera code and the camera code are set so that the converted image becomes an image specified by the camera code 0. The correspondence with the image is changed.

  In the next step 3, “correlation search” of “two-dimensional measurement” is executed. Since this processing item also belongs to the two-dimensional image processing, the processing target image is specified based on the two-dimensional management information as in step 2. [0] is also set in the two-dimensional management information at this stage, but since the image corresponding to the camera code 0 is changed to the converted image, in step 3, the converted image is processed. Set to image.

  In step 4, as “two-dimensional camera switching”, a process of changing the two-dimensional management information from [0] to [1] is executed. As a result, the processing target image for the subsequent two-dimensional image processing is switched to the image specified by the camera code 1 (the input image from the camera C1).

In step 5, "model position correction" similar to step 2 is executed. In this process, the input image from the camera C1 is set as the processing target image based on the two-dimensional management information changed in the previous step 4.
Also in this step 5, as in the previous step 2, the processing target image is converted by “model position correction”. Therefore, when the processing is completed, the converted image becomes an image specified by the camera code 1. The correspondence between the camera code and the image is changed.

  Next, in step 6, “3D edge position” of “3D measurement” is executed. Up to this stage, the three-dimensional management information is maintained at the initial values “0” and “1”, but the images corresponding to these camera codes are updated by “model position correction” in steps 2 and 5. Yes. Therefore, also in step 6, the process is executed for these updated images.

  Thereafter, in step 7, a process of changing the two-dimensional management information from [1] to [2] is executed as “two-dimensional camera switching”. Thereby, the processing target image of the subsequent two-dimensional image processing is switched to an image corresponding to the camera code 2 (an input image from the camera C2).

  In step 8, “model position correction” is executed on the input image from the camera C 2 based on the changed two-dimensional management information. As a result of this processing, as in the previous step 2 and step 5, the correspondence between the camera code and the image is changed so that the image after conversion in the “model position correction” becomes the image specified by the camera code 2. The

  In step 9, “ECM search” in “two-dimensional measurement” is executed using the converted image generated in step 8 as a processing target image based on the set value [2] of the management information.

  In step 10, a process of changing the two-dimensional management information from [2] to [3] is executed as “two-dimensional camera switching”. By this processing, the processing target image for the subsequent two-dimensional image processing is switched to an image corresponding to the camera code 3 (an input image from the camera C3).

  In step 11, “model position correction” is executed on the input image from the camera C3 based on the changed two-dimensional management information (step 11). Also in this case, the correspondence between the camera code and the image is changed so that the image after conversion in the “model position correction” becomes an image specified by the camera code 3.

  In the next step 12, as “three-dimensional camera switching”, a process of changing the three-dimensional management information set so far to [0] [1] to [2] [3] is executed. As a result, the processing target image for the subsequent three-dimensional measurement is switched to an image corresponding to the camera codes 2 and 3.

In step 13, “3D edge position” is executed based on the 3D management information changed in step 12.
Here, the images corresponding to the camera codes 2 and 3 have been changed to images after conversion by “model position correction” in steps 8 and 11, respectively. Set to

  Thereafter, a determination process (not shown), a process for outputting a process result, and the like are executed, and the process ends.

  The above sequence uses the image generated by the first imaging unit 1A to execute the two-dimensional and three-dimensional measurement processes, and then switches the processing target to the image generated by the second imaging unit 1B. Similarly, two-dimensional and three-dimensional measurement processes are set to be executed.

  Here, for input images from the cameras C0 and C2 used for specific two-dimensional measurement, by selecting “model position correction” as the pre-processing of the two-dimensional measurement, what designation operation is performed for the three-dimensional measurement. Even without this, the processing result of “model position correction” can be directly applied to the three-dimensional measurement.

FIG. 6 shows another example of a sequence capable of executing the same processing as in FIG.
Also in this example, the processing of steps 1 to 3 is the same as the example of FIG. That is, after inputting an image from each of the cameras C0 to C3, based on the initial value of the two-dimensional management information, “model position correction” is executed on the input image from the camera C0, and “correlation search” is further executed. .

Next, in this example, as “two-dimensional camera switching”, the two-dimensional management information is changed from the initial value [0] to [2] (step 4). Then, using the image (input image from the camera C2) corresponding to the changed two-dimensional management information, “model position correction” is executed, and “ECM search” is further executed (steps 5 and 6).
In step 3 and step 6, the converted image generated by the previous “model position correction” is set as the processing target image.

  Next, as “two-dimensional camera switching”, the two-dimensional management information is changed from “2” to “1”, and an image corresponding to the changed two-dimensional management information (an input image from the camera C1) is targeted. Then, “model position correction” is executed (steps 7 and 8). Furthermore, as the “switching two-dimensional camera” again, the two-dimensional management information is changed from “1” to “3”, and an image corresponding to the changed two-dimensional management information (an input image from the camera C3) is targeted. Then, “model position correction” is executed (steps 9 and 10).

  As described above, the processing target images of the two-dimensional image processing are sequentially switched to execute the two-dimensional image processing. During this time, the three-dimensional management information is maintained at the initial value [0] [1]. In the next step 11, based on the initial value of the three-dimensional management information, “three-dimensional edge position” is executed for the images corresponding to the camera codes 0 and 1. Since the images corresponding to the camera codes 0 and 1 are updated to the images converted by the “model position correction” in steps 2 and 8, respectively, in step 11, the processing for these updated images is executed. Is done.

  Thereafter, in step 12, as “three-dimensional camera switching”, the three-dimensional management information is changed from the initial value to [2] [3]. In step 13, “3D edge position” is executed for the images corresponding to the camera codes 2 and 3 based on the changed 3D management information. The images corresponding to the camera codes 2 and 3 are also updated to the images converted by the “model position correction” in steps 5 and 10, respectively. In step 12, processing for these updated images is executed. Is done.

  According to the sequence shown in FIG. 6, after the two-dimensional image processing for the front-view images by the cameras C0 and C2 is executed with priority, the pre-processing for three-dimensional measurement is performed on the input images from the cameras C1 and C3. Thereafter, three-dimensional measurement is performed for each of the imaging units 1A and 1B. Although the order of processing items and the order of camera codes selected for switching management information are different, processing is performed on the same image as in FIG. 5 for each of the two-dimensional and three-dimensional processing items. Therefore, even in this sequence, the same processing result as that in the sequence of FIG. 5 can be obtained.

  As described above, in the image processing apparatus according to this embodiment, it is possible to set a plurality of sequences for processing of the same content, so depending on circumstances such as processing priority, processing efficiency, and user preference. Thus, the contents of the sequence can be set flexibly.

  Moreover, for 3D measurement, it is only necessary to specify a processing target image by “switching 3D camera” and select one processing item for 3D measurement. Therefore, 3D measurement is performed for each combination of a plurality of cameras. Even when the setting is performed, the setting is not complicated, and the setting can be performed efficiently and without error.

  In addition, a plurality of processing items having different processing contents are set as processing items for 3D measurement, and a plurality of processing items for preprocessing are set as processing items for 2D image processing. Depending on the combination with the three-dimensional measurement, the contents of the processing to be executed can be set differently. Therefore, the degree of freedom of processing to be executed is increased, and it is possible to flexibly cope with any processing regardless of the purpose of inspection or measurement and the shape and type of the measurement target part.

  In the above-described embodiment, two imaging units each including two cameras C are provided so that three-dimensional measurement can be performed for each imaging unit. It is also possible to perform three-dimensional measurement with respect to more parts by using three or more parts.

  On the other hand, it is also possible to perform a plurality of three-dimensional measurements using three or more independent cameras without disposing a unit-type imaging unit as in the above embodiment. For example, one camera is arranged with its optical axis oriented in the vertical direction to generate a front-view image, two cameras are arranged with its optical axis inclined, and these cameras are used for front-view cameras. These may be combined to perform three-dimensional measurement. Also in this case, various processes can be easily and accurately performed by executing the process using the front-view image as the processing target image for the two-dimensional image processing and the reference image for the three-dimensional measurement.

It is explanatory drawing which shows the structure and arrangement | positioning example of an imaging part. It is a block diagram which shows the structure of an image processing apparatus. It is explanatory drawing which shows the data structure of a process item table. It is a functional block diagram of the system regarding the assembly and execution of a sequence. It is explanatory drawing which matches and shows the specific example of a sequence, and the change of the management information accompanying execution of this sequence. It is explanatory drawing which matches and shows the other example of a sequence, and the change of the management information accompanying execution of this sequence.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1A, 1B Image pick-up part C0, C1, C2, C3 Camera 10 Main part 11 Monitor 12 Operation part 105 Image memory 106 Control processing part 107 Memory 201 Process item table 202 Sequence preservation | save part 203 Program storage part 204 Designated operation reception part 205 Sequence assembly Part 206 Sequence execution part

Claims (4)

A predetermined number of processing items and processes indicated by these processing items for two-dimensional image processing using an image generated by one camera and three-dimensional measurement using an image generated by two cameras, respectively. A memory storing a program necessary for executing the process, sequence assembling means for accepting an operation for selecting a process item and assembling a sequence according to each selected process item, and a process item included in the assembled sequence. In an image processing apparatus comprising sequence execution means for executing a corresponding program according to the sequence,
3D measurement parameters derived from calibration of these cameras are associated with the camera codes of 3 or more cameras, each identified by a unique camera code, and the 2 cameras used for 3D measurement. A registration means capable of registering registration information for a plurality of combinations of cameras;
As processing items that can be incorporated into the sequence, processing items representing processing for switching processing target images for two-dimensional image processing and processing items representing processing for switching processing target images for three-dimensional measurement are individually provided.
The sequence assembling means accepts a selection operation for each processing item representing processing for switching the processing target image together with an operation for selecting camera codes of one or two cameras corresponding to the processing target image after switching, Register the selected process item in association with the selected camera code,
The sequence execution means includes
For two-dimensional image processing and three-dimensional measurement, as information for specifying the processing target image, management information by the camera code of the camera corresponding to the processing target image is set.
When a processing item representing processing for switching a processing target image of two-dimensional image processing is read, management information for two-dimensional image processing is updated with one camera code associated with the processing item,
When the processing item representing the processing for switching the processing target image for 3D measurement is read, the management information for 3D measurement is updated with the two camera codes associated with the processing item,
When a measurement process using a process item for 3D measurement in a sequence is executed, 3D measurement corresponding to these codes from the registration unit is performed by two camera codes constituting management information for 3D measurement at that time. Read out the parameters for and execute the operation using the read parameters,
An image processing apparatus.   The sequence execution means initializes the camera code of one specific camera as management information for two-dimensional image processing and the camera code of two specific cameras as management information for three-dimensional measurement, respectively. The management information corresponding to the processing item is rewritten in response to the reading of the processing item representing the processing for switching the processing target image with the camera code associated with the read processing item. Image processing device. The processing item for the two-dimensional image processing includes at least one processing item representing processing for converting the processing target image into a new image,
The sequence execution means initializes an image generated by each camera as a processing target image specified by the camera code of each camera, and executes a processing item involving image conversion for two-dimensional image processing. The image processing apparatus according to claim 1, wherein a processing target image specified by a camera code included in management information for two-dimensional image processing at the time of execution is updated to the image after the conversion processing.   The processing item for the three-dimensional measurement corresponds to a process for extracting a measurement target part from the reference image using one of the two process target images as a reference image, and a measurement target part detected from the reference image in the other image. 4. The method according to claim 1, wherein each content of processing for specifying a part and processing for calculating a coordinate representing a spatial position using coordinates of a measurement target part in each image is represented. Image processing apparatus.

Images