JP2023111297A - Program, inspection method, and inspection device - Google Patents

Abstract

To provide a program with which it is possible to identify a mismatch point where an object and design data do not match by a reduced amount of calculation.SOLUTION: A program pertaining to the present invention causes a computer to execute the steps of: acquiring a captured image P of an object 30 which is placed on an inspection table 5; extracting the geometric feature of the object 30 on the basis of the captured image P and generating a first plane image 51 in which the extracted geometric feature is included; extracting the geometric feature of a three-dimensional model 40 on the basis of a two-dimensional shape indicated by the three-dimensional model 40 while the three-dimensional model 40 is directed in a first direction corresponding to the bearing of the object 30 on the inspection table 5, and generating a second plane image 52 in which the extracted geometric feature is included; identifying a mismatch point 53 where the geometric feature of the object in the first plane image 51 and the geometric feature of the three-dimensional model in the second plane image 52 do not match; and outputting information that indicates the mismatch point 53.SELECTED DRAWING: Figure 2

Description

The present invention relates to a program, an inspection method, and an inspection apparatus.

Conventionally, there is known a technique for superimposing and displaying edge lines of a three-dimensional structure extracted from a captured image of the three-dimensional structure and edge lines of a CAD model (see, for example, Patent Document 1).

JP-A-2017-91078

With the conventional technology, it is necessary to superimpose the edge lines extracted from the captured image on each edge line of the three-dimensional CAD model, or to measure the entire three-dimensional structure with a 3D scanner, etc., which requires a large amount of computational processing. The problem was that it was necessary.

Therefore, the present invention has been made in view of these points, and it is an object of the present invention to provide a program, an inspection method, and an inspection apparatus that can identify a mismatching point between an object and design data with a small amount of calculation.

A program according to the present invention for solving the above-mentioned problems includes, in a computer, an image acquisition step of acquiring a captured image generated by capturing an image of an object placed on an examination table with an imaging device; a first image generating step of extracting the shape features of the object based on the above, and generating a first planar image including the extracted shape features; Based on a two-dimensional shape shown by the three-dimensional model in a state in which the object is oriented in a first direction corresponding to the orientation on the examination table, geometric features of the three-dimensional model are extracted, and the extracted shape features are a second image generating step of generating a second planar image to be included; and a mismatched portion where the geometric features of the object in the first planar image do not match the geometric features of the stereo model in the second planar image. and an output step of outputting information indicating the inconsistent portion.

In the image acquisition step, the captured image generated by capturing an image of the object together with a figure pattern arranged in a predetermined positional relationship with respect to the object with the imaging device is acquired, The 2-image generating step includes: identifying the orientation of the object based on the obtained image of the figure pattern; generating said second planar image by orienting in said first orientation.

In the step of identifying the orientation of the object, the orientation of the graphic pattern is identified based on the image of the graphic pattern, and the orientation of the graphic pattern is identified by referring to information indicating the positional relationship between the object and the graphic pattern. An orientation of the object corresponding to the orientation of the graphic pattern may be identified.

In the image acquisition step, the imaged image generated by imaging a jig figure pattern arranged in a predetermined positional relationship with respect to a jig for positioning the object, which is placed on the inspection table. is obtained, and the second image generating step includes: identifying the orientation of the jig based on the obtained image of the jig graphic pattern; and identifying the orientation of the object corresponding to the identified orientation of the jig. generating the second planar image by orienting the three-dimensional model in the first direction such that the direction of the three-dimensional model corresponds to the direction of the three-dimensional model.

In the second image generating step, the direction of the figure pattern may be specified by specifying the direction of a vector connecting a plurality of figures included in the figure pattern.

The second image generating step determines whether a degree of matching between the shape feature of the object indicated by the first planar image and the shape feature based on the stereo model corresponding to the first orientation is equal to or greater than a threshold. extracting an updated geometric feature based on a two-dimensional shape shown by the solid model in a state of being oriented in a second direction different from the first direction if the degree of matching is less than the threshold; and the second plane including the updated geometric feature when a degree of matching between the geometric feature of the object indicated by the first plane image and the updated geometric feature is equal to or greater than the threshold value. and generating an image.

The second image generating step refers to a plurality of the second plane images generated based on the stereo model, and selects one having the highest similarity to the first plane image among the plurality of the second plane images. You can choose one.

The second image generation step includes: specifying an orientation of the object based on images of a plurality of structural parts formed on the object; Correspondingly, generating said second planar image by orienting said solid model in said first orientation.

The step of identifying the orientation of the object based on the images of the plurality of structural parts refers to the three-dimensional CAD data and determines whether each of the structural parts has a predetermined position and/or a predetermined shape. and determining the orientation of the object based on the images of the plurality of structures when three or more of the structures have predetermined positions and/or predetermined shapes.

The image acquisition step includes acquiring a plurality of captured images generated by capturing images of the object from a plurality of orientations with a plurality of the imaging devices, and the mismatching portion identifying step includes capturing each of the captured images. identifying the inconsistent points in a plurality of orientations of the object based on the first planar image and the second planar image generated based on.

The image acquisition step includes a step of acquiring the captured images generated by photographing a plurality of the objects by one or a plurality of the imaging devices, and the mismatching part identifying step includes: The step of identifying the discrepancies for each may be included.

The first image generation step includes a step of continuously photographing the object with the imaging device to update the first planar image, and the discrepancy identifying step includes updating the first planar image and The method may include identifying the mismatched portion based on the first planar image after updating when the degree of matching with the first planar image before updating is less than a predetermined threshold.

The geometrical features of the object and the geometrical features of the three-dimensional model are edge lines of the object and the three-dimensional model, and in the discrepancy identifying step, edge lines of the object and edges of the object A location where the portion corresponding to the line and the edge line of the three-dimensional model are separated by a predetermined threshold or more may be specified as the mismatch location.

The output step may include a step of outputting an inspection result image, which is an image obtained by superimposing the edge line, which is the shape feature of the stereo model, on the image of the object in the captured image.

In the output step, light indicating the position of the non-matching portion or light indicating the position and/or shape of the structural portion of the designed object represented by the three-dimensional CAD data is projected onto the object. may include the step of

The output step includes information indicating the presence of the non-matching portion, information indicating the position of the non-matching portion, or position and/or shape of the structural portion of the design object represented by the three-dimensional CAD data. A step of outputting the information to a display may be included.

The mismatching point identifying step includes identifying the mismatching point based on the first planar image, and then re-identifying the mismatching point based on the updated first planar image. When the number or range of mismatched points identified in the step of re-identifying the mismatched points based on the updated first planar image is less than a predetermined reference value, outputting information to that effect to the display unit. may

The outputting step may include outputting work information associated with the inconsistent portion.

An inspection method of the present invention includes an image acquisition step of acquiring a captured image generated by capturing an image of an object placed on an inspection table with an imaging device, executed by a computer; a first image generating step of extracting the geometrical features of the object and generating a first planar image containing the extracted geometrical features; Based on the two-dimensional shape shown by the three-dimensional model while being oriented in the direction corresponding to the orientation on the examination table, the shape features of the three-dimensional model are extracted, and a second planar image including the extracted shape features. a second image generating step of generating a non-matching portion identification of identifying a mismatched portion between the shape feature of the object in the first planar image and the shape feature of the three-dimensional model in the second planar image; and an output step of outputting information indicating the inconsistent portion.

An inspection apparatus of the present invention includes an image acquisition unit that acquires a captured image generated by capturing an image of an object placed on an inspection table with an imaging device, and a shape feature of the object based on the captured image. a first image generation unit that extracts and generates a first planar image that includes the extracted shape features; a second planar image that extracts geometric features of the three-dimensional model based on the two-dimensional shape shown by the three-dimensional model while being oriented in a direction corresponding to a second planar image that includes the extracted geometric features; an image generating unit, a mismatching point identifying unit that identifies mismatching points where the geometrical features of the object in the first planar image and the geometrical features of the stereo model in the second planar image do not match, and the mismatching part. and an output unit that outputs information indicating the location.

ADVANTAGE OF THE INVENTION According to this invention, it is effective in the ability to provide the program, the inspection method, and the inspection apparatus which can specify the non-matching part of a target object and design data with a small amount of calculations.

1 is a block diagram showing the configuration of an inspection system of this embodiment; FIG. 2 is a diagram for explaining an outline of an inspection method by the inspection system of FIG. 1; FIG. It is a perspective view which shows the example of a target object. It is an example of the captured image image|photographed with the imaging device. It is a specific example of the first plane image. It is a specific example of the second plane image. It is an example of an inspection result image showing an inspection result. It is an example of the operation flow by an inspection system. It is a figure which shows the inspection system of 2nd Embodiment. It is an example of the image of the object which the imaging device image|photographed. 1 is a diagram illustrating an example of an inspection system that includes multiple imaging devices; FIG. 3 is a block diagram showing a configuration in which an output unit outputs to a projection device; FIG.

<First embodiment>
[Overview of inspection system S100]
FIG. 1 is a block diagram showing the configuration of an inspection system S100 of this embodiment. FIG. 2 is a diagram for explaining an outline of an inspection method by the inspection system S100 of FIG. In the following description, the same or corresponding elements are denoted by the same or corresponding reference numerals, and overlapping descriptions are omitted.

(Outline of inspection method by inspection system S100)
The inspection system S100 includes an imaging device 1 and an inspection device 2, as shown in FIG. As shown in FIG. 2, the imaging device 1 captures an object 30 placed on an examination table 5 in real space to generate a captured image P. As shown in FIG. The inspection device 2 generates a first planar image 51 based on the captured image P generated by the imaging device 1 . The first planar image 51 is a planar image including two-dimensional shape features extracted from the object 30 appearing in the captured image P. FIG.

The inspection device 2 also generates a second planar image 52 based on the three-dimensional model 40 indicated by the three-dimensional CAD data, which is the design data of the object 30 . The second planar image 52 is a two-dimensional shape extracted from the shape shown by the three-dimensional model 40 in the CAD space when the three-dimensional model 40 is oriented in the first direction corresponding to the orientation of the object 30 on the examination table 5. 2 is a planar image containing the characteristic features.

The inspection device 2 compares the first planar image 51 and the second planar image 52 to identify mismatching portions 53 where the shape features do not match each other, and outputs the identified result.

According to such a configuration, the inspecting apparatus 2 identifies and outputs the mismatched points between the geometrical features of the object 30 and the geometrical features of the three-dimensional model 40, so that the user can, for example, specify the object 30 as the design data. You can check if it is manufactured correctly. In addition, since the inspection device 2 identifies the non-matching portion 53 based on the two-dimensional shape of the object 30 and the three-dimensional model 40 rather than the three-dimensional shape, the computational cost for identifying the non-matching portion 53 can be reduced.

[Detailed Configuration of Inspection System S100]
The details of the inspection system S100 will be described below using the case where the inspection system S100 inspects an object 30 having a shape as shown in FIG. 3 as an example. FIG. 3 is a perspective view showing a specific example of the target object 30. As shown in FIG. 3(a) is a perspective view of an actually manufactured object 30, and FIG. 3(b) is a three-dimensional model 40 of the object 30. FIG.

The object 30 has a body portion 31 and a projecting portion 32 . As an example, the body portion 31 is formed with four holes 31a, 31b, 31c and 31d. The protruding portion 32 protrudes from the upper portion of the main body portion 31 .

The three-dimensional model 40 is a model indicated by three-dimensional CAD data, which is design data of the object 30 . A difference in shape between the three-dimensional model 40 and the object 30 will be described later with reference to other drawings.

(Imaging device 1)
The imaging device 1 is a camera for imaging an object 30 as shown in FIGS. 1 and 2, and has an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging device 1 is, for example, a single focus camera. In the first embodiment, a configuration in which only one imaging device 1 is provided will be described. An embodiment including a plurality of imaging devices 1 will be described later.

The imaging device 1 is supported by, for example, a camera stand (not shown) and directed toward the examination table 5 . The imaging device 1 may have an illumination device (not shown) so that the object 30 can be photographed well. Any system can be used as the illumination device, and for example, a transmissive illumination system, a reflected illumination system, or a coaxial epi-illumination system can be used.

The imaging device 1 captures an image of an object 30 and generates a captured image P. FIG. FIG. 4 is an example of a captured image P captured by the imaging device 1. As shown in FIG. As shown in FIG. 4, the imaging device 1 images the object 30 placed on the inspection table 5 and the figure pattern 60 together to generate a captured image P. As shown in FIG.

The figure pattern 60 is arranged in a predetermined positional relationship with respect to the target object 30 . The graphic pattern 60 is for specifying the orientation of the object 30 placed on the inspection table 5, as will be described later. A figure pattern 60 includes a plurality of figures 61 arranged at predetermined intervals. The figure pattern 60 is a circle grid pattern, and has, for example, figures 61 of circular dots arranged at regular intervals in a grid. As an example, the graphic pattern 60 is attached by the user at a position having a predetermined positional relationship with respect to the object 30 .

It should be noted that the figure pattern 60 can be modified in various ways other than the one shown in FIG. A graphic pattern may be applied to a portion of the object 30 . The graphic pattern may be formed separately from the object 30 or integrally formed with the object 30 . For example, resin parts or the like may be arranged at a plurality of locations on the object 30, and the plurality of resin parts may function as a graphic pattern.

The graphic pattern may also be, for example, a square grid pattern containing a plurality of rectangular graphics, or a two-dimensional barcode. A square grid pattern is exemplified in the second embodiment. The inspection apparatus 2 can also identify the orientation of the object 30 based on images of a plurality of structural portions formed on the object 30 instead of the graphic pattern. This will be explained in a third embodiment.

(Inspection device 2)
The inspection device 2 is a computer installed with a computer program according to one embodiment of the present invention. The inspection apparatus 2 has an input device 21, a display section 22, a storage section 23, and a control section 24, as shown in FIG.

The input device 21 is a device for receiving input from a user. The input device 21 is any device such as a keyboard, mouse, touch display, or the like. The display unit 22 is one or more displays that display various types of information.

The storage unit 23 is a storage medium that stores various data, and includes a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, and the like. The storage unit 23 stores computer programs executed by the control unit 24 . The storage unit 23 stores information indicating the positional relationship of the graphic pattern 60 with respect to the object 30 . The storage unit 23 may store the three-dimensional CAD data of the target object 30 in association with the identification information of the target object 30 . The storage unit 23 may store information indicating the positional relationship of the jig figure pattern 60A ( FIG. 9 ) with respect to the jig 70 . The storage unit 23 may store, for example, threshold information of the distance between the edge line of the object 30 and the edge line of the three-dimensional model 40, which the inspection device 2 refers to when identifying the mismatched portion.

The control unit 24 has a CPU (Central Processing Unit) and the like. The control unit 24 functions as an image acquisition unit 241, a first image generation unit 242, a second image generation unit 243, a mismatched part identification unit 244, and an output unit 245 by executing computer programs stored in the storage unit 23. do.

(Regarding each functional unit of the control unit 24)
The function of each functional unit of the control unit 24 will be described below with reference to specific examples of the first planar image 51 and the second planar image 52 . FIG. 5 is a specific example of the first planar image 51. As shown in FIG. FIG. 6 is a specific example of the second plane image 52. As shown in FIG.

The image acquisition unit 241 acquires a captured image P generated by capturing an image of the object 30 placed on the inspection table 5 by the imaging device 1 . Specifically, the image acquisition unit 241 acquires the captured image P ( FIG. 4 ) generated by the imaging device 1 capturing the graphic pattern 60 and the target object 30 together.

(Processing for Generating First Planar Image 51)
The first image generating section 242 generates the first planar image 51 (see FIG. 2) based on the captured image P acquired by the image acquiring section 241 . Specifically, the first image generator 242 extracts the shape feature of the target object 30 of the captured image P. As shown in FIG. More specifically, the first image generator 242 extracts the edge line of the object 30 as a shape feature. Extraction of edge lines can be performed by a conventionally known technique. As shown in FIG. 5, the first image generator 242 generates an image including the extracted edge line E1 as a first planar image 51, which is a two-dimensional image.

(Processing for Generating Second Planar Image 52)
The second image generator 243 reads the data of the three-dimensional model 40 corresponding to the object 30 from the storage unit 23 . Specifically, the second image generation unit 243 reads from the storage unit 23 the data of the three-dimensional model 40 associated with the identification information of the object 30 input by the user. The second image generator 243 may read the data of the three-dimensional model 40 from an external storage device (not shown) different from the inspection device 2 .

The second image generator 243 generates the second planar image 52 based on the stereo model 40 . Specifically, the second image generation unit 243 generates a three-dimensional shape based on the two-dimensional shape shown by the three-dimensional model 40 when the three-dimensional model 40 is oriented in a first direction corresponding to the orientation of the object 30 on the examination table 5 . Geometric features of the model 40 are extracted. The second image generator 243 identifies the orientation of the object 30 in the captured image P based on the image of the figure pattern 60 in the captured image P, as shown in FIG.

Specifically, the second image generation unit 243 detects the coordinates of each of the plurality of figures 61 included in the figure pattern 60, and identifies the direction of the vector connecting the figures 61, thereby generating the image corresponding to the direction of the vector. The orientation of the graphic pattern 60 is specified. Then, the second image generation unit 243 generates information on the orientation of the specified graphic pattern 60 and information indicating the arrangement position of the graphic pattern 60 with respect to the object 30 (what kind of position and what kind of orientation with respect to the object 30 (information indicating whether or not the graphic pattern 60 is arranged) is referred to, the orientation of the object 30 corresponding to the identified orientation of the graphic pattern 60 is identified.

In the present embodiment, since the storage unit 23 stores information about the position and orientation of the graphic pattern 60 with respect to the object 30, the second image generation unit 243 The orientation of the object 30 on the inspection table 5 can be identified based on the information and the orientation information identified based on the image of the graphic pattern 60 .

After the orientation of the object 30 is identified, the second image generator 243 generates the stereo model 40 in the CAD space so that the orientation of the object 30 and the orientation of the stereo model 40 correspond (for example, substantially match). The stereo model 40 is oriented in a first orientation by rotating the orientation of the stereo model 40 , enlarging or reducing the stereo model 40 , and/or moving the position of the stereo model 40 .

The second image generation unit 243 extracts the geometric features of the three-dimensional model 40 based on the two-dimensional shape indicated by the three-dimensional model 40 facing the first direction. The geometric features are edge lines of the three-dimensional model 40, for example. In the present embodiment, the second image generator 243 extracts the geometric features of the stereo model 40 based on the two-dimensional shape represented by the stereo model 40 instead of the three-dimensional shape of the stereo model 40 .

The second image generator 243 generates a plane image including the edge line E2 extracted as described above as a second plane image 52, as shown in FIG.

Note that the second image generation unit 243 generates at least the first plane image 51 or the second plane image 52 so that the shape features of the generated second plane image 52 match the shape features of the first plane image 51 . You may perform the process which transforms either. The second image generation unit 243 generates at least one of the first plane image 51 and the second plane image 52 so that the outline shape of the second plane image 52 and the outline shape of the first plane image 51 match, for example. You can transform it.

(Processing for Adjusting Orientation of Solid Model 40)
If the degree of matching between the shape features of the second plane image 52 and the shape features of the first plane image 51 is lower than a predetermined threshold, the second image generator 243 determines the orientation of the stereo model 40 as follows. may be modified to generate the second planar image 52 again.

The second image generation unit 243 first determines the degree of matching between the shape features of the target object 30 indicated by the first planar image 51 and the shape features based on the three-dimensional model 40 corresponding to the first orientation (for example, FIG. 5 6) is equal to or greater than a predetermined threshold. If the degree of matching is less than the threshold, the second image generating unit 243 orients the stereo model 40 in a second direction different from the first direction, and in that state, updates the geometrical shape based on the two-dimensional shape shown by the stereo model 40 . Extract features (referred to as “updated geometric features”).

If the degree of matching between the shape feature of the target object 30 indicated by the first plane image 51 and the updated shape feature is equal to or greater than a threshold, the second image generation unit 243 generates a second plane including the updated shape feature. Generate an image.

According to such a configuration, even when the degree of matching between the shape features of the second plane image 52 and the shape features of the first plane image 51 is low, the direction of the stereo model 40 is changed to obtain the first plane image. It is possible to generate the shape features of the second plane image 52 that have a high degree of matching with the shape features of the one plane image 51 . As a result, the non-matching portion can be identified with higher accuracy.

In the above process, how much the three-dimensional model 40 is moved from the first orientation to the second orientation is appropriately set according to the shape, size, or the like of the three-dimensional model 40 .

When the degree of matching between the shape feature of the target object 30 indicated by the first plane image 51 and the updated shape feature is less than a threshold, the second image generation unit 243 generates the second plane image 52 as described above. Re-extraction may be repeated multiple times. If the degree of matching is less than the threshold even after the re-extraction is performed a predetermined number of times (that is, if the shape features of the second plane image 52 with a high degree of matching with the shape features of the first plane image 51 are not obtained), the first The two-image generation unit 243 may notify the output unit 245 and output to the user that the output unit 245 cannot acquire the second plane image 52 that matches the first plane image 51 .

(Processing for Identifying Mismatched Locations)
The mismatching point identification unit 244 compares the shape features of the object 30 in the first planar image 51 and the shape features of the three-dimensional model 40 in the second plane image 52, and identifies mismatching points where the two do not match. As an example, the non-matching part identification unit 244 identifies a part where the edge line of the object 30 and the edge line of the three-dimensional model 40 corresponding to the edge line are separated by a predetermined threshold or more as a non-matching part. For example, the non-matching part identification unit 244 may refer to threshold information stored in the storage unit 23 in association with the object 30 and identify the non-matching part based on the threshold information.

The non-matching part identification unit 244 determines whether not only the outline shape of the object 30 but also the positions and/or sizes of the holes 31a to 31d, which are structural parts of the object 30, are determined by the hole 31a of the object 30 in the three-dimensional model 40. If the positions and/or sizes of ˜31d are different by a predetermined threshold value or more, that portion is specified as a non-matching portion.

It should be noted that in this type of inspection, the pass/fail of the inspection result differs depending on the kind of product to be inspected or the characteristics of the part to be inspected. For example, if the inspection item is whether or not a predetermined screw is attached according to design data, the differences in the shape features appearing on the image are relatively small. On the other hand, if the inspection item is, for example, whether a relatively large part is attached according to the design data, the difference in the shape features appearing on the image is greater than in the case of screws. Therefore, the storage unit 23 stores different thresholds according to inspection items, and the mismatching part specifying unit 244 uses the thresholds as a reference to specify mismatching parts according to the degree of matching or mismatching of images. good.

In addition, regarding the setting of the threshold, it is also assumed that, for example, as a result of actually performing the inspection, it will be necessary to adjust the extent to which the non-matching portion is determined. Therefore, the control unit 24 of the inspection device 2 may have a function of changing the threshold according to an input from the user or autonomously.

Regarding the setting of the threshold, in order to prevent the threshold from being set to a value outside the predetermined range, the control unit 24 sets the maximum set value or the minimum set value of the threshold stored in the storage unit 23 in association with the inspection target. , the threshold may be prohibited from being set to a value equal to or greater than the set maximum value, or may be prohibited to be set to a value less than the set minimum value.

(output processing)
The output unit 245 outputs information indicating the non-matching portion identified by the non-matching portion identification unit 244 . FIG. 7 is an example of an inspection result image 55 showing inspection results. As an example, the output unit 245 causes the display unit 22 to display the inspection result image 55 of FIG. The inspection result image 55 is an image obtained by superimposing the edge lines E1 of the object 30 and the edge lines E2 of the three-dimensional model 40 on the image of the object 30 captured by the imaging device 1 . Note that the inspection result image 55 may be obtained by superimposing only the edge line E2 of the three-dimensional model 40 on the image of the object 30 .

In this example, the shape of the protruding portion 32 of the three-dimensional model 40 and the shape of the protruding portion 32 of the actual object 30 are different, so this portion is the inconsistent portion 53 . Specifically, as an example, the projecting portion 32 of the actual object 30 is formed shorter than the projecting portion 32 indicated by the design data. Moreover, since the position of the hole 31d of the three-dimensional model 40 and the position of the hole 31d of the object 30 are different, this portion is also a non-matching portion 53. FIG.

Since the inspection result image 55 is displayed on the display unit 22, the user can confirm the mismatching part 53 included in the inspection result image 55 and, for example, whether the target object 30, which is a product, is manufactured according to the design data. can be confirmed. As the information indicating the non-matching portion 53, the edge line E2 generated based on the three-dimensional model 40, which is the design data, may simply be superimposed on the image of the object 30 and displayed, or the non-matching portion may be displayed. Text information indicating that 53 exists, an image or coordinate information indicating the position of the mismatched portion 53, or the like may be displayed.

The manner in which the output unit 245 outputs the non-matching portion can be changed in various ways. The output unit 245 may also output information indicating the position of the mismatched part 53 to the display unit 22 .

In addition to outputting to the display unit 22, the output unit 245 may output, for example, information indicating a mismatched portion as audio information through a speaker (not shown). Further aspects of the output unit 245 will be described later.

(Accumulation of result data)
Note that the control unit 24 further has a data processing unit (not shown), and this data processing unit stores the data of the mismatched portion specified by the mismatched portion specifying unit 244 in a storage (not shown) different from the storage unit 23 or the inspection device 2 . It may be stored in the device. Specifically, the data processing unit may store the data of the discrepancy in association with the product to be inspected and/or in association with the structure identified as the discrepancy.

The data processing unit may identify, for example, an inspection target in which defects frequently occur when a mismatching portion is specified at a number and/or frequency equal to or greater than a certain reference value. The data processing unit may also present to the user that the threshold value for the inspection target is changed when mismatched points are identified with a number and/or frequency equal to or greater than a certain reference value.

The data processing unit lists the mismatching points identified by the mismatching point identification unit 244 in association with the product to be inspected and/or the structural part identified as the mismatching point, and presents the listed data to the user. good. According to such a configuration, the user sequentially confirms one or more inconsistent points displayed in the list, and, for example, attaches a predetermined part to the target object 30 to erase the inconsistent points. can be easily done.

(Operation of inspection system S100)
The operation of the inspection system S100 configured as described above will be described below. FIG. 8 is an example of an operation flow by the inspection system S100. As a premise for the inspection by the inspection apparatus 2, it is assumed that the object 30 to which the figure pattern 60 is attached is placed on the inspection table 5 by the user.

First, in step S1, the image acquisition unit 241 acquires a captured image P (see FIG. 4) generated by the imaging device 1 capturing an image of the target object 30. FIG.

Next, in step S2, the first image generating section 242 generates the first planar image 51 (see FIG. 5) based on the captured image P acquired by the image acquiring section 241. FIG. As described above, the first image generator 242 extracts the edge lines E1 from the two-dimensional image of the object 30 and generates the first planar image 51 including the edge lines E1.

Next, in step S<b>3 , the second image generator 243 identifies the orientation of the graphic pattern 60 based on the image of the graphic pattern 60 . Specifically, the second image generator 243 identifies the orientation of the graphic pattern 60 by identifying the orientation of a vector connecting the plurality of graphics 61 of the graphic pattern 60, for example. Then, the second image generation unit 243 generates information on the orientation of the specified graphic pattern 60 and information indicating the arrangement position of the graphic pattern 60 with respect to the object 30 (what kind of position and what kind of orientation with respect to the object 30 (information indicating whether or not the figure pattern 60 is arranged), and the orientation of the target object 30 is specified.

The second image generator 243 reads the data of the three-dimensional model 40 corresponding to the object 30 from the storage unit 23, and when the orientation of the object 30 is specified, the orientation of the object 30 corresponds to the orientation of the three-dimensional model 40. The direction of the three-dimensional model 40 is changed so that the three-dimensional model 40 is oriented in the first direction.

Next, in step S4, the second image generation unit 243 extracts the shape feature (edge line E2) of the three-dimensional model 40 directed in the first direction based on the two-dimensional shape shown by the three-dimensional model 40. Then, the second image generator 243 generates a plane image including the extracted edge line E2 as the second plane image 52, as shown in FIG.

In the steps up to this point, a first planar image 51 based on an image of the object 30 obtained by actually photographing the object 30 and a second planar image 52 based on the three-dimensional model 40 which is the design data of the object 30 and are generated.

Next, in step S<b>5 , the non-matching point identification unit 244 identifies the non-matching point 53 by comparing the shape features of the first plane image 51 and the shape features of the second plane image 52 . In the present embodiment, as described above with reference to FIG. 7, the locations of the projections 32 and the locations of the holes 31d in the object 30 match the shape of the actual object 30 and the shape of the three-dimensional model 40. A non-matching portion 53 is obtained.

Next, in step S<b>6 , the output unit 245 outputs the inspection result image 55 including information indicating the non-matching part 53 to the display unit 22 . In the inspection result image 55 displayed on the display unit 22 , the image of the edge line E2 extracted based on the three-dimensional model 40 is superimposed on the image of the object 30 .

(Action and effect of the first embodiment)
As described above, according to the configuration of the present embodiment, the inspection device 2 identifies the location where the shape feature of the object 30 and the shape feature of the three-dimensional model 40 do not match, and the inspection result image 55 is displayed on the display unit 22. be done. Therefore, the user can confirm, for example, whether the target object 30 is manufactured according to the shape of the three-dimensional model 40 . In particular, the inspection apparatus 2 uses the first planar image 51, which is a two-dimensional image generated based on the captured image of the object 30 instead of the three-dimensional shapes of the object 30 and the three-dimensional model 40, and the three-dimensional model 40. Since the non-matching portion 53 is specified using the second planar image 52 which is a two-dimensional image generated by the above method, the calculation cost in the arithmetic processing for specifying the non-matching portion 53 can be reduced.

According to the configuration of this embodiment, the inspection apparatus 2 identifies the orientation of the object 30 by identifying the orientation of the graphic pattern 60 arranged in a predetermined positional relationship with respect to the object 30 . In other words, even if the object 30 placed on the inspection table 5 is not oriented in a strictly predetermined direction with respect to the imaging device 1, the inspection apparatus 2 can detect the object 30 placed on the inspection table 5 It is possible to specify a mismatched portion according to the orientation of the . Therefore, it is not necessary for the user to precisely position and place the object 30 on the inspection table 5 for inspection. As a result, the object 30 can be inspected with good workability.

Further, the second image generation unit 243 generates the stereo model 40 when the degree of matching between the shape feature of the target object 30 indicated by the first planar image 51 and the shape feature based on the stereo model 40 is less than a threshold. In a second direction different from the first direction, the shape features are extracted again based on the stereo model 40 in that state, a second planar image 52 is generated, and inconsistent portions are identified. According to such a configuration, even when the degree of matching between the shape features of the second plane image 52 and the shape features of the first plane image 51 is low, the direction of the stereo model 40 is changed to obtain the first plane image. A second planar image 52 having a high degree of matching with the shape features of the planar image 51 can be generated, and a non-matching portion can be specified with high precision.

<Second embodiment>
FIG. 9 is a diagram showing an inspection system S100A of the second embodiment. The inspection system S100A photographs the object 30 placed on the inspection table 5 via the jig 70 . The inspection system S100A differs from the inspection system S100 of the above-described embodiment in that the orientation of the object 30 is specified based on the image of the jig graphic pattern 60A provided on the jig 70 . Other than that, it is the same as the inspection system S100 of the first embodiment, so redundant description will be omitted.

As shown in FIG. 9 , the jig 70 has a support surface 71 on which the object 30 is placed, and an alignment portion 72 for aligning the object 30 on the support surface 71 . The jig 70 is not limited to the configuration shown in FIG. It is not limited to the configuration placed on the

The jig 70 has a figure pattern 60A. The figure pattern 60A is arranged in a predetermined positional relationship with respect to the jig 70. As shown in FIG. The figure pattern 60A is arranged on the support surface 71 as an example. The figure pattern 60A is a square grid pattern including a plurality of squares in this example, but may be a dot pattern including circular dots as in the first embodiment.

In this embodiment, the graphic pattern 60A is positioned such that it is covered by the object 30 when the object 30 is placed on the support surface 71 . However, the position of the figure pattern 60A is arbitrary, and the figure pattern 60A may be arranged at a position where it can be photographed by the imaging device 1 without being covered by the object 30 when the object 30 is arranged.

The jig 70 is placed on the inspection table 5 by the user, for example, when the object 30 is inspected. The position of the jig 70 may be fixed with respect to the inspection table 5 by a fixture (not shown). The jig 70 may be arranged at a predetermined position with respect to the inspection table 5 , for example, by an engaging means such as an uneven structure provided between the jig 70 and the inspection table 5 .

(Processing for Identifying Orientation of Object 30)
The inspection device 2 of the inspection system S100A has a control section 24 (FIG. 1) as in the first embodiment. The image acquisition unit 241 acquires a captured image of the jig 70 placed on the inspection table 5 without the object 30 placed thereon, captured by the inspection apparatus 2 . An image of the figure pattern 60A is also shown together with the jig 70 in the captured image.

Based on the object 30 positioned on the jig 70, the first image generation unit 242 extracts the shape features of the object 30 and generates the first planar image 51, as in the first embodiment.

The second image generator 243 identifies the orientation of the jig 70 on the inspection table 5 based on the image of the figure pattern 60A of the jig 70 . The orientation of the figure pattern 60A can be identified by the same method as in the first embodiment. In this embodiment, the graphic pattern 60 is not attached to the object 30, but the jig 70 is provided with the graphic pattern 60A. Since the jig 70 is a member that positions the object 30, if the orientation of the jig 70 is specified, the orientation of the object 30 positioned on the jig 70 is also indirectly specified. .

Therefore, the second image generator 243 identifies the orientation of the jig 70 based on the image of the graphic pattern 60A, just as the orientation of the object 30 is identified based on the graphic pattern 60 in the first embodiment.

After that, the second image generation unit 243 orients the three-dimensional model 40 in the first orientation so that the orientation corresponding to the orientation of the object 30 corresponding to the specified orientation of the jig 70 corresponds to the orientation of the three-dimensional model 40. turn. Specifically, the second image generation unit 243 refers to information indicating the relationship between the orientation of the jig 70 and the orientation of the target object 30 stored in the storage unit 23, specifies the orientation of the target object 30, and determines the orientation of the target object 30. The three-dimensional model 40 is oriented in a first orientation so as to correspond to the orientation of the object 30. - 特許庁

After orienting the three-dimensional model 40 in the first direction, the second image generation unit 243 extracts the shape features of the three-dimensional model 40 and generates the second planar image 52, as in the first embodiment.

After that, as in the first embodiment, the mismatching part identifying unit 244 compares the first planar image 51 and the second planar image 52 (see FIG. 2) to identify the mismatching part, and the output part 245 identifies the mismatching part. Output information.

In this embodiment, the figure pattern 60A is provided on the jig 70 instead of the figure pattern 60 being attached to the object 30 as in the first embodiment. Therefore, the variation in the direction specified by the second image generation unit 243 due to the positional deviation between the graphic pattern 60 and the object 30 that occurs when the graphic pattern 60 is attached to the object 30 is reduced, and the object It becomes possible to specify the orientation of 30 with high accuracy.

In this embodiment, once the orientation of the jig 70 is specified, the orientation of the object 30 is specified by analyzing the image of the graphic pattern 60 attached to the object 30 each time the object 30 is inspected. No need. That is, it is not necessary for the object 30 to have the graphic pattern 60 arranged thereon. However, the graphic pattern 60 may be arranged on the object 30, and the inspection apparatus 2 may identify the non-matching portion of the object 30 arranged on the jig 70 by the same method as in the first embodiment.

<Third Embodiment>
In the above-described embodiment, the orientation of the target object 30 is identified using the graphic pattern 60 of the target object 30 or the graphic pattern 60A of the jig 70 . In this embodiment, identifying the orientation of the object 30 based on images of multiple structures of the object itself is described. FIG. 10 is an example of an image of the object 30 captured by the imaging device 1 . The object 30 has four circular holes 31a to 31d of the same shape. The configuration of the inspection system S100 is the same as that of the first embodiment.

In this embodiment, the image acquisition unit 241 acquires the captured image P generated by capturing the object 30 by the imaging device 1 . The object 30 is placed on the inspection table 5 by the user, and no graphic pattern 60 is attached to the object 30 .

Based on the captured image P, the first image generation unit 242 generates the first planar image 51 (see FIG. 5) by extracting the shape feature from the image of the target object 30, as in the first embodiment.

The second image generator 243 identifies the orientation of the object 30 based on the images of the holes, which are the plurality of structural parts formed in the object 30 . Specifically, the second image generator 243 identifies the holes 31a to 31d included in the image of the object 30 in the captured image P. As shown in FIG. After specifying the holes 31a to 31d, the second image generation unit 243 refers to the three-dimensional CAD data, which is the design data, and determines whether each of the holes 31a to 31d has a predetermined position and/or a predetermined shape. do.

The reason for performing such a process is that the process of specifying the direction of the object 30 according to this embodiment is performed so that the plurality of structural parts of the object 30 are formed at positions and/or shapes within a predetermined threshold with respect to the design data. This is because it is assumed that In this embodiment, it is assumed that the hole 31d among the holes 31a to 31d is formed at an incorrect position with respect to the design data. The second image generator 243 determines that the holes 31a to 31c are at predetermined positions and shapes, and the hole 31d is not at a predetermined position and/or shape.

In order to specify the orientation of the target object 30 based on the structural parts of the target object 30, the number of reference structural parts must be at least three or more. Therefore, the second image generation unit 243 determines whether or not there are a reference number (at least three) or more of structural parts formed in a predetermined position and/or in a predetermined shape, If the number is equal to or greater than the number of structural parts, a process of specifying the orientation of the target object 30 is executed based on the images of the plurality of structural parts.

In this embodiment, the second image generator 243 identifies the orientation of the object 30 based on the images of the holes 31a to 31c. Specifically, the second image generator 243 identifies the orientation of the object 30 by identifying the orientation of the vector connecting the images of the holes 31a to 31c.

The second image generator 243 orients the three-dimensional model 40 in the first direction, as in the first embodiment, so that the orientation of the object 30 specified as described above corresponds to the orientation of the three-dimensional model 40. . The second image generator 243 then extracts the shape features of the three-dimensional model 40 to generate a second planar image, as in the first embodiment.

Based on the first planar image 51 and the second planar image 52 generated in this way, as in the first embodiment, the non-matching part identification unit 244 specifies the non-matching part, and the output part 245 outputs information on the non-matching part. Output.

According to such a configuration, for example, the orientation of the object 30 can be specified using the image of the structural part of the object 30 itself without attaching the figure pattern 60 to the object 30, and the non-matching portion can be specified. can be implemented.

In the above description, the direction of the object 30 is specified based on the images of the holes 31a to 31c of the object 30. The orientation of the object 30 may be identified based on the image.

<Fourth Embodiment>
FIG. 11 is a diagram showing an example of an inspection system S100B that includes multiple imaging devices 1. As shown in FIG. An inspection system S100B in FIG. 11 includes a first imaging device 1a and a second imaging device 1b. The first imaging device 1a photographs the object 30 from a first direction, and the second imaging device 1b photographs the object 30 from a second direction different from the first direction. As an example, the first imaging device 1a photographs the object 30 from above the object 30, and the second imaging device 1b photographs the object 30 from the right side of the object 30. FIG.

The image acquisition unit 241 of the inspection device 2 acquires a plurality of captured images generated by capturing images of the object 30 from a plurality of directions by the first imaging device 1a and the second imaging device 1b.

The first image generation unit 242 extracts the shape features of the target object 30 viewed from above based on the first captured image generated by the first imaging device 1a, and generates a first planar image. The first image generator 242 also extracts the shape features of the object 30 viewed from the right side based on the second captured image from the second imaging device 1b, and generates another first planar image. Generate.

Although illustration is omitted in FIG. 11, as an example, the graphic pattern is arranged at a position photographed by both the first imaging device 1a and the second imaging device 1b. Alternatively, the graphic pattern captured by the first imaging device 1a and the graphic pattern captured by the second imaging device 1b may be provided on the object 30 separately.

The second image generator 243 identifies the orientation of the target object 30 based on the first captured image from the first imaging device 1a, and generates a second planar image corresponding to the orientation. The second image generator 243 also identifies the orientation of the target object 30 based on the second captured image from the second imaging device 1b, and generates another second planar image corresponding to the orientation.

The inconsistent part identification unit 244 compares the first planar images and the second planar images in a plurality of directions generated as described above, and identifies inconsistent parts in each direction. The output unit 245 outputs information on the non-matching portions in the plurality of directions identified by the non-matching portion identification unit 244 .

As described above, the inspection apparatus of the present embodiment captures a plurality of images of the object 30 based on the first planar image and the second planar image generated based on the respective captured images acquired from the plurality of imaging devices. It is also possible to identify discrepancies in orientation.

<Fifth Embodiment>
FIG. 12 is a block diagram showing a configuration in which the output unit 245 outputs to the projection device 25. As shown in FIG. The inspection system S100C further includes a projection device 25 in addition to the configuration of FIG. The projection device 25 is an optical projector, for example, and has a light output unit (not shown) that outputs light.

The projection device 25 projects light representing arbitrary information such as image information and character information toward the projection target according to the control signal from the output unit 245 .

In the present embodiment, the output unit 245 transmits a control signal to the projection device 25, and the projection device 25 projects light indicating, for example, the position of the non-matching portion onto the object 30 according to the received control signal. . The projection device 25 may also project light indicative of the position and/or shape of features of the design object 30 represented by the design data.

According to the configuration in which the projection device 25 projects light onto the target object 30 in this way, the user can see the information represented by the light projected onto the target object 30 and can confirm, for example, a non-matching portion. Moreover, according to the configuration in which the projection device 25 posts light indicating the position and/or shape of the designed structural portion to the object 30, the user can confirm the correct position and shape of the designed structural portion.

Note that the projection device 25 may project light onto something other than the object 30 instead of projecting the light directly onto the object 30 . The projection device 25 is not necessarily limited to a device that projects light representing image information or character information, and may be a device such as a laser pointer. Also, together with the projection device 25 for projecting light, or instead of the projection device 25, there may be provided a hardware device capable of pointing to any part of the object. The hardware device may include, for example, a specific pointer that points to an arbitrary location on the object 30 according to the output from the output unit 245, and a drive unit that moves the pointer.

The output unit 245 may output information indicating the position and/or shape of the designed structural part to the display unit 22 for display. Further, the output unit 245 may output guidance information to the display unit 22, for example, indicating in what procedure the mismatched portion identifying unit 244 processes or repairs the specific portion.

<Modification 1>
Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist thereof. be. In the above embodiment, the number of objects 30 is one, but the inspection system S100 may be configured to inspect a plurality of objects 30 at the same time.

As an example, the image acquisition unit 241 of the inspection device 2 acquires one or more captured images generated by photographing the plurality of objects 30 , and the mismatched part identification unit 244 detects the mismatch for each of the plurality of objects 30 You can specify the location.

In this case, the image acquisition unit 241 may acquire a captured image captured by one imaging device, or may acquire a plurality of captured images captured by a plurality of imaging devices.

<Modification 2>
In the above embodiment, the stereo model 40 is oriented in the direction corresponding to the orientation of the figure pattern 60 to generate the second plane image. The second planar image may be determined by selecting one second planar image from . Specifically, the second image generation unit 243 of the inspection apparatus 2 refers to the plurality of second plane images generated based on the stereo model 40, and selects, for example, the first plane image from among the plurality of second plane images. Select the one with the highest similarity to the planar image. The second image generator 243 may thus select one second plane image from a plurality of second plane image candidates. The plurality of second planar images may include, for example, geometric features extracted from two-dimensional images shown by the three-dimensional model 40 oriented in different directions.

<Modification 3>
Even if the object 30 contains some manufacturing or assembly defects, the defect condition may be resolved by, for example, the user correcting the defects. Therefore, in the inspection system S100, the imaging device 1 may continuously photograph the object 30, and the inspection device 2 may perform the following processing. In addition, the imaging device 1 photographs the object 30 at predetermined time intervals, for example. The time intervals are arbitrary and are not limited to fixed time intervals. The number of shots may be two or more.

The first image generation unit 242 of the inspection device 2 generates a first plane image different from the already generated first plane image based on the captured image of the object 30 captured by the imaging device 1, and generates a first plane image. Update the planar image.

In order to generate the second plane image 52 corresponding to the updated first plane image 51 , the second image generation unit 243 already generates the second plane image 52 based on the updated captured image of the object 30 captured by the imaging device 1 . A second planar image that is different from the second planar image that has been generated is generated. The mismatched part identifying unit 244 compares the updated first planar image and the second planar image generated in this way to identify the mismatched part.

Note that even when the imaging device 1 continuously captures the target object 30, the mismatching part specifying unit 244 does not need to perform the mismatching part specifying process each time a captured image of the target object 30 is generated. For example, the inconsistent part identification unit 244 may compare the first planar image after updating and the first planar image before updating so that the process of identifying the inconsistent part is performed only when the user corrects the defective part of the target object 30 . is less than a predetermined threshold, the non-matching portion may be specified based on the updated first planar image.

In the case where the imaging device 1 is configured to continuously image the object 30 as described above, the inspection device 2 may be further configured as follows. Specifically, after identifying the mismatching portion based on the first planar image, the mismatching portion identifying section 244 again identifies the mismatching portion based on the updated first planar image. Then, the non-matching part identification unit 244 determines whether or not the number or range of non-matching parts is less than a predetermined reference value. This reference value may be set to a different value according to the shape and/or characteristics of the structure of the target object 30 identified as the non-matching portion, and may be stored in the storage unit 23 as an example.

The output unit 245 outputs information indicating the fact to the display unit 22 when the mismatching part specifying unit 244 determines that the number or range of mismatching parts of the object 30 is less than a predetermined reference value. The output unit 245 causes the display unit 22 to display character information such as “○” and “X” or “Pass” and “Fail”, for example.

<Modification 4>
The inspection apparatus 2 according to one embodiment of the present invention may output work information associated with the portion where the output unit 245 is the mismatched portion 53 as described below. Specifically, first, the non-matching part identification unit 244 of the inspection device 2 identifies which part of the object the one or more identified non-matching parts 53 are. Work information is read from the storage unit 23, for example.

Specifically, for example, assume that the non-matching portion 53 is a portion of a certain object in which "yy screw is tightened using xx driver". In this case, the inconsistent part identification unit 244 identifies the inconsistent part 53 and reads the work information (“tighten the yy screw using the xx driver”) associated with that part. Next, the output unit 245 outputs the work information read by the inconsistent part identification unit 244 so as to present it to the user. The output unit 245 outputs work information to the display unit 22, for example.

According to such a configuration, it is possible to automatically present to the user the tools and the like necessary for the work for repairing the defective portion according to the defective portion, so that what kind of work the user should perform. can be easily grasped. In the above description, screws, screwdrivers, and the like have been described as examples of work information. / Or at least one of information indicating required time and the like may be included.

According to the configuration as described above, when the object 30 includes some manufacturing or assembly defect and the user corrects the defect, the determination result by the non-matching part identification unit 244 is, for example, , the status changes from "failed" in which unmatched portions remain to "accepted", so that the user can confirm that the processing and repair of the object 30 have been completed by looking at the inspection results.

Note that the imaging device 1 may capture the object 30 for the second and subsequent times, for example, when an input instruction from the user is received instead of at predetermined time intervals.

The present invention has been described above using the embodiments, but as described above, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications can be made within the scope of the gist thereof. and can be changed. For example, all or part of the device can be functionally or physically distributed and integrated in arbitrary units. In addition, new embodiments resulting from arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

1 imaging device 1a first imaging device 1b second imaging device 2 inspection device 5 inspection table 21 input device 22 display unit 23 storage unit 24 control unit 25 projection device 30 object 31 main body 31a to 31d hole 32 projection 40 three-dimensional model 51 First planar image 52 Second planar image 53 Mismatched portion 55 Inspection result image 60 Graphic pattern 60A Graphic pattern 61 Graphic 70 Jig 71 Support surface 72 Alignment unit 241 Image acquisition unit 242 First image generation unit 243 Second image generation Unit 244 Mismatched part identification unit 245 Output unit E1 Edge line E2 Edge line P Captured images S100, S100A to S100C Inspection system

Claims (20)

to the computer,
an image acquisition step of acquiring a captured image generated by capturing an image of an object placed on an inspection table with an imaging device;
a first image generating step of extracting a shape feature of the object based on the captured image and generating a first planar image including the extracted shape feature;
The stereo model represented by the three-dimensional CAD data of the object is oriented in a first direction corresponding to the orientation of the object on the examination table, and the stereo model is based on a two-dimensional shape shown by the stereo model. a second image generation step of extracting a shape feature of and generating a second planar image containing the extracted shape feature;
a mismatching point identifying step of identifying mismatching points where the geometric features of the object in the first planar image and the geometric features of the stereo model in the second planar image do not match;
an output step of outputting information indicating the inconsistent portion;
program to run the In the image acquisition step,
Acquiring the captured image generated by capturing an image of a figure pattern arranged in a predetermined positional relationship with respect to the object and the object together with the imaging device,
The second image generation step includes:
identifying the orientation of the object based on the acquired image of the graphic pattern;
generating the second planar image by orienting the three-dimensional model in the first direction so that the orientation of the identified object and the orientation of the three-dimensional model correspond;
having
A program according to claim 1. In determining the orientation of the object,
The object corresponding to the specified orientation of the figure pattern by specifying the orientation of the figure pattern based on the image of the figure pattern and referring to information indicating the positional relationship between the object and the figure pattern. identify the orientation of the
3. A program according to claim 2. In the image acquisition step,
Acquiring the imaged image generated by imaging a jig figure pattern arranged in a predetermined positional relationship with respect to a jig for positioning the object, which is placed on the inspection table;
The second image generation step includes:
identifying the orientation of the jig based on the acquired image of the jig figure pattern;
generating the second planar image by orienting the three-dimensional model in the first direction so that the orientation of the object corresponding to the identified orientation of the jig corresponds to the orientation of the three-dimensional model;
4. The program according to any one of claims 1 to 3, comprising: In the second image generating step, the direction of the figure pattern is specified by specifying the direction of a vector connecting a plurality of figures included in the figure pattern.
4. A program according to claim 2 or 3. The second image generation step includes:
determining whether or not a degree of matching between the shape features of the object indicated by the first planar image and the shape features based on the three-dimensional model corresponding to the first orientation is equal to or greater than a threshold;
a step of extracting an updated shape feature based on a two-dimensional shape shown by the three-dimensional model while being oriented in a second direction different from the first direction, if the degree of matching is less than the threshold;
generating the second planar image including the updated geometrical features when a degree of matching between the geometrical features of the object indicated by the first planar image and the updated geometrical features is equal to or greater than the threshold; a step;
6. The program according to any one of claims 1 to 5, comprising: The second image generation step includes:
referring to the plurality of second planar images generated based on the stereo model, and selecting one of the plurality of second planar images that has the highest degree of similarity to the first planar image;
A program according to any one of claims 1 to 6. The second image generation step includes:
determining an orientation of the object based on images of a plurality of structures formed on the object;
generating the second planar image by orienting the three-dimensional model in the first direction so that the orientation of the identified object and the orientation of the three-dimensional model correspond;
having
A program according to any one of claims 1 to 7. Identifying the orientation of the object based on the images of the plurality of structures comprises:
determining whether or not each structural portion has a predetermined position and/or a predetermined shape by referring to the three-dimensional CAD data; determining the orientation of the object based on the images of the plurality of structures, if
9. A program according to claim 8. The image acquisition step includes:
Acquiring a plurality of the captured images generated by photographing the object from a plurality of directions with a plurality of the imaging devices,
The discrepancy identifying step includes:
Based on the first planar image and the second planar image generated based on each of the captured images, identifying the inconsistent points in a plurality of orientations of the object,
A program according to any one of claims 1 to 9. The image acquisition step includes:
Acquiring the captured images generated by one or more of the imaging devices capturing a plurality of the objects,
The discrepancy identifying step includes:
identifying the non-matching location for each of the plurality of objects;
A program according to any one of claims 1 to 10. The first image generation step includes:
continuously photographing the object with the imaging device to update the first planar image;
The discrepancy identifying step includes:
identifying the inconsistent portion based on the first planar image after updating when the degree of matching between the first planar image after updating and the first planar image before updating is less than a predetermined threshold; include,
A program according to any one of claims 1 to 11. wherein the geometric features of the object and the geometric features of the solid model are edge lines of the object and the solid model;
In the discrepancy identifying step,
identifying a location where the edge line of the object and the edge line of the stereo model corresponding to the edge line of the object are separated by a predetermined threshold or more as the mismatch location;
A program according to any one of claims 1 to 12. The output step includes:
outputting an inspection result image that is an image obtained by superimposing the edge line, which is the shape feature of the stereo model, on the image of the object in the captured image;
14. A program according to any one of claims 1-13. In the output step, for the object,
Light indicating the position of the mismatched portion, or
Projecting light indicating the position and/or shape of the structural part of the design object represented by the three-dimensional CAD data;
15. A program according to any one of claims 1-14. The output step includes:
information indicating that the mismatched portion exists;
Information indicating the position of the mismatched part, or
Information on the position and/or shape of the structural part of the design object represented by the three-dimensional CAD data,
including a step of outputting to the display unit,
16. A program according to any one of claims 1-15. The discrepancy identifying step includes:
After identifying the inconsistent portion based on the first planar image, re-identifying the inconsistent portion based on the updated first planar image;
The output step includes:
When the number or range of mismatched points identified in the step of re-identifying the mismatched points based on the updated first planar image is less than a predetermined reference value, outputting information to that effect to the display unit. do,
17. A program according to any one of claims 1-16. The output step includes:
including the step of outputting work information associated with the inconsistent portion;
18. A program according to any one of claims 1-17. the computer runs
an image acquisition step of acquiring a captured image generated by capturing an image of an object placed on an inspection table with an imaging device;
a first image generating step of extracting a shape feature of the object based on the captured image and generating a first planar image including the extracted shape feature;
The shape of the three-dimensional model represented by the three-dimensional CAD data of the object based on the two-dimensional shape shown by the three-dimensional model when the object is oriented in the direction corresponding to the orientation of the object on the examination table. a second image generating step of extracting a physical feature and generating a second planar image including the extracted shape feature;
a mismatching point identifying step of identifying mismatching points where the geometric features of the object in the first planar image and the geometric features of the stereo model in the second planar image do not match;
an output step of outputting information indicating the inconsistent portion;
inspection method. an image acquisition unit that acquires a captured image generated by capturing an image of an object placed on an inspection table with an imaging device;
a first image generation unit that extracts a shape feature of the object based on the captured image and generates a first planar image that includes the extracted shape feature;
The shape of the three-dimensional model represented by the three-dimensional CAD data of the object based on the two-dimensional shape shown by the three-dimensional model when the object is oriented in the direction corresponding to the orientation of the object on the examination table. a second image generation unit that extracts a physical feature and generates a second planar image that includes the extracted shape feature;
a mismatching point identification unit that identifies mismatching points where the shape features of the object in the first plane image and the shape features of the stereo model in the second plane image do not match;
an output unit that outputs information indicating the inconsistent part;
inspection device.