WO2020003887A1

WO2020003887A1 - External-appearance inspection system, method for displaying external-appearance inspection result, and program for displaying external-appearance inspection result

Info

Publication number: WO2020003887A1
Application number: PCT/JP2019/021682
Authority: WO
Inventors: 加藤　豊; 正広高山; 佑二山内; 伸悟稲積
Original assignee: オムロン株式会社
Priority date: 2018-06-27
Filing date: 2019-05-31
Publication date: 2020-01-02
Also published as: JP7131127B2; JP2020003300A

Abstract

The present invention provides assistance in a procedure for confirming the results of external-appearance inspection performed by acquiring images of an inspection target (W) from a plurality of points of view while moving an image-acquisition device (10). An external-appearance inspection system (1) for performing external-appearance inspection on the inspection target (W) is provided with: a display device (23); a robot (30) for moving the image-acquisition device (10); an inspection unit (21) for inspecting, on the basis of individual images obtained from the image-acquisition device (10) as a result of the image-acquisition device (10) respectively acquiring images of a plurality of inspection sites of the inspection target (W) while the robot (30) is moving the image-acquisition device (10), each of the plurality of inspection sites for the presence/absence of a defect; and a display control unit (22) for causing the display device (23) to display an inspection-result matrix (25) representing, for the respective inspection targets (W), the?] inspection results for each of the plurality of inspection sites obtained by the inspection unit (21).

Description

Appearance inspection system, appearance inspection result display method, and appearance inspection result display program

The present disclosure relates to an appearance inspection system that inspects an inspection object using a captured image, a method for displaying an appearance inspection result, and a display program for an appearance inspection result.

外観 A number of visual inspection systems for inspecting inspection objects such as resins, metals, and substrates using image processing techniques have been proposed.

For example, Japanese Patent Application Laid-Open No. 2013-213323 (Patent Document 1) discloses an appearance inspection system for inspecting the quality of a substrate. The appearance inspection system displays an inspection result of the board on a two-dimensional map image. In the two-dimensional map image, the serial numbers of the boards are arranged on the horizontal axis, and the component information in the boards is arranged on the vertical axis. In the two-dimensional map image, cells are provided for each combination of the serial number of the board and the component information in the board, and the inspection result of each component on each board is displayed on the corresponding cell.

JP 2013-213323 A

外観 The appearance inspection system disclosed in Patent Document 1 inspects the appearance of a substrate by a fixed imaging device imaging a substrate being conveyed on a production line. When the imaging device is fixed, the number of places on the substrate where images can be taken is also limited, so that the number of images used for inspection is relatively small.

On the other hand, in a visual inspection system that captures an image of an inspection target while moving the imaging device, one inspection target can be imaged from all directions, and thus the number of images used for inspection tends to be enormous. The number of images used for the appearance inspection of one inspection object may be several hundred times. As a result, the output inspection results become enormous, and it becomes difficult for the user to check the inspection results one by one.

The present disclosure has been made in order to solve the above-described problems, and an object in one aspect is to perform a visual inspection performed by imaging an inspection object from a plurality of viewpoints while moving an imaging device. It is an object of the present invention to provide a visual inspection system capable of supporting the confirmation work of the result of the above. An object in another aspect is to provide a display method capable of supporting a check operation of a result of an appearance inspection performed by imaging an inspection target from a plurality of viewpoints while moving an imaging device. . Another object of the present invention is to provide a display program capable of supporting a check operation of a result of a visual inspection performed by imaging an inspection target from a plurality of viewpoints while moving an imaging device. .

In an example of the present disclosure, a visual inspection system that performs a visual inspection of an inspection target includes a display device, a robot for moving the imaging device, and the imaging device while the robot is moving the imaging device. An inspection unit configured to inspect each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained from the imaging device by imaging each of the plurality of inspection parts of the inspection object; and And a display control unit for displaying, on the display device, an inspection result matrix in which inspection results of the inspection unit for each of the inspection parts are displayed for each inspection object.

According to this disclosure, the inspection result of each inspection part of each inspection object is represented in two dimensions, so that the user can immediately grasp which inspection part of which inspection object has a defect. it can. The display of the inspection result in two dimensions becomes effective as the inspection result becomes enormous.

{In one example of the present disclosure, each row or each column of the inspection result matrix is grouped in row units or column units according to a predetermined classification rule. The display control unit, when receiving an aggregation instruction for a group of rows or a group of columns that are grouped, displays the inspection results of the group of rows or the group of columns in one row or one column, and displays the inspection result. When a deployment instruction is received for the inspection result that is displayed as a result, the display of the inspection result that is collectively displayed is returned to the state before the aggregation.

According to this disclosure, even when the number of rows or columns of the inspection result matrix is too large to be displayed on the display device, the user can efficiently check the inspection results by utilizing the aggregated display and the expanded display. can do.

In one example of the present disclosure, the display control unit displays the inspection result indicating a defect among the inspection results included in the inspection result matrix in a display mode different from other inspection results.

According to this disclosure, the inspection result indicating a defect is displayed in a display mode different from other inspection results, so that the user can immediately determine the inspection result indicating a defect, and It is possible to more easily grasp which inspection part has a defect.

In an example of the present disclosure, the visual inspection system accepts a selection operation of selecting one or more inspection results from the inspection results included in the inspection result matrix, and thereby performs one or more inspection objects and one inspection result. The operation unit capable of selecting the above inspection part and the inspection unit or the display control unit are selected as the selected inspection target based on the operation unit receiving the selection operation. And performing processing relating to at least one of the inspection parts.

According to the present disclosure, it is possible to provide an interface for easily specifying a combination of an inspection object and an inspection portion.

In one example of the present disclosure, the display control unit displays at least one of an image used for the inspection of the selected inspection part and an imaging condition of the image on the display device.

According to the present disclosure, the user can easily confirm the imaging condition of an arbitrary inspection portion indicated in the inspection result matrix.

In one example of the present disclosure, the visual inspection system further includes a storage device for storing a three-dimensional model representing a shape of the inspection object. The plurality of inspection parts are set in advance for the three-dimensional model. The display control unit displays the three-dimensional model on the display device, and displays an inspection result of the selected inspection part in a corresponding part on the three-dimensional model.

According to this disclosure, the user can check the inspection result of an arbitrary inspection part on the three-dimensional model, and can easily check which part of the inspection target has a defect.

In one example of the present disclosure, when a plurality of inspection results are selected by the selection operation, the display control unit displays a statistical result of the plurality of inspection results on the display device.

According to this disclosure, the user can easily analyze the inspection result for an arbitrary inspection portion.

In one example of the present disclosure, the inspection unit re-inspects the selected inspection part based on the image used for the inspection of the selected inspection part.

According to this disclosure, the user can easily execute the reexamination of an arbitrary inspection portion.

In an example of the present disclosure, the display control unit displays, on the display device, a comparison result between the expected value matrix indicating a true correct value for each test result included in the test result matrix and the test result matrix. .

According to this disclosure, the user can easily determine whether or not each inspection result indicated in the inspection result matrix is as expected. Such comparison processing becomes more effective as the number of test results shown in the test result matrix 25 increases.

In another example of the present disclosure, a method for displaying an appearance inspection result performed by an imaging device imaging a plurality of inspection portions of an inspection target includes the imaging device while the robot is moving the imaging device. Acquiring each image obtained by imaging each of the plurality of inspection parts, and inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained in the acquiring step. And displaying on a display device an inspection result matrix representing inspection results for each of the plurality of inspection portions for each inspection object.

In another example of the present disclosure, a display program of an appearance inspection result performed by an imaging device imaging a plurality of inspection portions of an inspection target includes, while a computer is moving the imaging device, a robot. A step of acquiring each image obtained by the imaging device imaging each of the plurality of inspection parts; and a step of acquiring a defect for each of the plurality of inspection parts based on each image obtained in the acquiring step. The step of inspecting the presence / absence and the step of displaying on a display device an inspection result matrix representing the inspection results for each of the plurality of inspection portions for each inspection object are executed.

において In a certain situation, it is possible to support the work of confirming the result of the appearance inspection performed by imaging the inspection target object from a plurality of viewpoints while moving the imaging device.

1 is a schematic diagram showing an outline of a visual inspection system according to an embodiment. FIG. 5 is a diagram showing an example of an inspection result displayed on a display device of the image processing device according to the embodiment. FIG. 3 is a diagram showing a modification of the inspection result matrix shown in FIG. 2. FIG. 4 is a diagram illustrating an example of a data structure of a classification rule. FIG. 4 is a diagram illustrating an example of a data structure of a classification rule. FIG. 11 is a diagram showing screen transition of an inspection result matrix when an expand / aggregate button is pressed. FIG. 11 is a diagram showing screen transition of an inspection result matrix when an expand / aggregate button is pressed. FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an image processing apparatus. FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of a PLC (Programmable Logic Controller). FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of a setting device. FIG. 3 is a diagram illustrating an example of a data structure of a project file. FIG. 4 is a diagram illustrating an example of a data structure of an inspection result file. 6 is a flowchart illustrating an example of a processing flow in the setting device. It is a figure showing an example of the screen where the schematic diagram showing the design appearance of the work was displayed. It is a figure showing an example of the screen where the inspection object part was displayed. It is a figure showing an example of a point on an inspection object field. It is a figure showing an example of an inspection object position determined from an inspection object field, and an effective visual field corresponding to it. It is a figure showing all the inspection object parts determined from the inspection object field, and an effective visual field. FIG. 4 is a diagram illustrating an example of a method of determining a work shooting position by a setting device. FIG. 9 is a diagram illustrating an example of a shooting path determined for a shooting position. 5 is a flowchart illustrating an example of a processing flow in a PLC. 9 is a flowchart illustrating an example of a flow of an inspection process performed by the image processing apparatus. It is a flowchart which shows an example of the flow of a display process of an inspection result matrix. FIG. 14 is a diagram illustrating a specific example 1 of a process executed in response to a cell selection operation. FIG. 19 is a diagram illustrating a specific example 2 of a process performed in response to a cell selection operation. FIG. 21 is a diagram illustrating a specific example 3 of a process executed in response to a cell selection operation. FIG. 15 is a diagram illustrating a specific example 4 of a process executed in response to a cell selection operation. FIG. 21 is a diagram illustrating a specific example 5 of a process performed in response to a cell selection operation. FIG. 4 is a diagram illustrating a modification of the inspection result matrix illustrated in FIG. 3. FIG. 4 is a diagram illustrating an example of a data structure of a classification rule. FIG. 11 is a diagram showing screen transition of an inspection result matrix when an expand / aggregate button is pressed. FIG. 11 is a diagram showing screen transition of an inspection result matrix when an expand / aggregate button is pressed. It is a figure which shows roughly the comparison processing of an inspection result matrix and an expectation value matrix. It is a figure showing an example of a creation process of an expectation value matrix. It is a flow chart which shows an example of the flow of three-dimensional display processing of an inspection result. It is a figure which shows the specific example 1 of a process performed according to the selection operation of the part to be examined shown in a three-dimensional model. It is a figure which shows the example 2 of a process performed according to the selection operation of the inspection target part shown by a three-dimensional model. It is a figure which shows the process of expanding / consolidating the part to be inspected shown by a three-dimensional model. It is a figure showing an appearance inspection system concerning a modification. It is a figure showing another form which changes a relative position between a work and an imaging device. FIG. 11 is a diagram showing still another mode in which the relative position between the work and the imaging device is changed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are the same. Therefore, detailed description thereof will not be repeated.

<A. Application example>
First, an example of a scene to which the present invention is applied will be described with reference to FIGS. FIG. 1 is a schematic diagram showing an outline of a visual inspection system 1 according to the present embodiment.

Appearance inspection system 1 according to the present embodiment includes, for example, a plurality of inspection target portions on an inspection target (hereinafter, also referred to as “work W”) mounted on stage 90 in an industrial product production line or the like. Is imaged, and the appearance of the work W is inspected using the obtained image. In the appearance inspection, the work W is inspected for scratches, dirt, presence or absence of foreign matter, dimensions, and the like.

When the appearance inspection of the work W placed on the stage 90 is completed, the next work W is transported onto the stage 90. At this time, the work W is placed at a predetermined position on the stage 90 in a predetermined posture.

As shown in FIG. 1, the visual inspection system 1 includes an imaging device 10, an image processing device 20, a robot 30, a robot controller 40, a PLC 50, and a setting device 60.

The imaging device 10 is for imaging a subject existing in an imaging field of view and generating image data in accordance with a command from the image processing device 20, and captures a workpiece W to be subjected to a visual inspection as a subject.

The image processing device 20 outputs an imaging command to the imaging device 10 in accordance with the command from the PLC 50. The image processing device 20 includes an inspection unit 21, a display control unit 22, and a display device 23. The inspection unit 21 and the display control unit 22 are functional modules executed by the processor 110 (see FIG. 8) of the image processing device 20. The inspection unit 21 determines the appearance of the work W by performing a predetermined process on the image data generated by the imaging device 10. The display control unit 22 causes the display device 23 to display the determination result by the inspection unit 21. The output of the inspection result may be output to the display device 23 provided in the image processing device 20 or may be output to a display device (for example, a display 366 described later) provided in the setting device 60.

The robot 30 is, for example, a vertical articulated robot in which a plurality of arms 32 are connected on a base 31. Each connecting portion of the plurality of arms 32 includes a rotation shaft. The imaging device 10 is attached to the distal end of the distal arm 32a. The robot controller 40 controls the robot 30 according to a command from the PLC 50 to change the relative position between the work W and the imaging device 10 and the posture of the imaging device 10 with respect to the work W.

As described above, the work W is placed at a predetermined position on the stage 90 in a predetermined posture. Therefore, the robot 30 can change the relative position between the work W and the imaging device 10 and the posture of the imaging device 10 with respect to the work W by changing the relative position and the posture of the imaging device 10 with respect to the stage 90. . In other words, the robot 30 moves the imaging device 10 using a coordinate system having a point on the stage 90 as the origin, whereby the relative position between the work W and the imaging device 10 and the posture of the imaging device 10 with respect to the work W Can be changed.

The PLC 50 controls the robot controller 40 and the image processing device 20 so that the imaging device 10 sequentially captures a plurality of inspection target portions on the work W. The PLC 50 controls the robot controller 40 according to a path that satisfies the imaging conditions set by the setting device 60. Further, the PLC 50 controls the image processing device 20 so that the imaging device 10 outputs an imaging command at a timing that satisfies the designated imaging condition.

The setting device 60 sets a path that satisfies imaging conditions including a relative position between the work W and the imaging device 10 for sequentially imaging a plurality of inspection target portions on the work W. The setting device 60 sets a path that satisfies the imaging conditions suitable for the work W when a new product or a new type of work W needs to be visually inspected.

With reference to FIG. 2, a display mode of the inspection result by the visual inspection system 1 shown in FIG. 1 will be described. FIG. 2 is a diagram illustrating an example of an inspection result displayed on the display device 23 of the image processing device 20.

The inspection unit 21 of the image processing apparatus 20 performs a predetermined image process on each image obtained by photographing the workpiece W from a plurality of directions, thereby detecting a defect in each inspection target portion of the workpiece W. Check for presence. The inspection result is represented by, for example, one of “with defect” and “without defect”. The inspection process by the inspection unit 21 is executed each time the stage 90 transports the workpiece W to be inspected to a predetermined position. The inspection result by the inspection unit 21 is output to the display control unit 22 after being associated with the identification information of the work W and the inspection target portion of the work W. The display control unit 22 displays on the display device 23 an inspection result matrix 25 in which the inspection result of each inspection target portion of the work W is represented for each work W.

As shown in FIG. 2, on the horizontal axis of the inspection result matrix 25, the inspection target portions of the work are arranged by site name. Each part name may be registered in advance or may be set by the user. On the vertical axis of the inspection result matrix 25, identification information of inspected works is arranged. The work identification information is represented by, for example, a work serial number (hereinafter, also referred to as “work No.”) or a work name. In the example of FIG. 2, the work identification information is represented by work No.

{Circle around (2)} On the inspection result matrix 25, cells are arranged in association with each combination of the inspection target portion of the work and the work No. The display control unit 22 displays an inspection result of each inspection target portion of each work W on a corresponding cell. As described above, the inspection result of each inspection target portion of each work is represented two-dimensionally, so that the user can immediately grasp which portion of which work has a defect. The display of the inspection result in two dimensions becomes effective as the inspection result becomes enormous.

Typically, the display control unit 22 highlights an inspection result indicating a defect among the inspection results included in the inspection result matrix 25 in a display mode different from other inspection results. As an example, the cell of the inspection result indicating a defect may be represented by a specific color (for example, red) or may be represented by a blinking display. In the example of FIG. 2, cells indicating “with defect” are hatched, and cells indicating “no defect” are not hatched. Since the inspection result indicating the defect is highlighted in a display mode different from other inspection results, the user can easily determine the inspection result indicating the defect, and which part of which work has the defect. Can be grasped more easily.

In the above description, an example has been described in which the inspection target portions of the work are arranged on the horizontal axis of the inspection result matrix 25 and the work numbers are arranged on the vertical axis of the inspection result matrix 25. 25 may be arranged on the horizontal axis, and the inspection target site of the work may be arranged on the vertical axis of the inspection result matrix 25.

<B. Inspection result matrix development / aggregation function>
With reference to FIGS. 3 to 7, a description will be given of the function of developing / aggregating cells shown in the inspection result matrix 25. FIG. FIG. 3 is a diagram showing a modification of the inspection result matrix 25 shown in FIG. Each row or each column of the inspection result matrix 25 shown in FIG. 3 differs from the inspection result matrix 25 shown in FIG. 2 in that the rows or columns are grouped in units of rows or columns according to a predetermined classification rule.

In the example of FIG. 3, in the horizontal axis direction of the inspection result matrix 25, the work parts “A1” to “A5” are grouped as a group GV1, and the work parts “B1” to “B3” are grouped. Grouped as GV2. The grouping of the inspection target portion of the work is performed according to the classification rule 134A shown in FIG.

FIG. 4 is a diagram showing an example of the data structure of the classification rule 134A. As illustrated in FIG. 4, the inspection rule of the work W is hierarchically associated with the classification rule 134A. The correspondence between the parts may be registered in advance or may be arbitrarily set by the user. As an example, the inspection target part defined in the higher hierarchy has a relationship including the inspection target part defined in the lower hierarchy. In the example of FIG. 4, the inspection target portion “A” is included in the portions “A1” to “A5”. That is, the parts “A1” to “A5” are areas in the inspection target part “A”.

Referring to FIG. 3 again, in the vertical axis direction of inspection result matrix 25, work Nos. “001A” to “001E” are grouped as group GH1, and work Nos. “002A” to “002J” are grouped. Grouped as GH2. The grouping of the work numbers is performed according to the classification rule 134B shown in FIG.

FIG. 5 is a diagram showing an example of the data structure of the classification rule 134B. As shown in FIG. 5, each work to be inspected is hierarchically associated with the classification rule 134B. The hierarchical relationship of each work may be registered in advance or may be arbitrarily set by the user. As an example, each work is associated with a lot number of a production line. The lot number defined in the upper hierarchy has a relation including the work defined in the lower hierarchy. In the example of FIG. 5, the lot number “001” is included in the work numbers “001A” to “001E”. In other words, it indicates that the work numbers “001A” to “001E” were produced in the lot number “001”.

Referring again to FIG. 3, a development / aggregation button is assigned to each group of the inspection result matrix 25 in the vertical axis direction and the horizontal axis direction. In the example of FIG. 3, the expand / aggregate button BH1 is assigned to the group GH1. The group GH2 is assigned an expand / consolidate button BH2. The group GV1 is assigned an expand / consolidate button BV1. The GV2 is assigned an expand / consolidate button BV2.

FIG. 6 is a diagram showing screen transition of the inspection result matrix 25 when the deploy / aggregate button BV2 is pressed. The expansion / aggregation button BV2 alternately receives an aggregation instruction and an expansion instruction for cells in the group GV2 each time the button is pressed. That is, when the expansion / aggregation button BV2 is pressed while the cells of the group GV2 are expanded, the display control unit 22 of the image processing device 20 aggregates the cells of the group GV2 in a line. On the other hand, when the expansion / aggregation button BV2 is pressed in a state where the cells of the group GV2 are aggregated, the display control unit 22 returns the display of the cells of the group GV2 to the state before the aggregation.

If the inspection results to be consolidated include an inspection result indicating “defect” and an inspection result indicating “no defect”, the display control unit 22 changes the display of the inspection result “defect” to “no defect”. Is performed prior to the display of the inspection result of "." As an example, the inspection result within the broken line AR1 includes one inspection result indicating “defective” and two inspection results indicating “no defect”. By the aggregation processing, the three inspection results in the broken line AR1 are aggregated into one inspection result shown in the broken line AR2. At this time, since the inspection result before aggregation includes “defective”, the display control unit 22 expresses the inspection result after aggregation as “defective”.

As described above, when at least one inspection result indicating “defective” is included in the inspection result before aggregation, the display control unit 22 determines the inspection result after aggregation as “defect”. Yes ". On the other hand, when the inspection result before aggregation does not include any inspection result indicating “defective”, the display control unit 22 determines that the inspection result after aggregation is “No defect”. ".

FIG. 7 is a diagram showing screen transition of the inspection result matrix 25 when the deploy / aggregate button BH2 is pressed. The expansion / aggregation button BH2 alternately receives an aggregation instruction and an expansion instruction for cells in the group GH2 each time the button is pressed. That is, when the expansion / aggregation button BH2 is pressed while the cells of the group GH2 are expanded, the display control unit 22 aggregates the cells of the group GH2 into one row. On the other hand, when the expansion / aggregation button BH2 is pressed in a state where the cells of the group GH2 are aggregated, the display control unit 22 returns the display of the cells of the group GH2 to the state before the aggregation.

として As an example, the inspection results within the broken line AR3 include one inspection result indicating “defect” and nine inspection results indicating “no defect”. By the aggregation process, the ten inspection results in the broken line AR3 are aggregated into one inspection result shown in the broken line AR4. At this time, since the inspection result before aggregation includes “defective”, the display control unit 22 expresses the inspection result after aggregation as “defective”.

As described above, when the display control unit 22 of the image processing device 20 receives the aggregation instruction for the group of rows or the group of columns, the display control unit 22 displays the inspection result of the row group or the column group in one row. Or display them in a single row. In addition, when the display control unit 22 receives an instruction to expand the inspection results that are collectively displayed, the display control unit 22 returns the display of the inspection results that are collectively displayed to the state before the aggregation. Thus, even when the number of rows or columns of the inspection result matrix 25 is so large that it cannot be displayed on the display device 23, the user can efficiently check the inspection results by utilizing the aggregated display and the expanded display. be able to.

In the above-described example, an example has been described in which an aggregated cell is represented by two values, that is, “has a defect” and “no defect”, but the aggregated cell is necessarily represented by a binary value. There is no. As an example, the display control unit 22 may display the cells after aggregation more densely as the number of inspection results indicating “defective” is included in the cells to be aggregated.

<C. Hardware Configuration>
Next, hardware configurations of the image processing device 20, the PLC 50, and the setting device 60 shown in FIG. 1 will be described in order with reference to FIGS.

(C1. Hardware Configuration of Image Processing Device 20)
FIG. 8 is a schematic diagram illustrating an example of a hardware configuration of the image processing apparatus 20. Referring to FIG. 8, image processing device 20 typically has a structure according to a general-purpose computer architecture, and executes various programs such as a visual inspection process by executing a program installed in advance by a processor. Image processing is realized.

More specifically, the image processing apparatus 20 includes a processor 110 such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), a RAM (Random Access Memory) 112, a display controller 114, and a system controller 116. , An input / output (I / O) controller 118, a storage device 120 such as a hard disk, a camera interface 122, an input interface 124, a controller interface 126, a communication interface 128, and a memory card interface 130. These units are connected to each other so as to enable data communication with the system controller 116 as a center.

The processor 110 exchanges programs (codes) with the system controller 116 and executes them in a predetermined order, thereby realizing a target arithmetic processing.

The system controller 116 is connected to the processor 110, the RAM 112, the display controller 114, and the I / O controller 118 via buses, respectively, exchanges data with each unit, and performs processing of the entire image processing apparatus 20. Govern

The RAM 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory), and includes a program read from the storage device 120, a camera image (image data) acquired by the imaging device 10, The processing result for the camera image, the work data, and the like are stored.

The display controller 114 is connected to the display device 23, and outputs a signal for displaying various information to the display device 23 according to an internal command from the system controller 116.

The I / O controller 118 controls data exchange between a recording medium connected to the image processing apparatus 20 and an external device. More specifically, I / O controller 118 is connected to storage device 120, camera interface 122, input interface 124, controller interface 126, communication interface 128, and memory card interface 130.

The storage device 120 stores various data such as a project file 144 in addition to the image processing program 142 and the display program 143 executed by the processor 110. Details of the project file 144 will be described later. Typically, the storage device 120 may be a nonvolatile magnetic storage device such as a hard disk, a semiconductor storage device such as a flash memory, or a DVD-RAM (Digital Versatile Disk Random Access Memory). ) May be used.

The image processing program 142 and the display program 143 may be provided by being incorporated in a part of another program. In that case, the image processing program 142 itself and the display program 143 itself execute a predetermined process in cooperation with another program. That is, the image processing program 142 and the display program 143 may be incorporated in such another program. Alternatively, some or all of the functions provided by executing the image processing program 142 or the display program 143 may be implemented as a dedicated hardware circuit.

The camera interface 122 corresponds to an input unit that receives image data generated by photographing the work W, and mediates data transmission between the imaging device 10 and the processor 110. More specifically, the camera interface 122 can be connected to one or more imaging devices 10, and a shooting instruction is output from the processor 110 to the imaging device 10 via the camera interface 122. Thus, the imaging device 10 captures an image of a subject and outputs the generated image to the processor 110 via the camera interface 122.

The input interface 124 mediates data transmission between the processor 110 and input devices such as a keyboard 134, a mouse, a touch panel, and a dedicated console.

The controller interface 126 mediates data transmission between the PLC 50 and the processor 110. More specifically, the controller interface 126 transmits information on the state of the production line controlled by the PLC 50, information on the work W, and the like to the processor 110.

The communication interface 128 mediates data transmission between the processor 110 and another personal computer or server device (not shown). The communication interface 128 is typically made of Ethernet (registered trademark), USB (Universal Serial Bus), or the like.

The memory card interface 130 mediates data transmission between the processor 110 and the memory card 136 as a recording medium. The memory card 136 circulates with the image processing program 142 and the display program 143 executed by the image processing apparatus 20 stored therein, and the memory card interface 130 reads out these programs from the memory card 136. The memory card 136 may be a general-purpose semiconductor storage device such as SD (Secure Digital), a magnetic recording medium such as a flexible disk (Flexible Disk), or an optical recording medium such as a CD-ROM (Compact Disk Read Only Memory). Become. Alternatively, a program downloaded from a distribution server or the like may be installed in the image processing apparatus 20 via the communication interface 128.

(C2. Hardware configuration of PLC 50)
Next, a hardware configuration of the PLC 50 will be described with reference to FIG. FIG. 9 is a schematic diagram illustrating an example of a hardware configuration of the PLC 50.

The PLC 50 includes a chipset 212, a processor 214, a nonvolatile memory 216, a main memory 218, a system clock 220, a memory card interface 222, a communication interface 228, an internal bus controller 230, and a field bus controller 238. including. The chipset 212 and other components are respectively connected via various buses.

The processor 214 and the chipset 212 typically have a configuration according to a general-purpose computer architecture. That is, the processor 214 interprets and executes the instruction codes sequentially supplied from the chipset 212 according to the internal clock. The chipset 212 exchanges internal data with various connected components and generates an instruction code necessary for the processor 214. The system clock 220 generates a system clock having a predetermined cycle and outputs the generated system clock to the processor 214. The chipset 212 has a function of caching data and the like obtained as a result of execution of arithmetic processing by the processor 214.

The PLC 50 has a nonvolatile memory 216 and a main memory 218 as storage means. The non-volatile memory 216 stores an OS, a system program, a user program, log information, and the like in a non-volatile manner. The main memory 218 is a volatile storage area that holds various programs to be executed by the processor 214 and is also used as a work memory when executing various programs.

The PLC 50 has a communication interface 228, an internal bus controller 230, and a field bus controller 238 as communication means. These communication circuits transmit and receive data.

The communication interface 228 exchanges data with the image processing device 20 and the setting device 60. As an example, the PLC 50 outputs an imaging instruction to the image processing device 20 via the communication interface 228. Alternatively, the PLC 50 receives the result of the appearance inspection of the work W from the image processing device 20 via the communication interface 228.

(4) The internal bus controller 230 controls data exchange via the internal bus 226. More specifically, the internal bus controller 230 includes a DMA (Dynamic Memory Access) control circuit 232, an internal bus control circuit 234, and a buffer memory 236.

The memory card interface 222 connects the memory card 224 detachable to the PLC 50 and the processor 214.

The fieldbus controller 238 is a communication interface for connecting to a field network. The PLC 50 is connected to the robot controller 40 via the field bus controller 238. For the field network, for example, EtherCAT (registered trademark), EtherNet / IP (registered trademark), CompoNet (registered trademark), or the like is employed.

(C3. Hardware configuration of setting device 60)
Next, a hardware configuration of the setting device 60 will be described with reference to FIG. FIG. 10 is a schematic diagram illustrating an example of a hardware configuration of the setting device 60.

The setting device 60 includes a processor 362, a main memory 363, a storage device 364, a display 366, an input device 367, and a communication interface 368. These units are connected to each other via a bus 361 so as to be able to perform data communication.

The processor 362 executes programs (codes) including the setting program 365 installed in the storage device 364 in the main memory 363 and executes them in a predetermined order, thereby performing various operations. The main memory 363 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

The storage device 364 is an internal memory included in the setting device 60 and is a nonvolatile storage device, and stores various programs such as the setting program 365. Note that the storage device 364 may be a hard disk or a semiconductor storage device such as a flash memory.

The setting program 365 is a program showing a procedure for setting a change path of the imaging condition by the setting device 60. Various programs such as the setting program 365 need not be stored in the storage device 364, but may be stored in a server that can communicate with the setting device 60 or an external memory that can be directly connected to the setting device 60. For example, the program is distributed in a state where various programs executed by the setting device 60 and various parameters used in the various programs are stored in the external memory, and the setting device 60 reads the various programs and various parameters from the external memory. The external memory accumulates information of the program or the like by an electric, magnetic, optical, mechanical or chemical action so that the computer or other device or machine or the like can read the information of the recorded program or the like. Medium. Alternatively, programs and parameters downloaded from a server or the like communicably connected to the setting device 60 may be installed in the setting device 60.

The display 366 is, for example, a liquid crystal display. The input device 367 includes, for example, a mouse, a keyboard, a touch pad, and the like.

The communication interface 368 exchanges various data between the PLC 50 and the processor 362. Note that the communication interface 368 may exchange data between the server and the processor 362. Communication interface 368 includes hardware corresponding to a network for exchanging various data with PLC 50.

<D. Project file 144>
The project file 144 (see FIG. 8) will be described with reference to FIG. FIG. 11 is a diagram showing an example of the data structure of the project file 144.

(4) Various data relating to the inspection processing of the work W is associated with the project file 144, so that various data relating to the inspection processing of the work W are centrally managed.

As an example, the project file 144 includes an imaging condition file 146A, an image file group 146B, an inspection condition file 146C, an inspection result file 146D, a work information file 146E, a production information file 146F, and a classification rule file 146G. , And the expected value file 146H.

The imaging condition file 146A is data that defines imaging conditions for each inspection target portion of the work W. That is, by referring to the imaging condition file 146A, the image processing apparatus 20 can uniquely specify the imaging condition using the inspection target portion as a key. Alternatively, the imaging condition may be uniquely specified using a combination of the work No. and the inspection target portion as a key. The imaging conditions include, for example, the relative position between the work W and the imaging device 10, the position of illumination when the work W is imaged, and the like.

The image file group 146B is a data group obtained from the imaging device 10 by imaging the work W according to the defined imaging conditions. Each image file included in the image file group 146B is associated with a work number and an inspection target portion. That is, the image processing apparatus 20 can uniquely specify the image file used for the inspection using the combination of the work number and the inspection target portion as a key. Information on the combination of the work No. and the inspection target portion is specified, for example, in the file name of the image file or the header of the image file.

The inspection condition file 146C is data defining the inspection conditions for each inspection target portion of the work. That is, the image processing apparatus 20 can uniquely specify the inspection condition as the inspection target partial key by referring to the inspection condition file 146C. The inspection conditions include, for example, an inspection target portion in an image, an image processing flowchart to be executed, a measurement parameter read at the time of execution of the image processing, a threshold for determining the presence or absence of a defect, and the like.

The inspection result file 146D is data that defines measurement values, inspection results, and the like for each inspection target portion of each work. Details of the data structure of the inspection result file 146D will be described later.

The work information file 146E includes a three-dimensional model representing the shape of the work to be inspected, type information of the work to be inspected, and the like.

The production information file 146F is data for defining a lot number, a serial number, and the like of a work to be inspected.

The classification rule file 146G is data including at least one of the above-described classification rule 134A (see FIG. 4), the above-described classification rule 134B (see FIG. 5), and the below-described classification rule 134C (see FIG. 30). .

The expected value file 146H is data that specifies the expected value of the inspection result for each inspection target portion of each inspection target work. Details of the expected value file 146H will be described later.

<E. Inspection result file 146D>
The inspection result file 146D included in the project file 144 (see FIG. 11) will be described with reference to FIG.

FIG. 12 is a diagram showing an example of the data structure of the inspection result file 146D. The inspection result file 146D includes identification information of the work (for example, work No.), a part name indicating a part to be inspected of the work, a measurement value obtained as an output result of the inspection processing, and an inspection result indicating the presence or absence of a defect. Including.

(4) The image processing apparatus 20 can uniquely specify the measurement value and the inspection result by using the combination of the work number and the inspection target portion as a key by referring to the inspection result file 146D.

<F. Process flow in setting device>
FIG. 13 is a flowchart illustrating an example of a processing flow in the setting device 60. When the appearance inspection system 1 needs to inspect the appearance of a new product or a new type of work W, the setting device 60 performs processing according to, for example, a flowchart shown in FIG. A change path of the imaging condition suitable for is set. The three-dimensional design data indicating the design surface of the new or new type of workpiece W is stored in the storage device 364 of the setting device 60 in advance.

In the example shown in FIG. 13, first, in step S1, the processor 362 of the setting device 60 reads the three-dimensional design data from the storage device 364. Next, in step S2, the processor 362 displays a schematic diagram showing the design appearance of the work W indicated by the three-dimensional design data on the display 366 of the setting device 60, and specifies the inspection target area on the work W according to the user input. decide. At this time, on the assumption that the workpiece W is placed at a predetermined position on the stage 90 in a predetermined posture, the processor 362 changes the coordinate system of the three-dimensional design data to a point on the stage 90. Convert to the XYZ coordinate system as the origin. Therefore, the inspection target area is indicated by an XYZ coordinate system having a point on the stage 90 as an origin.

Next, in step S3, the processor 362 determines a plurality of inspection target positions from within the inspection target region so as to satisfy the inspection requirement corresponding to the inspection target region. That is, the processor 362 determines each of the plurality of inspection target positions determined in the inspection target region as an inspection target portion.

Next, in step S4, the processor 362 determines an imaging condition including a relative position between the workpiece W and the imaging device 10 at the time of inspection for each of the plurality of inspection target positions.

Next, in step S5, the processor 362 determines an imaging route so as to pass through the relative position between the workpiece W and the imaging device 10 determined in step S4.

The inspection target area determined in step S2, the inspection target position determined in step S3, the imaging condition determined in step S4, and the imaging path determined in step S5 are transmitted to the image processing device 20 and the PLC 50. .

In the above example, it is assumed that a plurality of inspection target positions are determined from the inspection target portion in step S3. However, a plurality of inspection target portions may be determined in step S2, and in step S3, at least one inspection target position may be determined from each of the plurality of inspection target portions. Also in this case, a plurality of inspection target positions are determined in step S3.

<G. Method for determining inspection target part>
An example of a method of determining the inspection target portion will be described with reference to FIGS. FIG. 14 is a diagram illustrating an example of a screen on which a schematic diagram illustrating a design appearance of the work W is displayed. FIG. 15 is a diagram illustrating an example of a screen on which an inspection target portion is displayed.

設定 As shown in FIG. 14, the setting device 60 causes the display 366 to display a screen 61a including the three-dimensional model ML representing the shape of the work W. The screen 61a includes a tool button 71 for rotating the three-dimensional model ML about a vertical direction and a tool button 72 for rotating the three-dimensional model ML about a horizontal direction. The user can appropriately rotate the three-dimensional model ML by operating the

tool buttons

71 and 72.

(4) The setting device 60 receives designation of a place to be inspected from the user. Specifically, the user uses the input device 367 to click a plurality of points on the three-dimensional model ML of the work W that the user wants to inspect. In the screen 61a shown in FIG. 14, a plurality of points clicked by the user are indicated by circles 74.

The setting device 60 cuts out an area including a plurality of circles 74 specified on the three-dimensional model ML as an inspection target area. Specifically, the setting device 60 obtains a range within a predetermined distance from the point on the surface of the work W corresponding to each of the designated circles 74 along the surface, and sets the union of the ranges to the inspection target area. Cut out as.

{Circle around (4)} The setting device 60 further adjusts the inspection target area so that the contour is a geometric figure such as a straight line or a circle. On the screen 61b of the display 366 shown in FIG. 15, the inspection target area 75 adjusted so that the contour is a straight line parallel to any one of the ridges of the workpiece W is shown.

設定 Furthermore, the setting device 60 receives a fine adjustment instruction of the inspection target area 75 from the user, and finely adjusts the inspection target area 75 according to the instruction. The screen 61b includes

tool buttons

76 and 77 for enlarging or reducing the inspection target area 75. The user uses the input device 367 to select one side of the contour of the inspection target area 75 and operates the

tool buttons

76 and 77 to input an instruction to enlarge or reduce the inspection target area 75. Alternatively, the user inputs an instruction to enlarge or reduce the inspection target area 75 by dragging a point 78 on one side constituting the contour of the inspection target area 75 using a mouse included in the input device 367. You may. Accordingly, the setting device 60 enlarges or reduces the inspection target area 75. Thus, the setting device 60 determines the inspection target area 75. In the example illustrated in FIG. 15, a region in which partial regions of four surfaces among the six surfaces of the rectangular parallelepiped work W are gathered is determined as the inspection target region 75.

<H. Method of determining inspection target position>
An example of a method of determining the inspection target position by the setting device 60 will be described with reference to FIGS. FIG. 16 is a diagram illustrating an example of a point on the inspection target area. FIG. 17 is a diagram illustrating an example of the inspection target position determined from the inspection target region and the corresponding effective visual field. FIG. 18 is a diagram illustrating all inspection target portions determined from the inspection target region and the effective visual field.

When the camera resolution is R (pix) and the required minimum defect size is D (mm), the field of view of the imaging device 10 that can satisfy the inspection requirement that a defect of the minimum defect size can be recognized. The FOV is set as the inspection target part. The diameter of the imaging field of view FOV is generally represented by a × D × R using a proportionality constant a.

The setting device 60 regards the inspection target area 75 as a set of points, and can investigate a three-dimensional shape near a point in the inspection target area 75 using the distribution of normal vectors. For example, based on the three-dimensional design data of the workpiece W, the setting device 60 obtains a distribution of normal vectors in a range within a distance L along a surface from a point in the inspection target area 75. The distance L is, for example, a value (= b × a × D × R) obtained by multiplying the diameter (= a × D × R) of the imaging field of view FOV by a proportional constant b.

では In the example shown in FIG. 16, the vicinity of the point P1 is flat. Therefore, all the normal vectors within the range of the distance L from the point P1 along the surface of the work W become the vector n2. Point P2 is located near the ridgeline where the two surfaces intersect. Therefore, the normal vector within the range of the distance L from the point P2 along the surface of the workpiece W includes the two vectors n2 and n4. Point P2 is located near the vertex where the three surfaces intersect. Therefore, the normal vectors within the range of the distance L from the point P3 along the surface of the workpiece W include three vectors n1, n2, and n4. Therefore, the setting device 60 determines whether the vicinity of the point is flat or a ridgeline exists near the point by the distribution of the normal vectors within the distance L along the surface from the point in the inspection target area 75. Or whether there is a ridgeline near the point.

As shown in FIG. 17, the setting device 60 selects one point selected at random from the inspection target area 75 as the inspection target position Bi. The setting device 60 obtains the effective field of view FOV2i of the imaging device 10 for the inspection target position Bi as the inspection target portion. The effective visual field FOV2i is a visual field that includes the inspection target position Bi and can be imaged and inspected by the imaging device 10 using one imaging condition. The setting device 60 determines the effective field of view FOV2i according to the three-dimensional shape near the inspection target position Bi. The maximum diameter that the effective visual field FOV2i can have is set to the diameter of the imaging visual field FOV (= a × D × R).

For example, the setting device 60 determines the effective visual field FOV2i so that the variation of the normal vector distribution in the effective visual field FOV2i falls within a predetermined range. When the vicinity of the inspection target position Bi is flat, the setting device 60 determines a range (that is, an imaging visual field FOV) within a distance a × D × R along the surface from the inspection target position Bi as the effective visual field FOV2i. On the other hand, when a ridgeline or a vertex exists near the inspection target position Bi, as shown in FIG. 17, the setting device 60 moves the inspection target position Bi along the surface within a distance a × D × R within one distance. The range excluding the range of the portion is defined as the effective field of view FOV2i. The excluded range is the range of the surface having normal vectors other than the normal vector showing the maximum distribution amount in the normal vector distribution.

Next, the setting device 60 removes the set of points belonging to the determined effective field of view FOV2i from the set of points belonging to the inspection target area 75, and selects one point selected at random from the set of remaining points. Is selected as the inspection target position B (i + 1). The setting device 60 also determines the effective visual field FOV2 (i + 1) for the selected inspection target position B (i + 1). The setting device 60 repeats this processing until the set of points belonging to the inspection target area 75 becomes 0. Thereby, as shown in FIG. 18, a plurality of inspection target positions Bi (i = 1, 2,...) Are determined from the inspection target region 75. All points in the inspection target area 75 are included in any one of the effective visual fields FOV2i of the plurality of inspection target positions Bi (i = 1, 2,...). As described above, the maximum possible diameter of the effective visual field FOV2i is set to the diameter of the imaging visual field FOV. Therefore, the inspection target position is determined so that the defect having the minimum defect size can be recognized in the entire inspection target region 75.

In the above description, the setting device 60 randomly extracts the inspection target position Bi from the inspection target region 75. However, the setting device 60 may extract the inspection target position Bi from the inspection target region 75 according to a predetermined geometric condition. Alternatively, the setting device 60 may extract the inspection target position Bi from the inspection target region 75 such that the plurality of extracted inspection target positions Bi are regularly aligned.

For example, when a defect is likely to occur near the ridge or the vertex in the work W, the inspection target position may be determined so as to satisfy the inspection requirement that the inspection near the ridge or the vertex is preferentially performed. For example, the inspection target position Bi may be preferentially extracted from a set of points in the inspection target area 75 where a ridgeline or a vertex exists within a predetermined distance.

<I. How to determine imaging conditions>
FIG. 19 is a diagram illustrating an example of a method of determining the photographing position Ci of the work W by the setting device 60. The setting device 60 determines the photographing position Ci on the normal line of the design appearance surface of the work W at the inspection target position Bi. More specifically, the photographing position Ci can capture an image of the effective field of view FOV2i corresponding to the inspection target position Bi, and is located at a position away from the inspection target position Bi by an optimal subject distance that is in focus with the inspection target position Bi. It is determined.

Next, the setting device 60 determines an imaging condition based on the determined imaging position Ci. Typically, the setting device 60 includes the inspection target position Bi in the field of view, and determines an optimal imaging condition for focusing on the inspection target position Bi.

The imaging conditions include, for example, an X coordinate, a Y coordinate, and a Z coordinate of the imaging device 10 in an XYZ coordinate system whose origin is a point on the stage 90, and θx, θy, and θz that specify the direction of the optical axis of the imaging device 10. Of the six parameters. θx is the angle between the line that projects the optical axis of the imaging device 10 on the XY plane and the X axis, and θy is the angle that the line that projects the optical axis of the imaging device 10 on the YZ plane and the Y axis. Here, θz is the angle between the line that projects the optical axis of the imaging device 10 on the ZX plane and the Z axis. The XYZ coordinates are parameters for specifying a relative position between the work W and the imaging device 10, and θx, θy, and θz are parameters for specifying the posture of the imaging device 10 with respect to the work W.

<J. How to determine the shooting route>
FIG. 20 is a diagram illustrating an example of the photographing route determined for the photographing position Ci.

The setting device 60 determines a photographing route so as to pass through the decided photographing positions C1 to C7. At this time, the setting device 60 determines a photographing route so as to satisfy a predetermined requirement. For example, when the predetermined requirement is a requirement that minimizes the travel time, the setting device 60 sets a route candidate that has the shortest travel time among route candidates that sequentially pass through a plurality of imaging positions as an imaging route. Set. For example, the setting device 60 may calculate an evaluation value for evaluating an item (for example, travel time) indicated by a predetermined requirement for each route candidate, and set an imaging route based on the calculated evaluation value. . Furthermore, even when a plurality of imaging position candidates exist for each of the plurality of inspection target positions, the setting device 60 selects one of the plurality of imaging position candidates as the imaging position based on the evaluation value. Good. In this way, a shooting path optimized to satisfy predetermined requirements is set.

FIG. 20 shows the determined photographing route PS. In the example of FIG. 20, the image sequentially passes through “shooting position C1” → “shooting position C4” → “shooting position C5” → “shooting position C6” → “shooting position C7” → “shooting position C3” → “shooting position C2”. The photographing path PS is determined so as to perform the operation.

<K. Control processing by PLC 50>
FIG. 21 is a flowchart illustrating an example of the flow of processing in the PLC 50. The processing illustrated in FIG. 21 is realized by the processor 214 of the PLC 50 executing a program. In other aspects, some or all of the processing may be performed by circuit elements or other hardware.

The PLC 50 controls the robot controller 40 so that the imaging device 10 passes through the determined imaging path PS (see FIG. 20), and executes imaging at the imaging position Ci (see FIG. 20) where the imaging device 10 is determined. The image processing device 20 is controlled so as to perform the operation.

First, in step S10, the processor 214 determines whether or not the workpiece W placed on the stage 90 has been set at a predetermined position. If the processor 214 determines that the work W has been installed at a predetermined position (YES in step S10), it switches the control to step S12. Otherwise (NO in step S10), processor 214 executes the process of step S10 again.

In step S12, the processor 214 sequentially outputs a command value to the robot controller 40 according to a preset photographing path PS. The robot controller 40 drives each axis of the robot 30 according to a command value from the PLC 50. Thereby, the imaging device 10 attached to the tip of the robot 30 sequentially moves along the imaging path PS.

In step S20, the processor 214 determines whether or not the imaging device 10 has reached a preset imaging position Ci (see FIG. 20). Whether or not the imaging device 10 has reached the preset imaging position Ci is determined, for example, based on a command value output to the robot controller 40 in step S12. When determining that the imaging device 10 has reached the preset imaging position Ci (YES in step S20), the processor 214 switches the control to step S22. Otherwise (NO in step S20), processor 214 switches the control to step S30.

In step S22, the processor 214 outputs a photographing instruction to the image processing device 20. Thereby, the image processing device 20 executes the photographing process.

In step S30, the processor 214 determines whether or not the imaging device 10 has reached the preset end point of the shooting path PS. Whether or not the imaging device 10 has reached the preset end point of the shooting path PS is determined, for example, based on a command value output to the robot controller 40 in step S12. If the processor 214 determines that the imaging device 10 has reached the preset end point of the shooting path PS (NO in step S30), the processor 214 returns the control to step S10. Otherwise (NO in step S30), processor 214 returns the control to step S12.

<L. Inspection Processing by Image Processing Device 20>
FIG. 22 is a flowchart illustrating an example of the flow of an inspection process performed by the image processing apparatus 20. The processing illustrated in FIG. 22 is realized by the processor 110 of the image processing device 20 executing the image processing program 142 (see FIG. 8). Note that part or all of the processing may be executed by a circuit element or other hardware.

In step S50, the processor 110 determines whether or not a photographing instruction has been received from the PLC 50. As described above with reference to FIG. 21, the PLC 50 outputs a shooting instruction to the image processing device 20 when the imaging device 10 reaches a preset shooting position Ci. When determining that the photographing instruction has been received from the PLC 50 (YES in step S50), the processor 110 switches the control to step S52. Otherwise (NO in step S50), processor 110 switches the control to step S60.

In step S52, the processor 110 outputs a shooting instruction to the imaging device 10 as the above-described inspection unit 21 (see FIG. 1). Thereby, the imaging device 10 executes the photographing process based on receiving the photographing instruction from the image processing device 20. Thereby, the image processing device 20 can cause the imaging device 10 to execute the imaging process at the predetermined imaging position Ci.

In step S <b> 54, the processor 110 performs the inspection processing by executing predetermined image processing on the image obtained from the imaging device 10 as the inspection unit 21 described above. The image processing to be executed is specified, for example, in the above-described inspection condition file 146C (see FIG. 11). As described above, the inspection condition file 146C is data that defines inspection conditions for each inspection target portion of the work W. That is, by referring to the inspection condition file 146C, the image processing apparatus 20 can uniquely specify the inspection condition using the inspection target portion as a key. The processor 110 performs image processing on an image obtained according to the inspection condition acquired from the inspection condition file 146C, and obtains an inspection result.

In step S56, the processor 110 writes the test result obtained in step S54 in the test result file 146D (see FIG. 12) as the test unit 21 described above. As an example, the processor 110 determines the identification information of the work W (for example, work No.), the name of the part indicating the inspection target portion of the work W, the measurement value obtained as an output result of the inspection processing, and the inspection indicating the presence or absence of a defect. After being associated with the result, the result is written into the inspection result file 146D.

In step S60, the processor 110 determines whether or not an instruction to end the inspection processing has been received. When determining that an instruction to end the inspection process has been received (YES in step S60), processor 110 ends the inspection process shown in FIG. Otherwise (NO in step S60), processor 110 returns the control to step S50.

In the example of FIG. 22, an example is described in which the inspection process is executed each time the imaging process of the work W is executed. However, the imaging process of the work W and the inspection process on the image are performed separately. May be done. That is, the image processing apparatus 20 may first accumulate images, and then inspect the accumulated images collectively.

<M. Display flow of inspection result matrix 25>
FIG. 23 is a flowchart illustrating an example of the flow of the above-described inspection result matrix 25 display process. The processing illustrated in FIG. 23 is realized by the processor 110 of the image processing device 20 executing the display program 143 (see FIG. 8). Note that part or all of the processing may be executed by a circuit element or other hardware.

In step S70, the processor 110 determines whether a display operation of the inspection result matrix 25 has been received. The display operation is performed on an operation unit provided in the image processing device 20. The operation unit includes, for example, a keyboard 134 (see FIG. 8), a mouse, a touch panel, and the like. When determining that the display operation of the inspection result matrix 25 has been received (YES in step S70), the processor 110 switches the control to step S72. Otherwise (NO in step S70), processor 110 executes the process of step S70 again.

In step S72, the processor 110 refers to the above-described inspection result file 146D (see FIG. 12) to determine the work No. defined in the inspection result file 146D and the inspection target part defined in the inspection result file 146D. To get.

In step S74, the processor 110 configures the vertical axis of the inspection result matrix 25 according to the work number acquired in step S72 as the display control unit 22 (see FIG. 1). Thus, the work numbers are arranged on the vertical axis of the inspection result matrix 25.

In step S76, the processor 110 configures the horizontal axis of the inspection result matrix 25 according to the inspection target portion of the workpiece acquired in step S74 as the display control unit 22 described above. As a result, the inspection target portions are arranged on the horizontal axis of the inspection result matrix 25.

In step S78, the processor 110, as the above-described display control unit 22, sets blank cells on the inspection result matrix 25 for each combination of the work No. acquired in step S72 and the inspection target portion of the work acquired in step S74. Deploy. Each cell is associated with a work number and an inspection target portion.

In step S80, the processor 110, as the display control unit 22, refers to the inspection result file 146D and reflects the inspection result in each cell of the inspection result matrix 25. More specifically, the processor 110 acquires an inspection result for each combination of the work No. and the inspection target portion, and reflects the acquired inspection result in a cell corresponding to each combination. Typically, the processor 110 reflects the inspection result on each cell in such a manner that the inspection result of “defective” and the inspection result of “no defect” can be distinguished. As an example, if the inspection result indicates “defective”, the processor 110 displays the corresponding cell in a specific color (for example, red). On the other hand, if the inspection result indicates “no defect”, the processor 110 displays the corresponding cell in another color (for example, white or green).

In step S82, the processor 110 determines whether any cell in the inspection result matrix 25 has been selected. The selection operation is performed on an operation unit provided in the image processing device 20. The operation unit includes, for example, a keyboard 134 (see FIG. 8), a mouse, a touch panel, and the like. When determining that any of the cells in the inspection result matrix 25 has been selected (YES in step S82), the processor 110 switches the control to step S84. Otherwise (NO in step S82), processor 110 switches the control to step S90.

(4) Since the inspection result matrix 25 is associated with the work No. and the inspection target portion, selecting each cell means specifying a combination of the work No. and the inspection target portion.

In step S84, the processor 110 executes a process according to the selected cell as the inspection unit 21 (see FIG. 1) or the display control unit 22. That is, the processor 110 executes a process according to the combination of the specified work No. and the inspection target portion. Details of the processing executed in response to the cell selection operation will be described later.

In step S90, the processor 110 determines whether or not an operation for closing the inspection result matrix 25 has been received. When determining that an operation of closing inspection result matrix 25 has been received (YES in step S90), processor 110 ends the process illustrated in FIG. Otherwise (NO in step S90), processor 110 returns the control to step S82.

<N. Function for selecting cells in inspection result matrix>
As shown in steps S82 and S84 in FIG. 24, the image processing device 20 executes a process according to the selected cell based on the selection of the cell in the inspection result matrix 25. Examples of the processing that can be executed include the processing of specific examples 1 to 5 described below. Hereinafter, these processes will be described in order.

Note that typically, the image processing device 20 executes at least one of the processes shown in the following specific examples 1 to 5 based on the reception of the cell selection operation. Alternatively, the image processing apparatus 20 may display the selection screen of the processing shown in the following specific examples 1 to 5 based on the reception of the cell selection operation and execute the processing selected on the selection screen. Good.

(N1. Specific example 1)
FIG. 24 is a diagram illustrating a specific example 1 of a process performed in response to a cell selection operation. In the example of FIG. 24, the cell CE1 in the inspection result matrix 25 is selected.

Based on the selection of the cell CE1, the display control unit 22 of the image processing device 20 displays, on the display device 23, the image used for the inspection of the inspection target portion corresponding to the cell CE1 and the imaging conditions of the image. indicate. Thereby, the user can visually check the image indicating the defect, and can confirm the validity of the imaging conditions. Further, the user can easily confirm which part of the work the inspection target part shown in the inspection result matrix 25 indicates.

{More specifically, the image processing apparatus 20 specifies a combination of the work No. and the inspection target portion associated with the selected cell CE1. Next, the image processing apparatus 20 acquires a corresponding image file from the above-described image file group 146B (see FIG. 11) using the combination of the specified work number and the inspection target portion as a key. Next, the image processing apparatus 20 acquires the corresponding imaging condition from the above-described imaging condition file 146A (see FIG. 11) using the specified inspection target portion as a key. The display control unit 22 displays the acquired image file and the imaging condition on the pop-up screen PU1 associated with the cell CE1. The displayed imaging condition is configured to be changeable to an arbitrary value.

In the example of FIG. 24, an example has been described in which both the captured image and the imaging condition are displayed according to the cell selection operation. However, the display control unit 22 determines the captured image according to the cell selection operation. Alternatively, either one of the imaging condition and the imaging condition may be displayed.

(N2. Specific example 2)
FIG. 25 is a diagram illustrating a specific example 2 of a process performed in response to a cell selection operation. In the example of FIG. 25, the cell group CE2 in the inspection result matrix 25 is selected.

The display control unit 22 of the image processing device 20 displays the three-dimensional model ML representing the shape of the work to be inspected on the display device 23 based on the selection of the cell group CE2, and corresponds to the cell group CE2. The inspection result of the inspection target portion is represented as a corresponding portion on the three-dimensional model ML. Thus, the user can easily confirm which part of the work has a defect.

More specifically, based on the selection of the cell group CE2, the display control unit 22 of the image processing apparatus 20 reads the three-dimensional model of the work to be inspected from the work information file 146E (see FIG. 11). ML is acquired, and the three-dimensional model ML is displayed on the pop-up screen PU2 associated with the cell group CE2.

(4) Next, the combination of the work No. associated with each selected cell and the inspection target portion is specified. Next, the image processing apparatus 20 acquires an inspection result corresponding to each combination of the specified work No. and the inspection target portion from the above-described inspection result file 146D (see FIG. 12). Next, the image processing device 20 displays the inspection result of each inspection target portion in a corresponding location of the three-dimensional model ML. As described above with reference to FIGS. 13 to 18, since the inspection target position Bi is set for the three-dimensional model ML, the relationship between the three-dimensional model ML and the inspection target position Bi is known. Therefore, the display control unit 22 of the image processing device 20 can reflect the inspection result of each inspection target portion on the three-dimensional model ML based on the known information.

In the example of FIG. 25, the inspection target portions A3 to A5 corresponding to the selected cell group CE2 are reflected on the three-dimensional model ML. At this time, the display control unit 22 displays, on the three-dimensional model ML, the inspection target portion indicating “having a defect” in a display mode different from “no defect”. The inspection target portion indicating "having a defect" is represented by a specific color (for example, red) on the three-dimensional model ML, and the inspection target portion indicating "no defect" is represented by another color on the three-dimensional model ML. (Eg, green). Alternatively, the inspection target portion indicating “defect” may be represented by blinking display.

{Preferably, the display control unit 22 further shows, on the three-dimensional model ML, an inspection target area 75A including the inspection target part A3 and an inspection target area 75B including the inspection target parts A4 and A5. Since the inspection target area is as described in FIG. 15, the description will not be repeated. The inspection target area 75B including the inspection target portion indicating “defective” is displayed in a different form from the inspection target region 75A not including the inspection target portion indicating “defective”. As an example, the inspection target area 75B may be represented by a specific color (for example, red) or may be represented by blinking display.

In addition, the display control unit 22 displays a tool button 71 for rotating the three-dimensional model ML around the vertical direction and a tool button 72 for rotating the three-dimensional model ML around the horizontal direction on the pop-up screen PU2. To display further. The user can appropriately rotate the three-dimensional model ML by operating the

tool buttons

71 and 72.

(N3. Specific example 3)
FIG. 26 is a diagram illustrating a specific example 3 of the process performed in response to the cell selection operation. In the example of FIG. 26, the cell group CE3 in the inspection result matrix 25 is selected.

The display control unit 22 of the image processing device 20 displays the statistical result of the inspection result corresponding to the cell group CE3 on the display device 23 based on the selection of the cell group CE3. Thereby, the user can easily analyze the inspection result of an arbitrary inspection target portion.

{More specifically, the image processing apparatus 20 specifies a combination of the work number and the inspection target portion associated with each cell of the selected cell group CE3. Next, the image processing apparatus 20 acquires the inspection result and the measurement value corresponding to each combination of the specified work No. and the inspection target portion from the above-described inspection result file 146D. Next, the image processing device 20 performs a predetermined statistical process on the acquired measurement value.

As an example, the image processing device 20 generates a frequency distribution (that is, a histogram) by executing a predetermined statistical process. The horizontal axis of the histogram represents the division of the measurement value, and the horizontal axis of the histogram represents the frequency of the measurement value included in each division. The display control unit 22 of the image processing device 20 displays the generated histogram on the pop-up screen PU3 associated with the cell group CE3.

Preferably, the frequency distribution of the measurement values is generated for each selected inspection target portion. In the example of FIG. 26, the frequency distribution is shown for each of the selected inspection target portions A2 and A3.

As another example, the image processing device 20 generates a measured value transition graph by executing a predetermined statistical process. The horizontal axis of the measured value transition graph represents the work number, and the vertical axis of the measured value transition graph represents the measured value. The display control unit 22 of the image processing device 20 displays the generated measurement value transition graph on the pop-up screen PU3 associated with the cell group CE3.

{Preferably, the measurement value transition graph is generated for each inspection target portion corresponding to the selected cell group CE3. In the example of FIG. 26, a measured value transition graph is shown for each of the selected inspection target portions A2 and A3.

In the example of FIG. 26, an example has been described in which two statistical results, a histogram and a measured value transition graph, are displayed in accordance with a cell group selection operation. However, the display control unit 22 selects the cell group. At least one statistical result may be displayed according to the operation.

(N4. Specific example 4)
FIG. 27 is a diagram illustrating a specific example 4 of the process performed in response to the cell selection operation. In the example of FIG. 27, the cell group CE4 in the inspection result matrix 25 is selected.

The inspection unit 21 of the image processing device 20 re-examines the inspection target portion based on the selection of the cell group CE4 and based on the image used for the inspection of the inspection target portion corresponding to the cell group CE4. . Thus, the user can easily execute the re-inspection for an arbitrary inspection target portion.

More specifically, the user resets the inspection conditions, and then selects the cell group CE4 in the inspection result matrix 25. The inspection unit 21 specifies a combination of the work No. associated with each cell of the selected cell group CE4 and the inspection target portion. Next, the inspection unit 21 acquires image data corresponding to each combination of the specified work No. and the inspection target portion from the image file group 146B. Next, the inspection unit 21 acquires the inspection conditions corresponding to each of the specified inspection target portions from the above-described inspection condition file 146C. Next, the inspection unit 21 performs image processing on each of the acquired image data according to the corresponding inspection conditions. Next, the display control unit 22 reflects the result of the reinspection on the inspection result matrix 25.

In the example of FIG. 27, an example has been described in which a plurality of cells are selected in the inspection result matrix 25. However, in this example, the number of selected cells may be one or more.

(N5. Specific example 5)
FIG. 28 is a diagram illustrating a specific example 5 of the process performed in response to the cell selection operation. In the example of FIG. 28, the cell group CE5 in the inspection result matrix 25 is selected.

The display control unit 22 of the image processing device 20 displays the three-dimensional model ML of the work to be inspected on the display device 23 based on the selection of the cell group CE5, and also displays the inspection target portion corresponding to the cell group CE5. Are represented by corresponding parts on the three-dimensional model ML. Thereby, the user can easily confirm the imaging condition of the selected portion.

More specifically, based on the selection of the cell group CE5, the display control unit 22 of the image processing apparatus 20 stores the three-dimensional model ML of the work to be inspected in the work information file 146E described above (see FIG. 11). And displays the three-dimensional model ML on the pop-up screen PU5 associated with the cell group CE5.

(4) Next, the combination of the work No. associated with each selected cell and the inspection target portion is specified. Next, the image processing apparatus 20 acquires the imaging condition corresponding to each combination of the specified work No. and the inspection target portion from the above-described imaging condition file 146A (see FIG. 11).

Next, the display control unit 22 of the image processing apparatus 20 determines any one of the work numbers (for example, the minimum work number) from the work numbers corresponding to the cell group CE5, and corresponds to the determined work number. The respective imaging conditions to be performed are represented at corresponding locations of the three-dimensional model ML. The pop-up screen PU5 displays a return button 81 and a forward button 82. By pressing the return button 81 or the forward button 82, the user can switch the imaging conditions to be displayed in the order of work No. it can.

The displayed imaging conditions include the imaging position of the imaging device 10 with respect to the three-dimensional model ML. As described above with reference to FIG. 19, the shooting position Ci of the imaging device 10 with respect to the three-dimensional model ML is determined by the setting process performed by the setting device 60. Is known. Therefore, the display control unit 22 of the image processing device 20 can display a schematic diagram representing the imaging device 10 on the imaging position Ci based on the known information. In the example of FIG. 28, a schematic diagram representing the imaging device 10 at the imaging positions C5 and C7 is displayed.

として As another example, the displayed imaging condition includes a portion to be inspected on the workpiece. As described above with reference to FIGS. 13 to 18, since the inspection target position Bi is set for the three-dimensional model ML, the relationship between the three-dimensional model ML and the inspection target position Bi is known. Therefore, the display control unit 22 of the image processing device 20 can reflect the inspection result of each inspection target portion on the three-dimensional model ML based on the known information. In the example of FIG. 28, the inspection target portions A5 and B1 are represented on the three-dimensional model ML.

In the above example, an example in which the imaging position and the inspection target portion are displayed as the imaging conditions has been described, but the displayed imaging conditions are not limited to the imaging position and the inspection target portion. For example, an optical condition (such as an imaging field of view) of the imaging device 10 at the time of imaging and an illumination condition at the time of imaging may be displayed.

<O. Modified Example of Inspection Result Matrix 25>
FIG. 29 is a diagram showing a modification of the inspection result matrix 25 shown in FIG. Each row of the inspection result matrix 25 shown in FIG. 3 is grouped by the lot number of the work, whereas each row of the inspection result matrix 25 shown in FIG. 29 is grouped by the type of the work.

(4) There is a possibility that a common inspection target portion exists between works of different types, or a specific inspection target portion exists. Therefore, when the inspection result of each work of a different kind is represented on one inspection result matrix 25, a device is required.

として As an example, it is assumed that the work type α has unique inspection target portions “A1” and “A3” which are not included in the work type β. In this case, as shown in FIG. 29, the inspection results are reflected in the cells corresponding to the inspection target portions “A1” and “A3” of the work type α, but the inspection target portions “A1” and “A1” For the cell corresponding to “A3”, a blank is displayed as “not applicable”.

(4) Assume that the work type β has unique inspection target portions “A5” and “B3” that are not included in the work type α. In this case, as shown in FIG. 29, the inspection result is reflected in the cells corresponding to the inspection target portions “A5” and “B3” of the work type β, but the inspection target portions “A5” and “A5” of the work type α are reflected. For the cell corresponding to “B3”, a blank is displayed as “not applicable”.

ワーク Further, it is assumed that the work types α and β have common inspection target portions “A2”, “A4”, “B1”, and “B2”. In this case, as shown in FIG. 29, the cells corresponding to the inspection target parts "A2", "A4", "B1", and "B2" of the work type α, and the inspection target parts "A2", The inspection result is reflected in both the cells corresponding to “A4”, “B1”, and “B2”.

検査 The inspection target portion for each type of work is specified, for example, in a classification rule 134C shown in FIG. FIG. 30 is a diagram illustrating an example of the data structure of the classification rule 134C.

As shown in FIG. 30, the inspection rule of the work W is hierarchically associated with the classification rule 134C. In the example of FIG. 30, “A1” to “A4” are associated with the inspection target portion “A” of the work type α. “B1” and “B2” are associated with the inspection target portion “B” of the work type α. “A2”, “A4”, and “A5” are associated with the inspection target portion “A” of the work type β. “B1” to “B3” are associated with the inspection target portion “B” of the work type β.

Referring again to FIG. 29, buttons are assigned to each group of the inspection result matrix 25 in the vertical axis direction and the horizontal axis direction. In the example of FIG. 29, an expand / consolidate button BH1 is assigned to the group GH1. The group GH2 is assigned an expand / consolidate button BH2. The group GV1 is assigned an expand / consolidate button BV1. The group GV2 is assigned an expand / consolidate button BV2.

FIG. 31 is a diagram showing a screen transition of the inspection result matrix 25 when the development / aggregation button BV2 is pressed. The expansion / aggregation button BV2 alternately receives an aggregation instruction and an expansion instruction for cells in the group GV2 each time the button is pressed. That is, when the expansion / aggregation button BV2 is pressed while the cells of the group GV2 are expanded, the display control unit 22 of the image processing device 20 aggregates the cells of the group GV2 in a line. On the other hand, when the expansion / aggregation button BV2 is pressed in a state where the cells of the group GV2 are aggregated, the display control unit 22 returns the display of the cells of the group GV2 to the state before the aggregation.

When the inspection results of “defective”, “no defect”, and “not applicable” are included in the cells to be consolidated, the display control unit 22 displays “defect existing” as “no defect”, “applicable”. Aggregation processing is performed with priority over the display of "None". When the inspection results of “no defect” and “not applicable” are included in the cell to be consolidated, the display control unit 22 gives priority to the display of “no defect” over the display of “not applicable”. To perform aggregation processing.

として As an example, the broken line AR10 includes one “defective” cell and two “not applicable” cells. By the aggregation process, the three cells within the dashed line AR10 are aggregated into one cell indicated within the dashed line AR12. At this time, since the cells before aggregation include cells having “defect”, the display control unit 22 expresses the inspection result after aggregation as “defective”.

として As another example, the broken line AR11 includes two “no defect” cells and one “non-corresponding” cell. By the aggregation process, the three cells within the dashed line AR11 are aggregated into one cell indicated within the dashed line AR13. At this time, since the cells before aggregation do not include a cell having “defect” and include a cell having “no defect”, the display control unit 22 compares the inspection result after aggregation with “defect”. "None".

FIG. 32 is a diagram showing screen transition of the inspection result matrix 25 when the deploy / aggregate button BH2 is pressed. The expansion / aggregation button BH2 alternately receives an aggregation instruction and an expansion instruction for cells in the group GH2 each time the button is pressed. That is, when the expansion / aggregation button BH2 is pressed while the cells of the group GH2 are expanded, the display control unit 22 aggregates the cells of the group GH2 into one row. On the other hand, when the expansion / aggregation button BH2 is pressed in a state where the cells of the group GH2 are aggregated, the display control unit 22 returns the display of the cells of the group GH2 to the state before the aggregation.

The broken line AR14 includes ten cells of “not applicable”. By the aggregation processing of the cells of the group GH2, the ten cells in the broken line AR14 are aggregated into one cell shown in the broken line AR15. At this time, since the cells before aggregation do not include the cells with “defect” and “without defect”, the display control unit 22 expresses the inspection result after aggregation as “not applicable”.

<P. Comparison function between inspection result matrix and expected value matrix>
The image processing apparatus 20 displays a comparison result between the expected value matrix indicating a true correct value (hereinafter, also referred to as an “expected value”) for each inspection result included in the inspection result matrix 25 and the inspection result matrix 25. It is displayed on the device 23. Thus, the user can easily determine whether each test result shown in the test result matrix 25 is as expected. Such comparison processing becomes more effective as the number of test results shown in the test result matrix 25 increases.

FIG. 33 is a diagram schematically showing a comparison process between the inspection result matrix 25 and the expected value matrix 27.

In the cells in the expected value matrix 27, expected values are defined for at least a part of the inspection results included in the inspection result matrix 25. The expected value of each cell of the expected value matrix 27 is set, for example, by a user input. The expected value that can be input is, for example, one of “defective”, “no defect”, and “invalid”. The expected value matrix 27 set by the user is stored in the image processing apparatus 20 as the above-described expected value file 146H (see FIG. 11).

(4) The image processing apparatus 20 compares cells in the same row and the same column between each cell in the inspection result matrix 25 and each cell in the expected value matrix 27.

More specifically, if the inspection result of the inspection result matrix 25 is “no defect” and the expected value of the expected value matrix 27 is “no defect”, “correct answer OK” is output as the comparison result. That is, "correct answer OK" means that the inspection result is as expected.

If the inspection result of the inspection result matrix 25 is “defective” and the expected value of the expected value matrix 27 is “defective”, “correct answer NG” is output as the comparison result. That is, “correct answer NG” means that the inspection result is as expected.

If the inspection result of the inspection result matrix 25 is “no defect” and the expected value of the expected value matrix 27 is “defective”, “missing” is output as the comparison result. “Missed” means that a defect to be detected has been missed. That is, "missing" indicates that the inspection result is not as expected. One reason that the comparison result is “missed” is that the threshold value for determining whether or not there is a defect is too loose.

If the inspection result of the inspection result matrix 25 is “defective” and the expected value of the expected value matrix 27 is “no defect”, “overdetected” is output as the comparison result. "Overdetection" means that a normal part has been detected as a defect. That is, “overdetected” indicates that the inspection result is not as expected. One reason that the comparison result is “overdetected” is that the threshold value for determining the presence / absence of a defect is too strict.

If at least one of the inspection result of the inspection result matrix 25 and the expected value of the expected value matrix 27 is “invalid”, “invalid” is output as the comparison result. “Invalid” means that at least one of the test result and the expected value does not exist.

A comparison result matrix 29 is output as a comparison result between the inspection result matrix 25 and the expected value matrix 27. The display control unit 22 displays the comparison result “correct answer OK”, “correct answer NG”, “missed”, “overdetected”, and “invalid” in a distinguishable manner. As an example, these comparison results may be distinguished by color or hatching type.

The “correct answer OK” and “correct answer NG” that are the expected comparison results may be displayed in the same color (for example, white).

Also, as with the inspection result matrix 25, the cell aggregation / expansion function may be implemented for the comparison result matrix 29. As an example, when at least one “missing” or “overdetection” is included in the cell to be aggregated, the display control unit 22 sets the cell after aggregation to “missing” or “overdetection”. When both the “missing” and “overdetection” are included in the cell to be aggregated, the display control unit 22 sets the cell to be aggregated in a display mode indicating that both “missing” and “overdetection” are included. Represents

<Q. Expectation matrix creation support function>
FIG. 34 is a diagram illustrating an example of a process of creating the expected value matrix 27. The image processing device 20 has a function of supporting creation of the expected value matrix 27.

More specifically, the user selects each cell of the inspection result matrix 25. The cell selection operation is performed on an operation unit provided in the image processing device 20. The operation unit includes, for example, a keyboard 134 (see FIG. 8), a mouse, a touch panel, and the like.

とする As an example, suppose that the cell group CE10 is selected by the user. Thereafter, the user copies the cell group CE10 of the inspection result matrix 25 to the expected value matrix 27 being edited. This eliminates the need for the user to input each cell of the expected value matrix 27 one by one, and reduces the time and effort of creating the expected value matrix 27.

The expected value matrix 27 created by the user is stored in the image processing apparatus 20 as the above-described expected value file 146H (see FIG. 11).

<R. Flow of 3D display of inspection results>
The image processing device 20 has a function of displaying the inspection result in three dimensions by reflecting the inspection result of the work on the three-dimensional model ML. The user can easily determine the location where the defect has occurred by checking the inspection result on the three-dimensional model ML.

Hereinafter, the three-dimensional display processing of the inspection result will be described with reference to FIG. FIG. 35 is a flowchart illustrating an example of the flow of the three-dimensional display processing of the inspection result. The processing illustrated in FIG. 35 is realized by the processor 110 of the image processing device 20 executing the display program 143 (see FIG. 8). Note that part or all of the processing may be executed by a circuit element or other hardware.

In step S90, the processor 110 determines whether or not an execution operation of the three-dimensional display processing of the inspection result has been received. The display operation is performed on an operation unit provided in the image processing device 20. The operation unit includes, for example, a keyboard 134 (see FIG. 8), a mouse, a touch panel, and the like. When determining that the execution operation of the three-dimensional display processing of the inspection result has been received (YES in step S90), processor 110 switches the control to step S92. Otherwise (NO in step S90), processor 110 executes the process of step S90 again.

In step S92, the processor 110 acquires the three-dimensional model ML of the work from the work information file 146E (see FIG. 11) as the display control unit 22 (see FIG. 1), and outputs the acquired three-dimensional model ML. It is displayed on the display device 23.

In step S100, the processor 110 determines whether an instruction to update the three-dimensional display of the inspection result has been received. The update instruction is issued, for example, based on a new inspection result obtained from the inspection unit 21. Alternatively, the update instruction is issued based on the fact that the inspection unit 21 has detected the inspection result indicating the defect. Alternatively, the update instruction is issued based on a user operation. When determining that the instruction to update the three-dimensional display of the inspection result has been received (YES in step S100), processor 110 switches the control to step S102. Otherwise (NO in step S100), processor 110 switches the control to step S120.

In step S102, the processor 110 refers to the above-described inspection result file 146D (see FIG. 12) to acquire inspection results for each inspection target portion of the work No. of the inspection target work.

In step S104, the processor 110, as the above-described display control unit 22, displays each inspection result acquired in step S102 in a corresponding location on the three-dimensional model ML displayed in step S92. As described above with reference to FIGS. 13 to 18, since the inspection target position Bi is set for the three-dimensional model ML, the relationship between the three-dimensional model ML and the inspection target position Bi is known. Therefore, the display control unit 22 of the image processing device 20 can reflect each inspection result acquired in step S102 on the three-dimensional model ML based on the known information.

Typically, when the inspection result of each inspection target portion is expressed as a corresponding portion on the three-dimensional model ML, the processor 110 displays a portion indicating a defect in a display mode different from other portions. As an example, a portion indicating a defect may be represented by a specific color (for example, red) or may be represented by a blinking display. This makes it easier for the user to determine the portion indicating the defect.

In step S110, the processor 110 determines whether any of the inspection target portions shown on the three-dimensional model has been selected. The selection operation is performed on an operation unit provided in the image processing device 20. The operation unit includes, for example, a keyboard 134 (see FIG. 8), a mouse, a touch panel, and the like. If the processor 110 determines that any of the inspection target portions represented on the three-dimensional model has been selected (YES in step S110), it switches the control to step S112. Otherwise (NO in step S110), processor 110 switches the control to step S120.

In step S112, the processor 110 executes a process corresponding to the inspection target portion selected in step S110 as the inspection unit 21 or the display control unit 22 described above. Details of the processing executed in accordance with the selected inspection target portion will be described later.

In step S120, the processor 110 determines whether or not an operation to close the three-dimensional display screen of the inspection result has been received. When determining that an operation of closing the three-dimensional display screen of the inspection result has been received (YES in step S120), processor 110 ends the process illustrated in FIG. Otherwise (NO in step S120), processor 110 returns the control to step S100.

As described above, the three-dimensional display of the inspection result is updated when the instruction to update the three-dimensional display of the inspection result is issued in step S100. As described above, the update instruction is issued, for example, based on the inspection result indicating the defect being detected by the inspection unit 21. That is, when a defect is detected in the work to be sequentially inspected, the processor 110 updates the inspection result represented by the three-dimensional model ML with the inspection result of each inspection target portion of the work. As a result, while no inspection result indicating a defect is detected, the inspection result displayed on the three-dimensional model ML is not updated. Thus, the latest inspection result indicating the defect is always displayed on the three-dimensional model ML. Therefore, it is difficult for the user to overlook the inspection result indicating the defect.

<S. Selection function of inspection target part for 3D model>
As shown in steps S110 and S112 in FIG. 35, the image processing apparatus 20 responds to the selected inspection target portion based on selection of any of the inspection target portions shown in the three-dimensional model ML. Execute the process. Examples of the processing that can be executed include the processing of specific examples 1 and 2 described below. Hereinafter, these processes will be described in order.

Note that, typically, the image processing device 20 executes at least one of the processes shown in the following specific examples 1 and 2 based on receiving an operation of selecting an inspection target portion on the three-dimensional model ML. . Alternatively, the image processing device 20 displays the selection screen of the processing shown in the following specific examples 1 and 2 based on the selection operation of the inspection target portion on the three-dimensional model ML, and makes a selection on the selection screen. The performed processing may be executed.

(S1. Specific example 1)
FIG. 36 is a diagram illustrating a specific example 1 of a process performed in response to a selection operation of an inspection target portion illustrated in the three-dimensional model ML.

As shown in FIG. 36, the display device 23 includes a three-dimensional model ML of the work to be inspected, a tool button 71 for rotating the three-dimensional model ML about the vertical direction, and a three-dimensional model ML about the horizontal direction. A tool button 72 for rotating the dimensional model ML is displayed. The user can appropriately rotate the three-dimensional model ML by operating the

tool buttons

71 and 72.

As described with reference to FIGS. 13 to 18, the inspection target position Bi is set with respect to the three-dimensional model ML, and thus the relationship between the three-dimensional model ML and the inspection target position Bi is known. Therefore, the display control unit 22 of the image processing device 20 can represent the inspection target portion on the three-dimensional model ML based on the known information. In the example of FIG. 36, the inspection target portions A1 to A4 are represented in the three-dimensional model ML.

{Circle around (2)} The display control unit 22 indicates the inspection result in the inspection target portions A1 to A4 on the three-dimensional model ML. As an example, the portion indicating “having a defect” may be represented by a specific color (for example, red) or may be represented by blinking display. The portion indicating “no defect” is represented by a different color (for example, green) from the portion indicating “defective”.

In the example of FIG. 36, the inspection target portion A1 indicates “defect”, and the inspection target portions A2 to A4 indicate “no defect”. Preferably, the display control unit 22 highlights a defective portion A1_1 indicating a defect in the inspection target portion A1 more than other portions.

The user can select any of the inspection target portions A1 to A4. In the example of FIG. 36, the inspection target portion A1 is selected. The image processing device 20 displays, on the display device 23, the image used for the inspection of the inspection target portion A1 and the imaging conditions of the image based on the selection of the inspection target portion A1. Thus, the user can visually check the image indicating the defect, and can confirm the validity of the imaging conditions.

More specifically, based on the fact that the inspection target portion A1 is selected, the image processing apparatus 20 uses the combination of the work No. of the inspection target work and the selected inspection target portion A1 as a key to generate a corresponding image file. From the image file group 146B (see FIG. 11). Next, the image processing apparatus 20 acquires the corresponding imaging condition from the above-described imaging condition file 146A (see FIG. 11) using the inspection target portion A1 as a key. The display control unit 22 displays the acquired image file and the imaging condition on the pop-up screen PU6 associated with the selected inspection target portion A1. The displayed imaging condition is configured to be changeable to an arbitrary value.

In the example of FIG. 36, an example has been described in which both the captured image and the imaging condition are displayed according to the operation of selecting the inspection target portion. However, the display control unit 22 performs Accordingly, either one of the captured image and the imaging condition may be displayed.

(S2. Specific example 2)
FIG. 37 is a diagram illustrating a specific example 2 of a process performed in response to an operation of selecting an inspection target portion illustrated in the three-dimensional model ML.

The three-dimensional model ML includes inspection target portions A1 to A4. The user can select any of the inspection target portions A1 to A4. In the example of FIG. 37, the inspection target portion A1 is selected. Based on the fact that the inspection target portion A1 is selected, the image processing device 20 uses the combination of the work No. of the inspection target work and the selected inspection target portion A1 as a key to store corresponding imaging conditions in the above-described imaging condition file. 146A (see FIG. 11). Next, the display control unit 22 of the image processing device 20 displays the acquired imaging conditions in corresponding locations of the three-dimensional model ML.

As an example, the displayed imaging conditions include the imaging position of the imaging device 10 with respect to the three-dimensional model ML. As described above with reference to FIG. 19, the shooting position Ci of the imaging device 10 with respect to the three-dimensional model ML is determined by the setting process performed by the setting device 60. Is known. Therefore, the display control unit 22 of the image processing device 20 can display the schematic diagram 10A representing the imaging device 10 on the photographing position Ci based on the known information. In the example of FIG. 37, a schematic diagram 10A of the imaging device 10 is displayed at the shooting position C1.

In the above example, an example in which the imaging position is displayed as the imaging condition has been described, but the displayed imaging condition is not limited to the imaging position. For example, an optical condition of the imaging device 10 at the time of imaging, an illumination condition at the time of imaging, and the like may be displayed.

<T. Deployment / aggregation function of inspection target part>
FIG. 38 is a diagram showing a process of developing / aggregating the inspection target portion shown in the three-dimensional model ML.

The inspection target portion shown in the three-dimensional model ML is hierarchically associated with the above-described classification rule 134A (see FIG. 4). The inspection target part defined in the higher hierarchy has a relationship including the inspection target part defined in the lower hierarchy. The lower inspection target parts included in the inspection target part defined in the upper hierarchy are regarded as the same group.

例 In the example of FIG. 38, inspection target portions A to C are shown on the three-dimensional model ML. As an example, it is assumed that lower inspection target portions A1 to A3 are associated with upper inspection target portion A. The inspection target portions A1 to A3 included in the upper inspection target portion A are regarded as the same group.

とする It is assumed that the lower inspection target portions A1_1 to A1_4 are associated with the upper inspection target portion A1. The inspection target portions A1_1 to A1_4 included in the upper inspection target portion A1 are regarded as the same group.

Inspection results are shown in inspection target portions A to C on the three-dimensional model ML. The inspection results of the inspection target parts A to C are obtained from the inspection result file 146D (see FIG. 12). As an example, the inspection target portion A indicates “with defect”, and the inspection target portions B and C indicate “without defect”.

(4) The development / aggregation button BT1 is assigned to the inspection target portion A. The development / aggregation button BT2 is assigned to the inspection target portion B. The development / aggregation button BT3 is assigned to the inspection target portion C. “+” Indicates a development button, and “−” indicates an aggregation button.

When the display control unit 22 receives the aggregation instruction for the grouped inspection target portions, the display control unit 22 aggregates the inspection results of the grouped inspection target portions, and displays the inspection results after the aggregation in the grouping. Each part on the three-dimensional model ML corresponding to the part to be inspected is represented. In addition, when the display control unit 22 receives a deployment instruction for the aggregated inspection results, the display control unit 22 returns the display of the aggregated inspection results to the state before the aggregation.

For example, when the deploy / aggregate button BT1 is pressed, the display control unit 22 deploys the inspection target part A to lower inspection target parts A1 to A3. Next, the display control unit 22 reflects the inspection results of the inspection target portions A1 to A3 on the inspection target portions A1 to A3. The inspection results of the inspection target parts A1 to A3 are obtained from the inspection result file 146D (see FIG. 12). As an example, the inspection target portion A1 indicates "having a defect", and the inspection target portions A2 and A3 indicate "no defect".

(4) Next, the display control unit 22 assigns the deploy / aggregate button BT1_1 to the inspection target portion A1. Similarly, the display control unit 22 assigns a development / aggregation button BT1_2 to the inspection target portion A2. Similarly, the display control unit 22 assigns the deploy / aggregate button BT1_3 to the inspection target portion A3.

When the “+” of the expand / aggregate button BT1_1 is pressed, the display control unit 22 expands the inspection target portion A1 into lower inspection target portions A1_1 to A1_4. Next, the display control unit 22 reflects the inspection results of the inspection target portions A1_1 to A1_4 on the inspection target portions A1_1 to A1_4. The inspection results of the inspection target portions A1_1 to A1_4 are obtained from the inspection result file 146D (see FIG. 12). As an example, the inspection target portions A1_1 to A1_3 indicate "no defect", and the inspection target portion A1_4 indicates "defect".

(4) When the expand / combine button BT1_1A is pressed, the display control unit 22 combines the inspection target parts A1_1 to A1_4 into a higher-level inspection target part A1. At this time, when at least one inspection result indicating “having a defect” is included in the inspection result of the inspection target portion to be consolidated, the display control unit 22 displays the inspection result of the inspection target portion after the aggregation as “defective”. ". On the other hand, when no inspection result indicating “having a defect” is included in the inspection result of the inspection target portion to be consolidated, the display control unit 22 displays the inspection result of the inspection target portion after the aggregation as “no defect”. ". Since the inspection target portion A1_4 indicates “having a defect”, the display control unit 22 sets the inspection result of the inspection target portion A1 obtained by integrating the inspection target portions A1_1 to A1_4 to “having a defect”.

When the “-” of the expand / combine button BT1 is pressed, the display control unit 22 combines the inspection target parts A1 to A3 into the inspection target part A. Since the inspection target portion A1 indicates "having a defect", the display control unit 22 sets the inspection result of the inspection target portion A obtained by integrating the inspection target portions A1 to A3 to "having a defect".

<U. Modification of Appearance Inspection System>
FIG. 39 is a diagram illustrating a visual inspection system according to a modification. The appearance inspection system shown in FIG. 39 is different from the appearance inspection system 1 shown in FIG. 1 in that it does not include the PLC 50 and includes an image processing device 20a instead of the image processing device 20. The image processing device 20a has both the configuration of the image processing device 20 and the configuration of the PLC 50.

FIG. 40 is a diagram showing another embodiment in which the relative position between the workpiece W and the imaging device 10 is changed. As shown in FIG. 40, the robot 30 may move the work W instead of the imaging device 10. In the example shown in FIG. 40, the imaging device 10 is fixed. By moving the work W in this manner, the relative position between the work W and the imaging device 10 may be changed.

FIG. 41 is a diagram showing still another mode in which the relative position between the workpiece W and the imaging device 10 is changed. As shown in FIG. 41, the workpiece W may be placed on the turntable 91. The rotary table 91 rotates according to an instruction from the robot controller 40. Thereby, the relative position between the workpiece W and the imaging device 10 can be easily changed.

Note that the robot 30 may be a robot other than the vertical articulated robot (for example, a horizontal articulated robot, an orthogonal robot, or the like).

In the above description, the imaging field of view FOV and the effective field of view FOV2 are described as being circular. However, the shapes of the imaging field of view FOV and the effective field of view FOV2 are not limited to circles, and may be, for example, rectangular (rectangular, square).

<V. Appendix>
As described above, this embodiment includes the following disclosure.

[Configuration 1]
An appearance inspection system (1) for performing an appearance inspection of an inspection object (W),
A display device (23);
A robot (30) for moving the imaging device (10);
While the robot (30) is moving the imaging device (10), the imaging device (10) images each of a plurality of inspection portions of the inspection object (W) to obtain the imaging device (10). An inspection unit (21) for inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained from
A display control unit for displaying, on the display device (23), an inspection result matrix (25) in which inspection results of the inspection unit (21) for each of the plurality of inspection parts are displayed for each inspection object (W). (22) An appearance inspection system comprising:

[Configuration 2]
Each row or each column of the inspection result matrix (25) is grouped in row units or column units according to a predetermined classification rule,
The display control unit (22) includes:
When an aggregation instruction is received for a group of rows or a group of columns that are grouped, the inspection results of the group of rows or the group of columns are aggregated and displayed in one row or one column,
3. The visual inspection system according to Configuration 1, wherein, when a deployment instruction for the inspection results that are collectively displayed is received, the display of the inspection results that are collectively displayed is returned to a state before the integration.

[Configuration 3]
The display control unit (22) according to the configuration 1 or 2, wherein an inspection result indicating a defect among the inspection results included in the inspection result matrix (25) is displayed in a display mode different from other inspection results. Visual inspection system as described.

[Configuration 4]
The visual inspection system (1) receives one or more inspection results from among the inspection results included in the inspection result matrix (25), thereby receiving one or more inspection objects (W). An operation unit (134) capable of selecting one and one or more inspection parts;
The inspection unit (21) or the display control unit (22) determines whether the selected inspection object (W) and the selected inspection part are based on the operation unit (134) receiving the selection operation. 4. The visual inspection system according to any one of Configurations 1 to 3, wherein the visual inspection system performs a process related to at least one of the following.

[Configuration 5]
The external appearance according to Configuration 4, wherein the display control unit (22) displays at least one of an image used for inspection of the selected inspection part and an imaging condition of the image on the display device (23). Inspection system.

[Configuration 6]
The visual inspection system (1) further includes a storage device for storing a three-dimensional model (ML) representing a shape of the inspection object (W), and the plurality of inspection parts include the three-dimensional model (ML). ) Is preset for
The display control unit (22) displays the three-dimensional model (ML) on the display device (23), and displays an inspection result of the selected inspection part on a corresponding part on the three-dimensional model (ML). A visual inspection system according to

configuration

4 or 5, wherein

[Configuration 7]
The display control unit (22) according to any one of Configurations 4 to 6, wherein, when a plurality of inspection results are selected by the selection operation, a statistical result of the plurality of inspection results is displayed on the display device (23). The visual inspection system according to the paragraph.

[Configuration 8]
The visual inspection according to any one of configurations 4 to 7, wherein the inspection unit (21) re-examines the selected inspection part based on an image used for inspection of the selected inspection part. system.

[Configuration 9]
The display control unit (22) displays a comparison result between an expected value matrix indicating a true correct value for each test result included in the test result matrix (25) and the test result matrix (25) on the display device. The visual inspection system according to any one of Configurations 1 to 8, which is displayed in (23).

[Configuration 10]
A method for displaying a result of a visual inspection performed by an imaging device (10) imaging a plurality of inspection portions of an inspection object (W),
Acquiring each image obtained by the imaging device (10) imaging each of the plurality of inspection portions while the robot (30) is moving the imaging device (10);
Inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained in the obtaining step;
Displaying a test result matrix (25) representing test results for each of the plurality of test portions for each test object (W) on a display device (23).

[Configuration 11]
A display program of an appearance inspection result obtained by imaging a plurality of inspection portions of an inspection object (W) by an imaging device (10),
The display program, the computer,
Acquiring each image obtained by the imaging device (10) imaging each of the plurality of inspection portions while the robot (30) is moving the imaging device (10);
Inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained in the obtaining step;
A step of displaying an inspection result matrix (25) representing inspection results for each of the plurality of inspection parts for each inspection object (W) on a display device (23);

実施 The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 appearance inspection system, 10 imaging device, 10A schematic diagram, 20, 20a image processing device, 21 inspection unit, 22 display control unit, 23 display device, 25 inspection result matrix, 27 expected value matrix, 29 comparison result matrix, 30 robot , 31 base, 32 arm, 32a tip arm, 40 robot controller, 50 PLC, 60 setting device, 61a, 61b screen, 71, 72, 76, 77 tool button, 74 circle, 75, 75A, 75B inspection target area , 81 back button, 82 forward button, 90 stage, 91 rotary table, 110, 214, 362 processor, 112 RAM, 114 display controller, 116 system controller, 118 I / O controller, 120, 364 storage device 122, camera interface, 124, input interface, 126, controller interface, 128, 228, 368, communication interface, 130, 222 memory card interface, 134 keyboard, 134A, 134B, 134C classification rule, 136, 224 memory card, 142 image processing program, 143 display program, 144 project file, 146A imaging condition file, 146B image file group, 146C inspection condition file, 146D inspection result file, 146E work information file, 146F production information file, 146G classification rule file, 146H expected value file, 212 chip Set, 216 non-volatile memory, 218 main memory, 220 System clock, 226 internal bus, 230 an internal bus controller, 232 control circuit, 234 an internal bus control circuit, 236 buffer memory, 238 a fieldbus controller, 361 bus, 363 main memory, 365 setting program 366 displays, 367 input device.

Claims

An appearance inspection system that performs an appearance inspection of an inspection object,
A display device;
A robot for moving the imaging device;
While the robot is moving the imaging device, the imaging device images each of the plurality of inspection portions of the inspection target, and based on each image obtained from the imaging device, the plurality of inspection portions. An inspection unit for inspecting each of the for the presence or absence of a defect,
A display control unit for displaying, on the display device, an inspection result matrix in which inspection results of the inspection unit for each of the plurality of inspection parts are displayed for each inspection object.
Each row or each column of the inspection result matrix is grouped in row units or column units according to a predetermined classification rule,
The display control unit,
When an aggregation instruction is received for a group of rows or a group of columns that are grouped, the inspection results of the group of rows or the group of columns are aggregated and displayed in one row or one column,
The visual inspection system according to claim 1, wherein, when a deployment instruction for the inspection results that are collectively displayed is received, the display of the inspection results that are collectively displayed is returned to a state before the integration.
The visual inspection system according to claim 1, wherein the display control unit displays an inspection result indicating a defect in the inspection results included in the inspection result matrix in a display mode different from other inspection results. .
The visual inspection system receives the selection operation of selecting one or more inspection results from the inspection results included in the inspection result matrix, and thereby connects one or more inspection objects and one or more inspection parts. An operation unit that can be selected,
The said inspection part or the said display control part performs the process regarding at least one of the selected test object and the selected test part based on the said operation part having received the said selection operation. 4. The visual inspection system according to any one of items 3 to 3.
The visual inspection system according to claim 4, wherein the display control unit displays at least one of an image used for inspection of the selected inspection part and an imaging condition of the image on the display device.
The visual inspection system further includes a storage device for storing a three-dimensional model representing a shape of the inspection object, wherein the plurality of inspection parts are preset for the three-dimensional model,
The visual inspection according to claim 4, wherein the display control unit displays the three-dimensional model on the display device and displays an inspection result of the selected inspection part in a corresponding part on the three-dimensional model. system.
The external appearance according to any one of claims 4 to 6, wherein when a plurality of inspection results are selected by the selection operation, the display control unit displays a statistical result of the plurality of inspection results on the display device. Inspection system.
The visual inspection system according to any one of claims 4 to 7, wherein the inspection unit re-inspects the selected inspection part based on an image used for inspection of the selected inspection part.
9. The display control unit according to claim 1, wherein the display control unit displays, on the display device, a comparison result between an expected value matrix indicating a true correct value for each inspection result included in the inspection result matrix and the inspection result matrix. The visual inspection system according to any one of claims 1 to 4.
A display method of a visual inspection result performed by an imaging device imaging a plurality of inspection portions of an inspection object,
Acquiring each image obtained by the imaging device imaging each of the plurality of inspection portions while the robot is moving the imaging device;
Inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained in the obtaining step;
Displaying an inspection result matrix representing inspection results for each of the plurality of inspection portions for each inspection object on a display device.
A display program of an appearance inspection result performed by an imaging device imaging a plurality of inspection portions of an inspection object,
The display program, the computer,
Acquiring each image obtained by the imaging device imaging each of the plurality of inspection portions while the robot is moving the imaging device;
Inspecting each of the plurality of inspection parts for the presence or absence of a defect based on each image obtained in the obtaining step;
A step of displaying, on a display device, an inspection result matrix representing inspection results for each of the plurality of inspection parts for each inspection object;