JP7380332B2 - Image processing device, control method and program for the image processing device - Google Patents

Description

The present disclosure relates to an image processing device, a method of controlling the image processing device, and a program.

2. Description of the Related Art In the field of FA (Factory Automation), technology for automatically inspecting objects to be inspected such as workpieces has become widespread. Workpiece inspection processing is realized by a combination of various processing items. Applications have been developed that provide user interfaces to easily set parameters used to execute each process item.

For example, Japanese Patent Application Publication No. 2012-123777 (Patent Document 1) discloses an information processing apparatus that extracts and displays image processing items for which parameters are set from an image processing flow. The information processing apparatus sets the image processing item specified by the user among the displayed image processing items as a reference destination for the parameter. When performing image processing, the information processing apparatus acquires parameters from the image processing item to which the reference destination is set, and executes the image processing.

Japanese Patent Application Publication No. 2012-123777

The technique described in Patent Document 1 is effective only when parameters of another image processing item can be used. That is, if the parameters of another image processing item cannot be used, it takes time and effort to set the parameters.

The present disclosure has been made in view of the above problems, and the purpose is to provide an image processing device and an image processing device that can reduce the effort of setting parameters used to execute processing items that constitute an image processing flow. The object of the present invention is to provide a control method and program.

According to an example of the present disclosure, an image processing apparatus that executes an image processing flow in which a plurality of processing items are arranged in an execution order includes a generation unit and a setting unit. The generation unit executes part or all of the image processing flow on the plurality of setting images in response to input of an instruction to set parameters used for executing a specified processing item among the plurality of processing items. As a result, a plurality of processed images in which the processing items up to the stage preceding the specified processing item have been executed are respectively generated. The setting unit sets parameter values using a plurality of processed images.

According to this disclosure, in response to an input of a parameter setting instruction, a plurality of processed images in which processing items up to a specified processing item are executed are automatically generated. Further, parameter values are set using the plurality of generated processed images. This reduces the effort required to collect multiple processed images. As a result, it is possible to reduce the effort required to set parameters used to execute processing items that constitute an image processing flow.

In the above disclosure, the setting unit executes a specified processing item on the plurality of processed images using each of the plurality of values that the parameter can take, thereby obtaining the plurality of processing results. get. The setting unit sets a value selected from the plurality of values as the value of the parameter based on the plurality of processing results.

According to the above disclosure, an appropriate value is selected from among a plurality of values that the parameter can take, with reference to a plurality of processing results.

In the above disclosure, the setting unit generates difference information indicating the difference between the plurality of processing results and the plurality of correct values set in advance for the plurality of setting images. The setting unit causes the display device to display the correspondence between each of the plurality of values and the difference information. The setting unit sets a value that has been selected from among the plurality of values as a value of the parameter.

According to the above disclosure, the user can select the optimal value while viewing the correspondence between each of the plurality of values and the difference information displayed on the display device.

In the above disclosure, the setting unit optimizes the values of the parameters so that the plurality of processing results approach the plurality of correct values set in advance for the plurality of setting images.

According to the above disclosure, the value of the parameter is automatically set so that the processing result of the designated processing item approaches the correct value. This can further reduce the effort required to set parameters used to execute the processing items that make up the image processing flow.

In the above disclosure, the setting unit optimizes the value of the parameter so that the feature amount obtained from the plurality of processing results approaches a preset reference value.

According to the above disclosure, the value of the parameter is automatically set so that the feature amount obtained from a plurality of processing results approaches the reference value. This can further reduce the effort required to set parameters used to execute the processing items that make up the image processing flow.

In the above disclosure, each of the plurality of setting images includes an object. The specified processing item is an item for determining the attributes of the object appearing in the image output from the previous processing item. The plurality of correct answer values each indicate the attributes of the object appearing in the plurality of setting images.

For example, the designated processing item is an item for determining the attributes of the object based on the comparison result between the feature amount calculated from the image output from the previous processing item and the value of the parameter.

According to the above disclosure, it is possible to easily set the parameters used to execute the processing item for determining the attributes of the object.

In the above disclosure, the specified processing item is an item for determining the attributes of an object using a learned model generated by performing machine learning using a plurality of processed images. The parameters define the trained model.

According to the above disclosure, a trained model suitable for determining the attributes of an object can be easily generated.

In the above disclosure, each of the plurality of setting images depicts an object including a characteristic portion. The specified processing item is an item for detecting the position of the characteristic portion appearing in the image output from the previous processing item using a model image in which the characteristic portion appears. The plurality of correct answer values each indicate the coordinates of the characteristic portion shown in the plurality of setting images.

According to the above disclosure, it is possible to easily set parameters used to execute a processing item for detecting the position of a characteristic portion.

In the above disclosure, each of the plurality of setting images includes text. The specified processing item is an item that performs character recognition of the text appearing in the image output from the previous processing item. The plurality of correct answer values each indicate the text appearing in the plurality of setting images.

According to the above disclosure, it is possible to easily set parameters used to execute a processing item for character recognition.

According to an example of the present disclosure, a method for controlling an image processing apparatus that executes an image processing flow in which a plurality of processing items are arranged in an execution order includes a first step and a second step. The first step is to perform part or all of the image processing flow for a plurality of setting images in response to an input instruction for setting the value of a parameter used to execute a specified processing item among a plurality of processing items. This is a step of generating each of a plurality of processed images in which the processing items up to the stage preceding the designated processing item have been executed. The second step is to set parameter values using a plurality of processed images.

According to an example of the present disclosure, a program causes a computer to execute the above control method. These disclosures can also reduce the effort required to set parameters used to execute processing items that constitute an image processing flow.

According to the present disclosure, it is possible to reduce the effort required to set parameters used to execute processing items that constitute an image processing flow.

1 is a schematic diagram showing the overall configuration of an image processing system including an image processing device according to an embodiment. FIG. 3 is a diagram illustrating a reference example of a method for setting parameters used for processing items that constitute an image processing flow. FIG. 7 is a diagram illustrating another reference example of a method for setting parameters used for processing items that constitute an image processing flow. FIG. 3 is a diagram illustrating an example of a parameter setting method in the image processing apparatus according to the present embodiment. FIG. 1 is a schematic diagram showing an example of a hardware configuration of an image processing device. 1 is a diagram illustrating an example of a functional configuration of an image processing device. 3 is a flowchart illustrating an example of the overall flow of parameter setting processing according to the present embodiment. 8 is a flowchart showing an example of the process flow of the subroutine of step S2 shown in FIG. 7. FIG. 8 is a flowchart showing an example of the process flow of the subroutine of step S3 shown in FIG. 7. FIG. 8 is a flowchart showing another example of the process flow of the subroutine of step S3 shown in FIG. 7. 3 is a diagram illustrating an image processing flow for which parameters are set in operation example 1. FIG. FIG. 7 is a diagram showing the contents of a processing item 35A for which parameters are set in operation example 1; FIG. 6 is a diagram showing a screen when setting a correct value for a setting image in operation example 1; 7 is a diagram showing a screen when setting parameters in operation example 1. FIG. 7 is a diagram showing an example of a window showing detailed information on processing results in operation example 1. FIG. 7 is a diagram showing a workpiece W in operation example 2. FIG. 7 is a diagram illustrating an image processing flow for which parameters are set in operation example 2. FIG. 7 is a diagram illustrating the contents of processing items for which parameters are set in operation example 2. FIG. 7 is a diagram illustrating an example of a setting image prepared in operation example 2. FIG. FIG. 7 is a diagram showing a screen when setting a correct value for a setting image in operation example 2; 7 is a diagram showing a screen when setting parameters in operation example 2. FIG. 7 is a diagram showing an example of a window showing detailed information on processing results in operation example 2. FIG. 23 shows a window displayed in response to the operation of button 86 shown in FIG. 22. FIG. 7 is a diagram showing an image processing flow for which parameters are set in operation example 3; 25 is a diagram showing a processing example of processing item 34C shown in FIG. 24. FIG. FIG. 7 is a diagram showing a screen when setting a correct value for a setting image in operation example 3; FIG. 7 is a diagram showing a screen when setting parameters in operation example 3; 12 is a diagram showing an example of a window showing detailed information on processing results in operation example 3. FIG.

Embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the figures are given the same reference numerals and the description thereof will not be repeated.

§1 Application Examples Application examples of the present invention will be described with reference to FIGS. 1 to 4. FIG. 1 is a schematic diagram showing the overall configuration of an image processing system including an image processing apparatus according to the present embodiment. As shown in FIG. 1, the image processing system SYS includes, as main components, an image processing device 1 also called a visual sensor, a camera 3 connected to the image processing device 1, and a camera 3 connected to the image processing device 1. The display device 6 includes a display device 6 and an input device 7.

The image processing device 1 is incorporated into a production line, and inspects the presence or absence of defects or dirt on a target object (hereinafter also referred to as "work W"), measures the size, position, orientation, etc. of the work W, and performs inspections on the surface of the work W. Performs image processing such as recognition of characters and figures above. That is, the image processing device 1 performs image processing on image data generated by imaging the workpiece W.

The camera 3, which is an imaging device, includes, for example, an optical system such as a lens, and an imaging element partitioned into a plurality of pixels, such as a CCD (Coupled Charged Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. be done. Image data acquired by imaging with the camera 3 (hereinafter also referred to as “camera image”) is transmitted to the image processing device 1. The image processing device 1 then performs image processing on the camera image captured by the camera 3. An illumination device that irradiates light onto the workpiece W imaged by the camera 3 may be further provided.

The image processing device 1 provides a user interface for creating an image processing flow according to the work W and the purpose of image processing. The user interface accepts selection of any processing item from a predefined group of processing items. The user can construct a desired image processing flow by selecting a plurality of processing items from the processing item group and specifying the execution order of the selected processing items. Note that a processing item in this specification is a functional unit having a specific purpose, and is also referred to as a "unit."

Each processing item constituting the image processing flow is executed according to the value of the corresponding parameter. The result of executing a processing item (processing result) depends on the value of the parameter. Therefore, the values of the parameters need to be appropriately set so as to obtain the desired processing results.

Normally, parameter values are set by checking the processing results when an image prepared in advance is input into a processing item. Images on which the previous processing items have been executed are input to the processing items that make up the image processing flow. Therefore, conventionally, in order to set the values of parameters used to execute the processing items that make up the image processing flow, it is necessary to set the values of the parameters used to execute the processing items that constitute the image processing flow. ) had to be prepared separately.

FIG. 2 is a diagram illustrating a reference example of a method for setting parameters used for processing items that constitute an image processing flow. FIG. 2 shows a method for setting the values of parameters used to execute processing item 33 in image processing flow 30 in which processing items 31 to 36 are executed in this order.

As shown in FIG. 2, apart from the image processing flow 30, an image collection flow 40 executes the processing items 31 and 32 up to the previous stage of the processing item 33 and the processing item 41 corresponding to "image collection" in this order. is created. Processing item 41 indicates processing for saving the input image in a designated folder. By executing the image collection flow 40, images that are the results of sequentially executing the processing items 31 and 32 on the input image are saved in the designated folder. In the example shown in FIG. 2, by executing processing item 32, the coordinates are transformed so that the orientation of the object in the image is constant.

By executing the image collection flow 40 on a plurality of setting images 22 prepared in advance, the user can execute the processing items 31 and 32 up to the processing item 33 for which parameters are to be set. A plurality of processed images 25 are collected. Then, the user sets the values of the parameters used to execute the processing item 33 so that the processing result when each of the plurality of processed images 25 is input to the processing item 33 is a desired result.

When using the setting method illustrated in FIG. 2, the user needs to create an image collection flow 40 separately from the image processing flow 30. Therefore, it takes time and effort on the part of the user.

In addition, when the value of a parameter used to execute the processing item 32 in the image processing flow 30 is changed, or when the processing item 32 is changed to another processing item, the same change is also reflected in the image collection flow 40. It is necessary to do so. If this reflection is neglected, the values of the parameters used to execute the processing item 33 are set using an image in a state different from the state of the image input to the processing item 33. As a result, the processing performance of the image processing flow 30 deteriorates.

Furthermore, when the image processing flow 30 is created for each type of workpiece W, the image collection flow 40 is also created for each type. Setting the value of the parameter of the processing item 33 in the image processing flow 30 for type A needs to be performed using the processed image obtained by executing the image collection flow 40 for type A. However, a mistake may occur in the combination of the image processing flow 30 and the image collection flow 40. For example, the value of the parameter of the processing item 33 in the image processing flow 30 for product type A may be erroneously set using the processed image obtained by executing the image collection flow 40 for product type B. As a result, the processing performance of the image processing flow 30 deteriorates.

FIG. 3 is a diagram illustrating another reference example of a method for setting parameters used for processing items that constitute an image processing flow. FIG. 3 shows a method for setting values of parameters used to execute processing item 33 in image processing flow 50 in which processing items 31 to 36 are executed in this order.

As shown in FIG. 3, in the image processing flow 50, a branch point 51 is provided between the processing item 32 and the processing item 33 to branch to the processing item 41 corresponding to "image collection". In the image processing flow 50, processing items 33 to 36 are executed in order after processing item 32 in response to receiving a normal instruction, and processing item 41 is executed after processing item 32 in response to receiving an image collection instruction. is executed.

When setting the value of a parameter used to execute a processing item 33, the user inputs an image collection instruction into the image processing flow 50 to set the processing item for a plurality of setting images 22 prepared in advance. 31 and 32 collect a plurality of processed images 25, respectively. Then, the user sets the values of the parameters used to execute the processing item 33 so that the processing result when each of the plurality of processed images 25 is input to the processing item 33 is a desired result.

In the setting method illustrated in FIG. 3, unlike the setting method illustrated in FIG. 2, the effort of creating an image collection flow separately from the image processing flow is reduced. Furthermore, as in the setting method illustrated in FIG. 2, it is possible to prevent performance degradation of the image processing flow due to an incorrect combination of the image processing flow and the image collection flow.

However, when it is desired to set parameter values for each of N processing items (N is an integer of 2 or more) constituting the image processing flow 50, N branch points 51 corresponding to each of the N processing items are set. It is necessary to provide this in the image processing flow 50. Therefore, the degree of difficulty in designing the image processing flow 50 increases. As a result, it takes time and effort to set up the image processing flow 50.

Furthermore, when the value of the parameter used to execute the processing item 32 in the image processing flow 50 is changed, or when the processing item 32 is changed to another processing item, the task of collecting the plurality of processed images 25 again is performed. and setting the parameter values. As a result, it takes time and effort to set the values of parameters used to execute the processing item 33.

In the image processing apparatus 1 according to the present embodiment, the following setting method is executed in order to reduce the effort of setting parameters used for executing processing items.

FIG. 4 is a diagram illustrating an example of a parameter setting method in the image processing apparatus according to the present embodiment. In the present embodiment, the image processing apparatus 1 processes a plurality of setting images 22 prepared in advance in response to an input of an instruction to set parameters used to execute the processing items 33 constituting the image processing flow 30. By executing the image processing flow 30, a plurality of processed images 25 in which the processing items 31 and 32 up to the stage before the processing item 33 have been executed are respectively generated. Furthermore, the image processing device 1 uses the plurality of generated processed images 25 to set values of parameters used to execute the processing items 33.

According to the image processing device 1 according to the present embodiment, a plurality of processing items 31 and 32 up to the preceding stages of the processing item 33 are executed in response to an input of an instruction to set parameters used for executing the processing item 33. A processed image 25 is automatically generated. Further, parameter values are set using the plurality of generated processed images 25. This eliminates the need to create an image collection flow 40 as illustrated in FIG. 2 in order to generate a plurality of processed images 25, and image processing including a branch point 51 as illustrated in FIG. There is no need to create the flow 50. As a result, the effort required to collect a plurality of processed images 25 is reduced.

Furthermore, even if the value of the parameter used to execute the preceding processing item 32 in the image processing flow 30 is changed, or even if the processing item 32 is changed to another processing item, the execution of the processing item 33 may be changed. A plurality of processed images 25 are automatically collected by inputting instructions for setting the parameters to be used. Therefore, the user does not need to perform a separate operation to collect the plurality of processed images 25.

As described above, according to the present embodiment, it is possible to reduce the effort required to set parameters used to execute the processing items 33 that constitute the image processing flow 30.

§2 Specific example <A. Hardware configuration of image processing device>
FIG. 5 is a schematic diagram showing an example of the hardware configuration of the image processing device. As shown in FIG. 5, the image processing device 1 typically has a structure that follows a general-purpose computer architecture, and a processor executes a preinstalled program to perform various types of processing as described below. Achieve processing.

More specifically, the image processing device 1 includes a processor 110 such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), a RAM (Random Access Memory) 112, a display controller 114, and a system controller 116. , an I/O (Input Output) controller 118, a hard disk 120, a camera interface 122, an input interface 124, a PLC interface 126, a communication interface 128, and a memory card interface 130. These units are connected to each other so as to be able to communicate data, centering on the system controller 116.

The processor 110 exchanges programs (codes) and the like with the system controller 116 and executes them in a predetermined order, thereby achieving the desired arithmetic processing.

The system controller 116 is connected to the processor 110 , RAM 112 , display controller 114 , and I/O controller 118 via buses, and exchanges data with each section, as well as performs overall processing of the image processing device 1 . in charge of

The RAM 112 is typically a volatile storage device such as DRAM (Dynamic Random Access Memory), and stores programs read from the hard disk 120, camera images (image data) acquired by the camera 3, and camera images. Stores processing results and work data.

The display controller 114 is connected to the display device 6, and outputs signals for displaying various information to the display device 6 according to internal commands from the system controller 116. The display device 6 includes, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, an organic EL, and the like.

The I/O controller 118 controls data exchange with recording media and external devices connected to the image processing apparatus 1 . More specifically, I/O controller 118 is connected to hard disk 120, camera interface 122, input interface 124, PLC interface 126, communication interface 128, and memory card interface 130.

The hard disk 120 is typically a nonvolatile magnetic storage device, and stores the control program 121 executed by the processor 110 and the like. The control program 121 installed on the hard disk 120 is distributed in a state stored in the memory card 106 or the like. Furthermore, the hard disk 120 stores camera images. Note that instead of the hard disk 120, a semiconductor storage device such as a flash memory or an optical storage device such as a DVD-RAM (Digital Versatile Disk Random Access Memory) may be used.

The camera interface 122 corresponds to an input unit that receives image data generated by imaging the workpiece W, and mediates data transmission between the processor 110 and the camera 3. More specifically, the camera interface 122 can be connected to one or more cameras 3, and an imaging instruction is output from the processor 110 to the camera 3 via the camera interface 122. Thereby, the camera 3 images the subject and outputs the generated image to the processor 110 via the camera interface 122.

Input interface 124 mediates data transmission between processor 110 and input device 7 such as a keyboard, mouse, touch panel, dedicated console, etc. That is, the input interface 124 receives an operation command given by the user operating the input device 7.

A PLC (Programmable Logic Controller) interface 126 mediates data transmission between the processor 110 and a PLC (not shown). More specifically, the PLC interface 126 transmits information related to the state of the production line controlled by the PLC, information related to the workpiece W, etc. to the processor 110.

Communication interface 128 mediates data transmission between processor 110 and other personal computers, server devices, etc. (not shown). The communication interface 128 typically includes Ethernet (registered trademark), USB (Universal Serial Bus), or the like. Note that, as will be described later, instead of installing the program stored in the memory card 106 on the image processing apparatus 1, a program downloaded from a distribution server or the like may be installed on the image processing apparatus 1 via the communication interface 128. It's okay.

Memory card interface 130 mediates data transmission between processor 110 and memory card 106, which is a recording medium. That is, the memory card 106 stores and distributes the control program 121 to be executed by the image processing apparatus 1 , and the memory card interface 130 reads the control program 121 from the memory card 106 . Furthermore, the memory card interface 130 writes the camera image acquired by the camera 3 and/or the processing result in the image processing device 1 to the memory card 106 in response to an internal command from the processor 110 . Note that the memory card 106 may be a general-purpose semiconductor storage device such as an SD (Secure Digital), a magnetic recording medium such as a flexible disk, or an optical recording medium such as a CD-ROM (Compact Disk Read Only Memory). Consists of etc.

When using a computer having a structure according to the general-purpose computer architecture as described above, in addition to an application for providing the functions according to this embodiment, an OS for providing the basic functions of the computer is required. (Operating System) may be installed. In this case, the program according to the present embodiment may execute processing by calling necessary modules in a predetermined order and/or timing from among the program modules provided as part of the OS. good. That is, the program itself according to the present embodiment does not include the above-mentioned modules, and may execute processing in cooperation with the OS.

Furthermore, the control program 121 according to this embodiment may be provided by being incorporated into a part of another program. Even in that case, the program itself does not include modules included in other programs to be combined as described above, and processing is executed in cooperation with the other programs. That is, the control program 121 according to the present embodiment may be incorporated into such another program.

Alternatively, part or all of the functions provided by executing the control program 121 may be implemented as a dedicated hardware circuit.

<B. Functional configuration of image processing device>
FIG. 6 is a diagram illustrating an example of the functional configuration of an image processing device. As shown in FIG. 6, the image processing device 1 includes a flow creation section 11, a flow execution section 16, and a storage section 20. The flow creation unit 11 and the flow execution unit 16 are realized by the processor 110 executing the control program 121. Storage unit 20 is realized by hard disk 120 and RAM 112.

The flow creation unit 11 provides a screen to the display device 6 to create an image processing flow, and creates an image processing flow desired by the user in response to input to the input device 7. Typically, the flow creation unit 11 receives the selection of one or more processing items from a group of predefined processing items and the designation of the execution order of the selected one or more processing items. Create an image processing flow according to the content. Note that an image processing flow consisting of a plurality of processing items is usually created. Therefore, below, a case will be described in which an image processing flow composed of a plurality of processing items is created.

The processing item group includes processing items related to image capture, processing items related to image correction, processing items related to inspection or measurement of characteristic parts in images, processing items related to assisting the above inspections or measurements, and processing related to output of processing results. This includes items, processing items related to display of processing results, processing items related to branching of image processing flow, etc.

Processing items related to image correction include, for example, geometric transformation, various types of filter processing, image synthesis for generating one image from a plurality of images, 3D imaging, and the like. Geometric transformations include positional displacement correction, reduction, enlargement, projective transformation, perspective transformation, etc. that perform parallel translation and rotation. Image synthesis includes, for example, generating high dynamic range images. 3D imaging is a process of visualizing information regarding distance and 3D shape by calculating a plurality of images, and includes multi-view stereo, photometric stereo, deflectometry processing, and the like.

Processing items related to inspection or measurement include shape identification and position detection of characteristic parts, edge position detection, labeling that assigns labels to pixels using threshold processing for light and color images, and labeling that applies labels to all pixels based on image recognition. This includes semantic segmentation, which assigns a label to the identified object.

The flow creation unit 11 provides the display device 6 with a screen including, for example, a list of objects corresponding to each processing item and a work area in which objects selected from the list can be arbitrarily arranged. The user selects a plurality of objects from the list that respectively correspond to a plurality of desired processing items, and edits the arrangement order of the selected plurality of objects in the work area so as to match the desired execution order. The flow creation unit 11 creates an image processing flow according to a plurality of objects arranged in the work area and their arrangement order.

The flow creation unit 11 generates flow data 21 indicating a plurality of processing items constituting the created image processing flow and the execution order of the plurality of processing items, and stores the generated flow data 21 in the storage unit 20. .

The flow creation unit 11 includes a management unit 12, a generation unit 13, and a setting unit as a configuration for setting the values of parameters used to execute each of a plurality of processing items that constitute the image processing flow indicated by the flow data 21. 14.

The management unit 12 manages correct values for each of a plurality of setting image files (hereinafter referred to as "a plurality of setting images 22") used for setting parameter values. The plurality of setting images 22 may be selected from camera images acquired in the past (images obtained by imaging with the camera 3), or may be selected from an external PC or the like via the communication interface 128 (see FIG. 5). It may be obtained from Alternatively, the management unit 12 may output a plurality of imaging instructions to the camera 3 and obtain a plurality of images respectively captured according to the plurality of imaging instructions as a plurality of setting images 22. The plurality of setting images 22 are stored in the storage unit 20. Alternatively, the plurality of setting images 22 may be stored in the external storage 2. The external storage 2 includes a memory card 106 (see FIG. 5), a USB (Universal Serial Bus) memory, an external HDD (Hard Disk Drive), an external SSD (Solid State Drive), a NAS (Network Attached Storage), etc. It will be done. In the following description, it is assumed that a plurality of setting images 22 are stored in the storage unit 20.

The management unit 12 manages correct values corresponding to processing items for which parameters are set. For example, in the case of a processing item that determines the attributes of the workpiece W (for example, the type of workpiece, the quality of the workpiece, etc.), the management unit 12 manages the attributes of the workpiece W shown in each setting image 22 as correct values. In the case of a processing item that measures the position of a characteristic part (for example, a printed mark) of the workpiece W, the management unit 12 manages the position of the characteristic part shown in each setting image 22 as a correct value. In the case of a processing item for character recognition of text printed on the workpiece W, the management unit 12 manages the text appearing in each setting image 22 as a correct value.

The management unit 12 may acquire the correct value from the input device 7 . The management unit 12 generates management data 23 indicating the correct value of each of the plurality of setting images 22, and stores the generated management data 23 in the storage unit 20.

The generation unit 13 receives an input of an instruction to set a parameter used to execute a specified processing item among a plurality of processing items that constitute the image processing flow indicated by the flow data 21 . The generation unit 13 executes the image processing flow indicated by the flow data 21 on the plurality of setting images 22 in response to the input of the parameter setting instruction, thereby generating the processing items up to the stage preceding the specified processing item. A plurality of processed images 25 are generated, respectively.

The setting unit 14 uses the plurality of processed images 25 generated by the generation unit 13 to set values of parameters used to execute the specified processing item. The setting unit 14 generates setting data 24 indicating the values of the set parameters, and stores the generated setting data 24 in the storage unit 20.

The flow execution unit 16 executes the image processing flow created by the flow creation unit 11. That is, upon receiving an execution instruction, the flow execution unit 16 sequentially executes a plurality of processing items forming the image processing flow indicated by the flow data 21 in accordance with the execution order. The flow execution unit 16 executes each processing item using the parameter values indicated by the setting data 24.

<C. Parameter setting process>
The flow of the parameter setting process will be explained with reference to FIGS. 7 to 10. FIG. 7 is a flowchart showing an example of the overall flow of the parameter setting process according to the present embodiment.

First, the processor 110 determines whether a parameter setting instruction has been received from the input device 7 (step S1). The parameter setting instruction includes designation of the processing item for which the parameter is to be set. If a parameter setting instruction has not been received (NO in step S1), the process returns to step S1.

When receiving a parameter setting instruction (YES in step S1), the processor 110, which operates as the generation unit 13, executes the image processing flow on the plurality of setting images 22, and processes up to the previous stage of the specified processing item. A plurality of processed images 25 in which the processing items have been executed are generated (step S2). Note that the processor 110 may execute all of the plurality of processing items that make up the image processing flow in accordance with the execution order, or may execute some of the plurality of processing items (at least up to the preceding stage of the specified processing item). (including processing items) may be executed.

Next, the processor 110 as the setting unit 14 uses the plurality of processed images 25 to set values of parameters used to execute the specified processing item (step S3). The processor 110 stores the setting data 24 indicating the values of the set parameters in the storage unit 20 (step S4). When step S4 is completed, the parameter setting process for the specified processing item is completed.

Note that the processor 110 may execute steps S1 to S4 for each of the plurality of processing items that constitute the image processing flow. As a result, parameter values are set for each of the plurality of processing items that constitute the image processing flow.

FIG. 8 is a flowchart showing an example of the process flow of the subroutine of step S2 shown in FIG. In the flowchart illustrated in FIG. 8, the processor 110 selects a plurality of setting images 22 in response to input to the input device 7 (step S21). Next, the processor 110 sets a correct value for each of the plurality of setting images 22 in accordance with the input to the input device 7 (step S22).

Next, the processor 110 determines whether a start instruction has been input to the input device 7 (step S23). If the start instruction has not been input (NO in step S23), the process returns to step S21.

When the start instruction is input (YES in step S23), the processor 110 executes the image processing flow for each of the plurality of setting images 22 (step S24). Then, the processor 110 temporarily stores in the storage unit 20 the plurality of processed images 25 in which the processing items up to the steps preceding the designated processing item have been executed (step S25). After step S25 ends, the process returns to step S3 (see FIG. 7).

FIG. 9 is a flowchart showing an example of the process flow of the subroutine of step S3 shown in FIG. In the flowchart illustrated in FIG. 9, the processor 110 executes a specified processing item on the plurality of processed images 25 using each of the plurality of values that the parameter can take. (step S31).

Next, the processor 110 generates difference information indicating the difference between the plurality of processing results and the plurality of correct values prepared in advance for the plurality of setting images 22 (step S32). The processor 110 identifies the correct value set for each of the plurality of setting images 22 from the management data 23 stored in the storage unit 20.

The processor 110 causes the display device 6 to display the generated difference information for each of the plurality of values that the parameter can take (step S33). The processor 110 sets the value selected from the plurality of values as the value of the parameter in response to the input from the input device 7 (step S34). After step S34 ends, the process returns to step S4 (see FIG. 7).

FIG. 10 is a flowchart showing another example of the process flow of the subroutine of step S3 shown in FIG. The flowchart illustrated in FIG. 10 includes step S35 instead of steps S33 and S34, compared to the flowchart illustrated in FIG. In step S35, the processor 110 selects the value with the smallest difference from the plurality of values, and sets the selected value as the value of the parameter. According to the flowchart exemplified in FIG. The parameter values are optimized so as to approach a plurality of correct values set in advance for each of the setting images 22.

<D. Operation example 1>
Next, an example of the operation of parameter setting in the image processing device 1 will be described.

FIG. 11 is a diagram illustrating an image processing flow for which parameters are set in operation example 1. The image processing flow 30A illustrated in FIG. 11 includes processing items 31A to 39A.

The processing item 31A is an item for acquiring an image to be subjected to image processing. The processing item 32A is an item for detecting the area where the workpiece W is captured from the image acquired by the processing item 31A. The processing item 33A is an item for performing coordinate transformation of the image so that the area in which the workpiece W is captured is located at the center of the image.

The processing item 34A is an item for performing filter processing to emphasize defects that may occur in the first portion of the workpiece W with low brightness (for example, the black or brown peripheral portion). By executing the processing item 34A, the shape, size, and brightness (pixel value) of the defect occurring in the first portion of the workpiece W change. The shape, size, and degree of change in pixel values of the defect depend on the values of the parameters used to perform process item 34A. The processing item 35A is an item for inspecting the presence or absence of defects in the first portion (for example, the outer peripheral portion) of the workpiece W using the image obtained by executing the processing item 34A.

The processing item 36A is an item for switching images. In the image processing flow 30A illustrated in FIG. 11, the processing item 36A is set so as to output an image obtained by executing the processing items 31A to 33A.

The processing item 37A is an item for performing filter processing for emphasizing defects that may occur in the second portion of the workpiece W having high brightness (for example, the white inner peripheral portion). By executing the processing item 37A, the shape, size, and brightness (pixel value) of the defect occurring in the second portion of the workpiece W change. The processing item 38A is an item for inspecting the presence or absence of defects in the second portion (for example, the inner peripheral portion) of the workpiece W using the image obtained by executing the processing item 37A. The processing item 39A is an item that outputs a comprehensive judgment result according to the test results of the processing item 35A and the test result of the processing item 37A.

Hereinafter, an example of the operation of the image processing apparatus 1 when receiving an instruction to set parameters used for executing the processing item 35A in the image processing flow 30A will be described.

<D-1. Processing items for which parameters are set>
FIG. 12 is a diagram showing the contents of the processing item 35A for which parameters are set in the first operation example. The processing item 35A is an item for determining whether or not the workpiece W shown in the input image includes a defect using the learned model 140 created in advance. The defects may be, for example, scratches, dirt, cracks, dents, burrs, color unevenness, foreign matter contamination, and the like.

The trained model 140 converts an input image (hereinafter referred to as "input image I10") into a feature amount, and from the feature amount obtained by the conversion, an image in which the input image I10 is restored (hereinafter referred to as "restored image I10"). This is a generative model configured to generate an image I11.

The trained model 140 is generated by machine learning using a plurality of images of non-defective products prepared in advance. A known technique can be used as a method for generating a trained model. The trained model 140 is configured by a neural network, eigenvectors derived by principal component analysis, and the like. Examples of neural networks include well-known autoencoders, GANs (Generative Adversarial Networks), and the like. Examples of the eigenvector include eigenvectors of known subspaces.

When executing processing item 35A, processor 110 generates difference image I12 by calculating the difference between input image I10 and restored image I11. A pixel that is different between the input image I10 and the restored image I11 has a larger pixel value in the difference image I12. On the other hand, the pixel value that is less different between the input image I10 and the restored image I11 becomes smaller in the difference image I12. In this embodiment, for convenience of explanation, the larger the difference between a pixel between the input image I10 and the restored image I11, the larger the actual pixel value of the corresponding pixel in the difference image I12, and the smaller the difference is, the larger the actual pixel value of the corresponding pixel in the difference image I12 becomes. Assume that the actual pixel value of the corresponding pixel in I12 is also smaller. However, "the pixel value is large" and "the pixel value is small" each indicate the relationship between the difference between the input image I10 and the restored image I11, and the actual pixel value of the pixel in the difference image I12. It does not have to correspond to For example, the difference image I12 may be calculated such that the larger the difference between pixels, the smaller the actual pixel value of the corresponding pixel, and the smaller the difference between the pixels, the larger the actual pixel value of the corresponding pixel.

In the example of FIG. 12, the pixels that are more different between the input image I10 and the restored image I11 are whiter in the difference image I12, and the pixels that are less white are blacker. For example, when the value of each pixel is expressed in 256 gradations, the maximum value of the pixel value of the pixels of the difference image I12 may be "255", and the minimum value may be "0". In this case, the larger the pixel value of the pixel of the difference image I12, the more the difference occurs between the input image I10 and the restored image I11, and the smaller the pixel value of the pixel of the difference image I12, This shows that there is no difference between the image I11 and the restored image I11. However, the relationship between the difference that occurs between the input image I10 and the restored image I11 and the pixel values of the difference image I12 is not limited to this example. For example, the relationship between the degree of difference between the input image I10 and the restored image I11 and the pixel value of the difference image I12 may be the opposite.

If a feature (such as defect 8) that does not appear or is unlikely to appear in the image used for machine learning of the learned model 140 appears in the input image I10, the reproducibility of that feature in the restored image I11 is low. Therefore, a relatively large difference may occur between the input image I10 and the restored image I11.

The processor 110 binarizes each pixel of the difference image I12 using the threshold Th1. For example, the processor 110 converts the pixel value of a pixel whose pixel value is equal to or greater than the threshold value Th1 to "255", and converts the pixel value of a pixel whose pixel value is less than the threshold value Th1 to "0". “More than” may be replaced with “more than” and “less than” may be replaced with “less than”. The same applies to the following description. Thereby, the processor 110 can generate the binarized image I13. By appropriately setting the threshold Th1, it is possible to obtain a binarized image I13 in which relatively low-level differences such as differences due to noise are excluded from the original difference image I12.

Differences due to the defect 8 and differences due to insufficient learning may appear in the binarized image I13. Among the causes of these differences, the defect 8 may have attributes related to shape, such as area, width, height, circumference, aspect ratio, and circularity. That is, when the defect 8 exists in the workpiece W shown in the input image I10, the corresponding position in the binarized image I13 is an area where white "255" pixels (hereinafter also referred to as white pixels) are gathered, A region having the same shape-related features as defect 8 appears. Therefore, by setting a threshold value Th2 for a feature amount related to the shape (for example, area), it is possible to determine whether or not the defect 8 is included in the input image I10.

The processor 110 identifies the area of continuous white pixels in the binarized image I13 as one area, and determines whether each area of white pixels satisfies the threshold Th2. Then, the processor 110 leaves the area that satisfies the threshold Th2 as it is, and converts the pixel values of pixels in the area that does not satisfy the threshold Th2 to "0". For example, if the threshold Th2 is set for area, the processor 110 determines whether the area of each region of white pixels is equal to or larger than the threshold Th2. Then, the processor 110 converts the pixel values of pixels within the area whose area is less than the threshold Th2 to "0". Thereby, the processor 110 can generate the detected image I14. By appropriately setting the threshold value Th2, it is possible to obtain a detected image I14 in which a white area that does not satisfy the attributes of the defect 8 is excluded from the binarized image I13.

The processor 110 determines whether the defect 8 is shown in the input image I10, depending on whether a white pixel area exists in the detected image I14. Specifically, when a region of white pixels exists in the detected image I14, the processor 110 determines that the defect 8 is shown in the corresponding region of the input image I10. On the other hand, if there is no area of white pixels in the detected image I14, the processor 110 determines that the defect 8 is not included in the input image I10.

Threshold Th1. Th2 is a parameter used to execute the processing item 35A. Threshold Th1. By appropriately setting Th2, the presence or absence of the defect 8 in the workpiece W can be determined with high accuracy.

The image on which the processing items 31A to 34A have been executed is input to the processing item 35A. As described above, if the image for which processing item 34A has been executed shows a workpiece W with a defect in the first portion, the shape, size, and pixel value of the defect will change by executing processing item 34A. . The degree depends on the value of the parameter used to execute the processing item 34A. The threshold values Th1 and Th2 need to be appropriately set according to the shape, size, and pixel value of the defect in the first portion that may appear in the image in which the processing item 34A is performed. Therefore, each time the value of the parameter used to execute the process item 34A is changed, it is necessary to reset the value of the parameter (that is, threshold value Th1, Th2) used to execute the process item 35A.

<D-2. Managing image data for settings>
FIG. 13 is a diagram showing a screen when setting a correct value for a setting image in operation example 1. A screen 60 illustrated in FIG. 13 is displayed on the display device 6.

The screen 60 includes a button 61 for setting a folder (hereinafter referred to as a "temporary storage folder") in which the processed image 25 is temporarily stored. In response to the operation of the button 61, the processor 110 displays a window (not shown) for accepting the designation of a temporary storage folder, and sets the temporary storage folder according to the designation.

The screen 60 includes a tab 80 for managing correct values for the setting image 22. FIG. 13 shows the screen 60 when the tab 80 is selected.

When the tab 80 is selected, a button 62 for selecting a folder in which a plurality of setting images 22 are stored is displayed on the screen 60. In response to the operation of the button 62, the processor 110 displays a window (not shown) for accepting the designation of the folder in which the plurality of setting images 22 are saved. The processor 110 causes the screen 60 to display a list 63 of the plurality of setting images 22 stored in the designated folder.

The list 63 includes a file name, an attribute, a comment, and a defect position for each of the plurality of setting images 22. The attribute indicates the correct value for the setting image 22. Specifically, the attribute indicates whether the workpiece W shown in the setting image 22 is a good product or a defective product. In FIG. 13, "OK" indicates a non-defective product, and "NG" indicates a defective product. Comments are arbitrarily entered by the user. The defect position is set for the setting image 22 whose attribute is "defective product".

A cursor 63a for selecting one setting image 22 is displayed in the list 63. Screen 60 includes an area 64 for displaying images. In the area 64, the setting image 22 selected by the cursor 63a is displayed. By operating the icon group 65, the user can enlarge or reduce the setting image 22 displayed in the area 64.

The user inputs attributes into the list 63 while checking the area 64. The user can input comments into the list 63 if necessary. When a defective product (“NG”) is input as the attribute, the processor 110 displays a frame line 64a in the area 64. The user adjusts the size and position of the frame line 64a so as to surround the defect 8. When the adjustment is completed, the user inputs into the list 63 that the setting of the defect position has been completed. In this way, the user inputs attributes, comments, and defect positions for each setting image 22.

The screen 60 includes a button group 66 for collectively inputting two or more setting images 22 included in the list 63. By operating the button group 66, the user can collectively input attributes and comments for two or more setting images 22.

Screen 60 includes a button 67 for instructing creation of management data 23. In response to the operation of button 67, processor 110 creates management data 23 based on the input to list 63 and the size and position of frame line 64a of area 64. Management data 23 according to operation example 1 indicates attributes, comments, and defect positions for each setting image 22.

Screen 60 includes a button 68 for closing screen 60. When the button 68 is operated, the processor 110 ends the process of setting the correct value for the setting image 22.

<D-3. Parameter settings>
FIG. 14 is a diagram showing a screen when setting parameters in operation example 1. Screen 60 includes a tab 71 for generating a trained model 140 by machine learning, and a tab 69 for setting parameters. FIG. 14 shows the screen 60 when the tab 69 is selected.

Screen 60 includes a button 70 for loading trained model 140. When the button 70 is operated, the processor 110 displays a window (not shown) for accepting the designation of the learned model 140. Processor 110 reads trained model 140 according to the specification. The trained model 140 is generated in advance according to the selection of the tab 71. Alternatively, the trained model 140 may be acquired from an external PC or the like via the communication interface 128 (see FIG. 5).

Screen 60 includes a button 72 for starting generation of processed image 25 and generation of difference information.

When the button 72 is operated, the processor 110 operating as the generation unit 13 executes the image processing flow 30A on the plurality of setting images 22, and the processing items 31A to 34A up to the stage before the processing item 35A are executed. A plurality of processed images 25 are respectively generated. The processor 110 saves the generated plurality of processed images 25 in a temporary storage folder set according to the operation of the button 61.

When the plurality of processed images 25 are saved in the temporary storage folder, the processor 110 operating as the setting unit 14 executes the processing item 35A on each of the plurality of processed images 25 and generates a processing result.

Specifically, as shown in FIG. 12, the processor 110 applies the learned model 140 to each of the plurality of processed images 25 to generate a restored image. Then, the processor 110 applies the threshold Th1 to the difference image between each processed image 25 and the restored image of the processed image 25 to generate a binarized image. Furthermore, the processor 110 applies a threshold Th2 to the binarized image to determine the presence or absence of defects. The processor 110 generates a processing result indicating the presence or absence of a defect.

The processor 110 generates difference information indicating the difference between the processing results for the plurality of processed images 25 each generated from the plurality of setting images 22 and the correct value set for each of the plurality of setting images 22. do. In operation example 1, the difference information includes the number of setting images 22 whose processing results and correct values match among the plurality of setting images 22 (hereinafter also referred to as "number of correct answers"), and the processing results and correct answers. The number of setting images 22 whose values do not match (hereinafter also referred to as "number of incorrect answers") is shown. Note that the difference information may indicate only the number of correct answers or the number of incorrect answers. Alternatively, the difference information may indicate at least one of the ratio of the number of correct answers to the total number of the plurality of setting images 22 and the ratio of the number of incorrect answers to the total number.

The processor 110 generates difference information for each of a plurality of values that the parameters (threshold Th1 and threshold Th2) used to execute the processing item 35A can take. The processor 110 displays on the screen 60 a table 73 that associates each of a plurality of possible values of the parameters (threshold value Th1 and threshold value Th2) used to execute the processing item 35A with difference information. It is preferable that the user selects a value with a large number of correct answers. Therefore, it is preferable that the processor 110 sorts the possible values of the parameters (threshold Th1 and threshold Th2) in descending order of the number of correct answers indicated by the difference information.

A cursor 73a is displayed on the table 73. The user can select one record in the table 73 by moving the cursor 73a.

Screen 60 includes a button 74 for checking detailed information on the processing result. In response to the operation of the button 74, the processor 110 displays detailed information on the processing result of the record selected by the cursor 73a on the display device 6.

FIG. 15 is a diagram showing an example of a window showing detailed information on processing results in operation example 1. The window 75 illustrated in FIG. 15 is displayed in response to the operation of the button 74 on the screen 60. Note that the window 75 may be displayed overlapping the screen 60.

Window 75 includes regions 76 and 77. In the area 76, a list of setting images 22 for which "OK" is set as an attribute (correct value) among the plurality of setting images 22 is displayed. The list displayed in the area 76 includes, for each setting image 22, a file name, a comment, and a flag indicating whether or not the processing result matches the correct value (marked as "overseen" in the figure). flag). In the area 76, the flag "○" indicates that the setting image 22 in which the workpiece W, which is a non-defective item, is shown is determined to have a defect.

In the area 77, a list of the setting images 22 for which "NG" is set as the attribute (correct value) among the plurality of setting images 22 is displayed. The list displayed in the area 77 includes, for each setting image 22, a file name, a comment, and a flag indicating whether or not the processing result matches the correct value (marked as "missed" in the figure). flag). In the area 77, the flag “◯” indicates that the setting image 22 showing the defective work W is determined to have no defects.

The character strings input in the comment field of the list 63 shown in FIG. 13 are displayed in the comment fields of the lists displayed in areas 76 and 77.

Window 75 includes a button 78 for closing window 75. In response to the operation of button 78, processor 110 closes window 75 and displays screen 60 shown in FIG. 14 on display device 6.

As shown in FIG. 14, the screen 60 includes buttons 79 for setting values of parameters (threshold Th1 and threshold Th2) used to execute the processing item 35A. In response to the operation of the button 79, the processor 110 sets the value of the parameter included in the record selected by the cursor 73a in the table 73 as the value to be used when executing the processing item 35A. That is, the processor 110 generates setting data 24 indicating the value of the parameter included in the record selected by the cursor 73a, and stores the generated setting data 24 in the storage unit 20.

As described above, a plurality of processed images 25 are automatically generated by executing the processing items 31A to 34A on the plurality of setting images 22, and the processing item 35A is automatically generated using the plurality of processed images 25. Threshold value Th1 and threshold value Th2 used for execution are set. Thereby, the threshold Th1 and the threshold Th2 can be appropriately set. Note that the processing items preceding the processing item 35A for which parameters are set are not limited to the processing items 31A to 34A shown in FIG. 11. For example, the above-mentioned geometric processing may be arranged as a processing item preceding the processing item 35A. In this case, the threshold Th1 and the threshold Th2 used to execute the processing item 35A are set using the plurality of processed images 25 on which the geometric processing has been performed.

The above operation example is also applied when receiving an instruction to set parameters used to execute the processing item 38A. Note that there are setting images 22 that have no defects on the inner periphery but have defects on the outer periphery, and there are also setting images 22 that have no defects on the outer periphery but have defects on the inner periphery. Therefore, the management data 23 is created for the processing item 38A.

Further, parameters may be set using a plurality of processed images 25 to define the trained model 140 used to execute the processing item 35A. That is, when the tab 71 shown in FIG. 14 is selected, one or more processed images generated from the setting image 22 for which "OK" is set as the attribute (correct value) among the plurality of processed images 25 are selected. A learned model 140 is generated by machine learning using 25 (good product images). The parameters define the neural network or eigenvectors that make up the trained model 140.

The image may be divided into a plurality of patch images, and the trained model 140 may be generated for each patch image. For example, an image is divided into N patch images. In this case, the processor 110 may generate the learned model 140 corresponding to the k-th patch image by machine learning using the k-th patch image acquired from each of the plurality of processed images 25. However, the processor 110 excludes from the machine learning images the patch images acquired from the processed image 25 corresponding to the setting image 22 in which the position of the k-th patch image and the defect position indicated by the management data 23 overlap. do.

<E. Operation example 2>
FIG. 16 is a diagram showing the workpiece W in operation example 2. As shown in FIG. 16, the workpiece W has a substantially rectangular shape in plan view. A mark 150 is printed on the upper surface of the workpiece W. The workpiece W is placed on an XY stage 4 having four pins 4a to 4d so as to be surrounded by the pins 4a to 4d.

FIG. 17 is a diagram illustrating an image processing flow for which parameters are set in operation example 2. The image processing flow 30B illustrated in FIG. 17 includes processing items 31B to 36B.

The processing item 31B is an item for acquiring an image to be subjected to image processing. The processing item 32B is an item for applying a filter for extracting pixels forming an edge to the image acquired by the processing item 31B. By executing processing item 32B, the shape, size, and brightness (pixel value) of the edge of mark 150 change. The shape and size of the edge of mark 150 and the degree of change in pixel value depend on the values of the parameters used to execute processing item 32B.

The processing item 33B is an item for detecting the position of the mark 150 by performing pattern matching using a model image on the image on which the processing item 32B has been executed. The processing item 34B is an item for setting and holding data regarding the XY stage 4. The processing item 35B is an item for calculating the amount of movement of the XY stage 4 required to move the mark 150 to the target position based on the position of the mark 150 obtained by executing the processing item 33B and the target position. . The processing item 36B is an item that outputs the amount of movement obtained by executing the processing item 35B.

Hereinafter, an example of the operation of the image processing apparatus 1 when receiving an instruction to set parameters used to execute the processing item 33B in the image processing flow 30A will be described.

<E-1. Processing items for which parameters are set>
FIG. 18 is a diagram showing the contents of processing items for which parameters are set in operation example 2. As shown in FIG. 18, the processing item 33B includes the processing of steps S101 to S106. Note that each step may use a known technique such as that disclosed in Japanese Patent Laid-Open No. 2011-191928, for example.

Step S101 is a process of registering image information included in a model image prepared in advance as model information. An image showing the mark 150 on the workpiece W is prepared as a model image. The model information includes an edge code image that associates the coordinate position of each edge point in the model image with an edge code that indicates the direction of change in density at the edge, and an edge code histogram that indicates the frequency of edge codes in the edge code image. is included.

Step S102 is a process of generating an edge code image and an edge code histogram from the input image (that is, the image output from the preceding processing item 32B).

Step S103 is a rough search process that uses an edge code histogram to identify a region (hereinafter also referred to as "candidate point") that may match the model image within the input image. Specifically, when the model image is superimposed on the region of interest of the input image, the degree of matching between the edge code histogram generated from the input image and the edge code histogram generated from the model image is determined by the threshold Th3. compared to When the degree of matching exceeds the threshold Th3, the position of the region of interest is extracted as a candidate point. By executing this process while shifting the position of the region of interest, one or more candidate points are extracted.

Step S104 marks each candidate point extracted in step S103 in the region of interest including the candidate point based on the degree of matching between the edge code image of the region of interest and the edge code image generated from the model image. This is a process of measuring the position of 150.

Step S104 may be performed in multiple stages. For example, in the first stage, the degree of matching is sequentially evaluated by moving the region of interest at preset intervals, and the position of the region of interest with the highest degree of matching is identified. After that, a plurality of regions of interest are set around the position with the highest degree of coincidence in the previous stage, and a position (coordinates) with the highest degree of coincidence is specified from among the plurality of attention regions. The interval between the plurality of attention areas is set smaller than the interval between the plurality of attention areas in the previous stage. In this way, in the input image, the position (measurement coordinates) with the highest degree of matching with the model image is measured.

In addition, in the final step, by interpolating the change characteristics of the degree of coincidence in the region of interest showing the highest degree of coincidence and the surrounding regions of interest, and specifying the peak position, the interval smaller than the preset movement interval ( The location of the region that matches the model image can be identified with high accuracy. Thereby, measurement coordinates can be calculated in units of subpixels smaller than one pixel.

Step S105 is an adjacent point removal process for deleting duplicates among the measurement results (measurement coordinates) obtained by the above-described process. In this proximity point removal process, one of the positions that are close to each other (that is, overlap) among the positions that match the model image identified in step S104 described above is removed. Step S106 is an output process for outputting the acquired measurement coordinates.

The model image, the amount of edge information (for example, the number of edge points) in steps S103 and S104, the threshold Th3, and the presence or absence of measurement in subpixel units are parameters used to execute the processing item 33B. These parameters are related to the measurement accuracy of the position of the mark 150 and the measurement time.

The image on which the processing items 31B and 32B have been executed is input to the processing item 33B. As described above, the shape, size, and pixel value of the edge of the mark 150 change by executing the processing item 32B. The degree of these changes depends on the values of the parameters used to execute the processing item 32B. The parameters used to execute the processing item 33B need to be appropriately set according to the state of the input image. Therefore, each time the value of the parameter used to execute the process item 32B is changed, it is necessary to reset the value of the parameter used to execute the process item 33B.

<E-2. Managing images for settings>
FIG. 19 is a diagram illustrating an example of a setting image prepared in operation example 2. The XY stage 4 is manually adjusted so that the mark 150 is located at the center of the image (coordinates (1000, 1000)). FIG. 19A shows a reference image 160 captured in this state (hereinafter referred to as "reference state"). An area 170 in which the mark 150 appears in the reference image 160 may be used as a model image.

Next, a setting image 22a (see FIG. 19(b)) is prepared in advance, which is captured when the XY stage 4 is moved by 1 cm in the X direction from the reference state. Similarly, the setting image 22b (see FIG. 19(c)) taken when the XY stage 4 was moved 1 cm in the -X direction from the reference state, and the setting image 22b taken when the XY stage 4 was moved 1 cm in the Y direction from the reference state. The setting image 22c captured when the XY stage 4 was moved by 1 cm from the reference state in the −Y direction (see FIG. 19(e)) are prepared in advance.

FIG. 20 is a diagram showing a screen when setting a correct value for a setting image in operation example 2. A screen 60A illustrated in FIG. 20 is displayed on the display device 6.

The screen 60A includes a button 61 for setting a temporary storage folder, similar to the screen 60 of operation example 1 (see FIG. 13). In response to the operation of button 61, processor 110 sets a temporary storage folder.

The screen 60A includes a tab 80 for managing the correct value for the setting image, similar to the screen 60 of the first operation example (see FIG. 13). FIG. 20 shows the screen 60A when the tab 80 is selected.

The screen 60A includes a button 62 for selecting the folder in which the setting image is saved, similar to the screen 60 of operation example 1 (see FIG. 13). In response to the operation of the button 62, the processor 110 displays a list 81 of setting images 22a to 22d (see FIG. 19) on the screen 60A.

The list 81 includes a file name and a mark position for each of the setting images 22a to 22d. The mark position indicates the correct value for each setting image.

The list 81 displays a cursor 81a for selecting one setting image. The screen 60A includes an area 64 for displaying images, similar to the screen 60 of Operation Example 1 (see FIG. 13). The setting image selected by the cursor 81a is displayed in the area 64. The user only has to input the mark position into the list 81 while checking the area 64. As described above, the setting images 22a, 22b, 22c, and 22d are images obtained by moving 1 cm from the reference state in the X direction, -X direction, Y direction, and -Y direction, respectively. The relationship between the actual amount of movement of the mark 150 and the number of 150 pixels of movement of the mark on the image is determined in advance by calibration. Therefore, the user only has to input the coordinates of the mark 150 appearing in each setting image into the list 81 according to the relationship. For example, if 1 cm of actual movement corresponds to 100 pixels, the mark positions in the setting images 22a, 22b, 22c, and 22d are (1100, 1000), (900, 1000), and (1000, 1100), respectively. ), (1000,900).

The screen 60A includes a button 67 for instructing the creation of the management data 23, similar to the screen 60 of the first operation example (see FIG. 13). In response to the operation of button 67, processor 110 creates management data 23 based on the input to list 81. The management data 23 according to operation example 2 indicates the mark position for each setting image.

<E-3. Parameter settings>
FIG. 21 is a diagram showing a screen when setting parameters in operation example 2. The screen 60A includes a tab 69 for setting parameters, similar to the screen 60 of operation example 1 (see FIG. 14). FIG. 21 shows the screen 60A when the tab 69 is selected.

The screen 60A includes a button 72 for starting generation of a processed image and generation of difference information, similar to the screen 60 of operation example 1 (see FIG. 14).

In response to the operation of the button 72, the processor 110 executes the image processing flow 30B on the four setting images 22a to 22d, and processes the four processes in which the processing items 31B and 32B up to the stage before the processing item 33B are executed. The completed images 25 are respectively generated. The processor 110 saves the generated four processed images 25 in a temporary storage folder.

When the four processed images 25 are saved in the temporary storage folder, the processor 110 executes the processing item 33B on each of the four processed images 25 and generates a processing result.

Specifically, the processor 110 executes steps S101 to S106 shown in FIG. 18, and outputs processing results (measured coordinates) for each of the four processed images 25.

The processor 110 generates difference information indicating the difference between the measurement coordinates for the four processed images 25 generated from the four setting images 22a to 22d and the correct value (mark position) indicated by the management data 23. . In operation example 2, the difference information indicates the maximum value of the Euclidean distance between the position indicated by the measurement coordinates and the mark position (hereinafter simply referred to as "error") for the four setting images 22a to 22d.

The processor 110 uses a plurality of parameters (model image, amount of edge information in steps S103 and S104 (for example, number of edge points), threshold Th3, presence or absence of measurement in subpixel units) used to execute the processing item 33B. Difference information is generated for each value of . The processor 110 displays on the screen 60A a table 82 in which each of a plurality of possible values of the parameter used to execute the processing item 33B is associated with the difference information. Preferably, the user selects the value with the smallest error. Therefore, it is preferable that the processor 110 sorts the possible values of the parameter in descending order of error.

A cursor 82a is displayed on the table 82. A user can select one record in table 82 by moving cursor 82a.

The screen 60A includes a button 74 for checking detailed information on the processing result, similar to the screen 60 of operation example 1 (see FIG. 14). In response to the operation of the button 74, the processor 110 displays detailed information on the processing result of the record selected by the cursor 82a on the display device 6.

FIG. 22 is a diagram illustrating an example of a window showing detailed information on processing results in operation example 2. Window 75A illustrated in FIG. 22 is displayed in response to operation of button 74 on screen 60A. Note that the window 75A may be displayed overlapping the screen 60A.

Window 75A includes areas 83, 84, and 85. In the area 83, the value of the parameter corresponding to the record selected by the cursor 82a is shown.

In FIG. 22, "model image path" indicates a folder or image file in which at least one image is saved. When the "model image path" indicates one image file, the "single image" is displayed in the "model image". When the "model image path" indicates a folder, the display of the "model images" is switched according to the number of images stored in the folder. The "model image" indicates a "single image" when the number of images stored in the folder is one, and indicates "average image" when the number of images stored in the folder is multiple. If the "model image" represents a "single image", one image specified by the "model image path" is set as the model image. If the "model image" represents an "average image", the average image of the plurality of images stored in the folder indicated by the "model image path" is set as the model image.

"Stability" indicates the amount of edge information (for example, the number of edge points) in the rough search process (step S103 in FIG. 18). "Accuracy" indicates the amount of edge information in the detailed search process (step S104 in FIG. 18). “Sub-pixel processing” indicates whether or not measurement is performed in sub-pixel units. "Candidate point level" indicates threshold Th3.

The area 84 shows the error in the value of the parameter corresponding to the record selected by the cursor 82a.

The area 85 shows details of the processing result for the value of the parameter corresponding to the record selected by the cursor 82a. Specifically, the area 85 displays the file name 85a of each of the four setting images 22a to 22d, the correct value coordinates 85b corresponding to each setting image, the measured coordinates 85c, and the coordinate difference 85d. be done. The correct value coordinates 85b indicate the position coordinates of the mark 150 in the setting image.

The window 75A includes a button 86 for displaying the positional relationship between the correct value coordinates 85b and the measurement coordinates 85c. In response to the operation of the button 86, the processor 110 causes the display device 6 to display a window showing the positional relationship between the correct value coordinates 85b and the measurement coordinates 85c for the four setting images 22a to 22d.

FIG. 23 shows a window displayed in response to the operation of button 86 shown in FIG. 22. The window 87 illustrated in FIG. 23 shows a graph of the image coordinate system in which a white circle indicating the correct value coordinate 85b and a black circle indicating the measurement coordinate 85c are plotted. By checking the window 87, the user can visually recognize the magnitude of the deviation between the correct value coordinates 85b and the measured coordinates 85c for the four setting images 22a to 22d.

Window 87 includes a button 88 for closing window 87. In response to the operation of button 88, processor 110 closes window 87 and displays window 75A shown in FIG. 22 on display device 6.

The window 75A illustrated in FIG. 22 includes a button 78 for closing the window 75A, similar to the window 75 of Operation Example 1 (see FIG. 15). In response to the operation of button 78, processor 110 closes window 75A and displays screen 60A shown in FIG. 21 on display device 6.

The user selects an appropriate record from the table 82 by checking the windows 75A, 87 while moving the cursor 82a of the table 82 shown in FIG.

As shown in FIG. 21, the screen 60A includes a button 79 for setting the value of the parameter used to execute the processing item 33B, similar to the screen 60 of the first operation example (see FIG. 14). In response to the operation of the button 79, the processor 110 sets the value of the parameter included in the record selected by the cursor 82a in the table 82 as the value to be used when executing the processing item 33B. That is, the processor 110 generates setting data 24 indicating the value of the parameter included in the record selected by the cursor 82a, and stores the generated setting data 24 in the storage unit 20.

As described above, a plurality of processed images 25 are automatically generated by executing processing items 31B to 32B on a plurality of setting images 22, and processing item 33B is automatically generated using a plurality of processed images 25. Parameter values used for execution are set. Thereby, the values of the parameters used to execute the processing item 33B can be appropriately set.

Note that the processing item 33C may be an item for detecting the mark 150, also taking into consideration the area of the mark 150. In this case, the processing items preceding the processing item 33B for which parameters are set include, for example, the above-mentioned labeling. Then, the parameters used to execute the processing item 33C are set using the plurality of processed images 25 on which labeling has been performed. In this way, the processing items up to the stage before the processing item 33C are not limited to those shown in FIG. 17.

<F. Operation example 3>
FIG. 24 is a diagram showing an image processing flow for which parameters are set in operation example 3. The image processing flow 30C illustrated in FIG. 24 includes processing items 31C to 36C.

The processing item 31C is an item for acquiring an image to be subjected to image processing. The processing item 32C is an item for detecting an area to be inspected from the image acquired by the processing item 31C. In operation example 3, a rectangular surface in a plan view (hereinafter referred to as a "printed surface") on which text is printed on the workpiece W is detected as an inspection target. In the processing item 32C, for example, similarly to the processing item 32B, the printed surface is detected by performing pattern matching using a moder image corresponding to the printed surface. Processing item 33C is an item for performing coordinate transformation of the image so that the printed surface is located at the center of the image.

The processing item 34C is an item for correcting distortion of the printed surface. If the normal direction of the printed surface and the optical axis direction of the camera 3 are misaligned, the printed surface in the image will be distorted into a trapezoid. The amount of distortion depends on the angle between the normal direction of the printed surface and the optical axis direction of the camera 3. By executing processing item 34C, such trapezoidal distortion is corrected.

FIG. 25 is a diagram showing a processing example of the processing item 34C shown in FIG. 24. FIG. 25 shows a printing surface 180 on the workpiece W. As shown in FIG. FIG. 25(a) shows the printed surface 180 before the processing item 34C is executed, and FIG. 25(b) shows the printed surface 180 after the processing item 34C is executed. As shown in FIG. 25, the trapezoidal distortion of the printing surface 180 is corrected by executing the processing item 34C. The degree of correction of trapezoidal distortion depends on the value of the parameter used to execute the processing item 34C.

The processing item 35C is an item for performing character recognition on the image for which the processing item 34C has been executed. The processing item 36C is an item that outputs the text recognized by executing the processing item 35C.

Hereinafter, an example of the operation of the image processing apparatus 1 when receiving an instruction to set parameters used to execute the processing item 35C in the image processing flow 30C will be described.

<F-1. Processing items for which parameters are set>
In the processing item 35C, character recognition is performed on the input image (ie, the image on which the previous processing items 31C to 34C have been executed) using a known technique. If the character width of the text to be recognized is not limited, the time required to execute process item 35C will be longer. Therefore, it is preferable to set the maximum character width that the text to be recognized can have, and to recognize only the text that is less than or equal to the maximum character width. Thereby, the time required to execute the processing item 35C can be shortened. This maximum character width is a parameter used to execute process item 35C.

An image on which the processing item 34C has been executed is input to the processing item 35C. As described above, the trapezoidal distortion of the printing surface 180 is corrected by executing the processing item 34C. However, the degree of correction depends on the value of the parameter used to execute the processing item 34C. Depending on the degree of correction of trapezoidal distortion, the maximum character width that can be taken by the text to be recognized may change. Therefore, each time the value of the parameter used to execute the process item 34C is changed, it is necessary to reset the value of the parameter used to execute the process item 35C.

<F-2. Managing images for settings>
FIG. 26 is a diagram showing a screen when setting a correct value for a setting image in operation example 3. A screen 60B illustrated in FIG. 26 is displayed on the display device 6.

Screen 60B includes a button 61 for setting a temporary storage folder, similar to screen 60 of Operation Example 1 (see FIG. 13). In response to the operation of button 61, processor 110 sets a temporary storage folder.

The screen 60B includes a tab 80 for managing the correct value for the setting image, similar to the screen 60 of the first operation example (see FIG. 13). FIG. 26 shows the screen 60A when the tab 80 is selected.

Screen 60B includes a button 62 for selecting a folder in which a plurality of setting images 22 are saved, similar to screen 60 of Operation Example 1 (see FIG. 13). In response to the operation of the button 62, the processor 110 displays a list 91 of the plurality of setting images 22 on the screen 60B.

The list 91 includes, for each of the plurality of setting images 22, the file name and the text printed on the print surface 180 (see FIG. 25) that appears in the setting image 22. The text is the correct value for each setting image.

A cursor 91a for selecting one setting image 22 is displayed in the list 91. Screen 60B includes an area 64 for displaying images, similar to screen 60 of Operation Example 1 (see FIG. 13). In the area 64, the setting image 22 selected by the cursor 91a is displayed. The user only needs to input text into the list 91 while checking the area 64.

Screen 60B includes a button 67 for instructing creation of management data 23, similar to screen 60 of operation example 1 (see FIG. 13). In response to the operation of button 67, processor 110 creates management data 23 based on the input to list 91. The management data 23 according to operation example 3 indicates, for each setting image 22, the text on the print surface 180 that appears in the setting image 22.

<F-3. Parameter settings>
FIG. 27 is a diagram showing a screen when setting parameters in operation example 3. Screen 60B includes a tab 69 for setting parameters, similar to screen 60 of operation example 1 (see FIG. 14). FIG. 27 shows a screen 60B when tab 69 is selected.

Screen 60B includes a button 72 for starting generation of a processed image and generation of difference information, similar to screen 60 of Operation Example 1 (see FIG. 14).

In response to the operation of the button 72, the processor 110 executes the image processing flow on the plurality of setting images 22, and generates a plurality of processed images in which the processing items 31C to 34C up to the stage before the processing item 35C have been respectively executed. Generate 25. The processor 110 saves the generated plurality of processed images 25 in a temporary storage folder set according to the operation of the button 61.

When the plurality of processed images 25 are saved in the temporary storage folder, the processor 110 executes the processing item 35C on each of the plurality of processed images 25 and generates a processing result.

Specifically, processor 110 performs character recognition on each of the plurality of processed images 25, and generates recognized text as a processing result.

For each of the plurality of setting images 22, the processor 110 generates a difference indicating the difference between the text recognized from the processed image 25 generated from the setting image and the correct value (text) indicated by the management data 23. Generate information. In operation example 3, the difference information includes the number of setting images 22 whose processing results and correct values match among the plurality of setting images 22 (hereinafter also referred to as "number of correct answers"), and the processing results and correct answers. The number of setting images 22 whose values do not match (hereinafter also referred to as "number of incorrect answers") is shown.

Processor 110 generates difference information for each of a plurality of possible values of the parameter (maximum character width) used to execute process item 35C. The processor 110 identifies the minimum value for which the number of incorrect answers is 0 among the multiple values that the parameter (maximum character width) can take, and displays optimal information 94 in which the identified value is associated with the difference information on the screen 60B. to be displayed.

The screen 60B includes a button 74 for checking detailed information on the processing result, similar to the screen 60 of the first operation example (see FIG. 14). In response to the operation of the button 74, the processor 110 displays detailed information on the processing result for the value of the parameter (maximum character width) indicated by the optimum information 94 on the display device 6.

FIG. 28 is a diagram showing an example of a window showing detailed information on processing results in operation example 3. Window 75B illustrated in FIG. 28 is displayed in response to operation of button 74 on screen 60B. Note that the window 75B may be displayed overlapping the screen 60B.

Window 75B includes areas 95, 96, and 97. In the area 95, the value of the parameter (maximum character width) indicated by the optimum information 94 is displayed. In the area 96, the number of correct answers and the number of incorrect answers indicated by the optimal information 94 are displayed.

In the area 97, a file name 97a of each of the plurality of setting images 22, a correct character 97b corresponding to each setting image, a recognized character 97c, and a determination result 97d are displayed.

Window 75B includes a button 78 for closing window 75B, similar to window 75 of Operation Example 1 (see FIG. 15). In response to the operation of button 78, processor 110 closes window 75B and displays screen 60B shown in FIG. 27 on display device 6.

As shown in FIG. 27, the screen 60B includes a button 79 for setting the value of the parameter used to execute the processing item 35C, similar to the screen 60 of the operation example 1 (see FIG. 14). In response to the operation of the button 79, the processor 110 sets the value of the parameter (maximum character width) indicated by the optimum information 94 as the value to be used when executing the processing item 35C. That is, the processor 110 generates setting data 24 indicating the value of the parameter (maximum character width) indicated by the optimum information 94, and stores the generated setting data 24 in the storage unit 20.

Although operation examples 1 to 3 for a relatively simple image processing flow have been described, the image processing flow may include a plurality of processing items appropriately selected from various processing items included in the processing item group. Regardless of the type of processing item that makes up the image processing flow, the value of the parameter used to execute one processing item that makes up the image processing flow is the same as the value of the parameter used to execute the processing item that makes up the image processing flow The settings are made using the completed image 25. For example, if the processing items for which parameters are set include the above-mentioned 3D imaging, geometric transformation, and semantic segmentation, multiple processed items that have undergone 3D imaging, geometric transformation, and semantic segmentation The image 25 is used to set the values of parameters used to execute the processing item.

<G. Modified example>
The processing results generated by executing processing items on the plurality of processed images 25 are not limited to the presence or absence of defects, measured coordinates, and recognized text. The processing result may be, for example, image data. Then, image data (hereinafter referred to as "correct image data") may be set as the correct value for each of the plurality of setting images 22. In this case, for example, a sum total of image differences may be generated as difference information indicating the difference between the processing results for the plurality of processed images 25 and the correct values set for the plurality of setting images 22, respectively. The sum of image differences is also called norm, and is the sum of brightness differences for each pixel. Alternatively, the number of pixels with a luminance difference greater than a predetermined value may be generated as the difference information. The processor 110 optimizes the parameters so that the difference indicated by the difference information becomes small.

In the above description, it is assumed that the correct value is set for the setting image 22. However, the correct value may not be set for the setting image 22. For example, the processor 110 operating as the setting unit 14 may optimize the values of the parameters so that the feature amount obtained from the plurality of processing results approaches a preset reference value.

For example, a plurality of setting images 22 showing the workpiece W arranged so that the mark 150 is in a fixed position are prepared in advance. Then, as in the second operation example, parameters used to execute the processing item 33B for detecting the position of the mark 150 are set. The processor 110 executes the processing item 33B on each of the plurality of processed images 25 obtained by executing the image processing flow on the plurality of setting images 22, and generates a processing result (measurement coordinates). do. The processor 110 may optimize the parameters of the processing item 33B so that the variance of the plurality of measurement coordinates generated from the plurality of processed images 25 approaches zero. That is, the processor 110 may select the value whose variance is closest to 0 from among the plurality of values that the parameter can take, and set the selected value as the value of the parameter.

Alternatively, a plurality of setting images 22 may be prepared by capturing images while moving the workpiece W at regular intervals. Then, as in the second operation example, parameters used to execute the processing item 33B for detecting the position of the workpiece W are set. The processor 110 executes the processing item 33B on each of the plurality of processed images 25 obtained by executing the image processing flow on the plurality of setting images 22, and generates a processing result (measurement coordinates). do. The processor 110 may optimize the parameters of the processing item 33B so that the index value (for example, variance) indicating the dispersion of the intervals between the plurality of measurement coordinates generated from the plurality of processed images 25 approaches zero. That is, the processor 110 may select the value with the smallest variation from among a plurality of values that the parameter can take, and set the selected value as the value of the parameter.

<H. Action/Effect＞
As described above, the image processing apparatus 1 according to the present embodiment executes an image processing flow in which a plurality of processing items are arranged in the order of execution. The image processing device 1 includes a generation section 13 and a setting section 14. The generation unit 13 executes part or all of the image processing flow on the plurality of setting images 22 in response to input of an instruction to set parameters used to execute a specified processing item among the plurality of processing items. By doing so, a plurality of processed images 25 are respectively generated in which the processing items up to the stage preceding the designated processing item have been executed. The setting unit 14 uses the plurality of processed images 25 to set parameter values.

According to the above configuration, in response to input of a parameter setting instruction, a plurality of processed images 25 are automatically generated in which processing items up to a specified processing item have been executed. Further, parameter values are set using the plurality of generated processed images 25. This eliminates the need to create an image collection flow 40 as illustrated in FIG. 2 in order to generate a plurality of processed images 25, and image processing including a branch point 51 as illustrated in FIG. There is no need to create the flow 50. As a result, the effort required to collect a plurality of processed images 25 is reduced.

Furthermore, even if you change the value of the parameter used to execute the previous processing item in the image processing flow, or change the previous processing item to another processing item, the specified processing item will be executed. A plurality of processed images 25 are automatically collected by inputting instructions for setting parameters used in the process. Therefore, the user does not need to perform a separate operation to collect the plurality of processed images 25.

In this way, according to the present embodiment, it is possible to reduce the effort required to set parameters used to execute processing items that constitute an image processing flow.

The setting unit 14 acquires a plurality of processing results by executing specified processing items on the plurality of processed images 25 using each of the plurality of values that the parameter can take. The setting unit 14 sets a value selected from a plurality of values based on a plurality of processing results as a value of a parameter.

According to the above configuration, an appropriate value is selected from a plurality of possible values of the parameter while referring to a plurality of processing results.

The setting unit 14 generates difference information indicating the difference between the plurality of processing results and the plurality of correct values set in advance for the plurality of setting images 22, respectively. The setting unit 14 causes the display device 6 to display the correspondence between each of the plurality of values and the difference information. The setting unit 14 may set a value for which a selection instruction has been received from among a plurality of values as the value of the parameter.

According to the above configuration, the user can select the optimal value while viewing the correspondence between each of the plurality of values and the difference information displayed on the display device 6.

Alternatively, the setting unit 14 may set a plurality of processing results each obtained by executing a specified processing item on a plurality of processed images 25 to a plurality of processing results set in advance for each of the plurality of setting images 22. The parameter values may be optimized so that they approach the correct value.

According to the above configuration, the value of the parameter is automatically set so that the processing result of the specified processing item approaches the correct value. This can further reduce the effort required to set parameters used to execute the processing items that make up the image processing flow.

The setting unit 14 may optimize the parameter values so that the feature amounts obtained from the plurality of processing results approach a preset reference value. According to the above configuration, the value of the parameter is automatically set so that the feature amount obtained from the plurality of processing results approaches the reference value. This can further reduce the effort required to set parameters used to execute the processing items that make up the image processing flow.

The workpiece W is shown in each of the plurality of setting images 22. The specified processing item is, for example, an item for determining the attributes of the workpiece W (for example, the type and quality of the workpiece W) appearing in the image output from the preceding processing item. The plurality of correct values each indicate the attributes of the workpiece W shown in the plurality of setting images 22.

For example, the designated processing item is an item for determining the attributes of the workpiece W based on the comparison result between the feature amount calculated from the image output from the previous processing item and the value of the parameter.

According to the above configuration, it is possible to easily set the parameters used to execute the processing item for determining the attributes of the workpiece W.

The specified processing item may be an item for determining the attributes of the workpiece W using a learned model 140 generated by performing machine learning using a plurality of processed images 25. The above parameters then define the trained model 140.

According to the above configuration, the trained model 140 suitable for determining the attributes of the workpiece W can be easily generated.

Each of the plurality of setting images 22 shows a workpiece W including a characteristic portion (for example, a mark 150). The specified processing item is an item for detecting the position of the characteristic portion appearing in the image output from the previous processing item using a model image in which the characteristic portion appears. The plurality of correct values each indicate the coordinates (correct value coordinates) of the characteristic portion appearing in the plurality of setting images 22.

According to the above configuration, it is possible to easily set the parameters used to execute the processing item for detecting the position of the characteristic portion.

Text appears in each of the plurality of setting images 22. The specified processing item is an item that performs character recognition of the text appearing in the image output from the previous processing item. The plurality of correct answer values each indicate the text appearing in the plurality of setting images 22.

According to the above configuration, it is possible to easily set parameters used to execute a processing item for character recognition.

§3 Supplementary notes As described above, this embodiment includes the following disclosures.

(Configuration 1)
An image processing apparatus (1) that executes an image processing flow (30) in which a plurality of processing items (31 to 36) are arranged in an execution order,
The image processing flow (30) is applied to a plurality of setting images (22) in response to an input of a parameter setting instruction used for executing a specified processing item among the plurality of processing items (31 to 36). a generating unit (13, 110) that generates a plurality of processed images (25) in which processing items up to the preceding stage of the specified processing item have been executed by executing part or all of the processing item;
An image processing device (1) comprising: a setting section (14, 110) that sets the value of the parameter using the plurality of processed images (25).

(Configuration 2)
The setting section (14, 110) includes:
For each of the plurality of values that the parameter can take, each of the plurality of processing results is obtained by executing the specified processing item on the plurality of processed images (25) using the value,
The image processing device (1) according to configuration 1, wherein a value selected from the plurality of values based on the plurality of processing results is set as the value of the parameter.

(Configuration 3)
The setting section (14, 110) includes:
generating difference information indicating a difference between the plurality of processing results and a plurality of correct answer values respectively set in advance for the plurality of setting images (22);
displaying a correspondence relationship between each of the plurality of values and the difference information on a display device;
The image processing device (1) according to configuration 2, wherein a value for which a selection instruction has been received from among the plurality of values is set as the value of the parameter.

(Configuration 4)
The setting unit (14, 110) is configured to optimize the values of the parameters so that the plurality of processing results approach a plurality of correct values respectively set in advance for the plurality of setting images. The image processing device (1) according to 2.

(Configuration 5)
The image processing apparatus according to configuration 2, wherein the setting unit (14, 110) optimizes the value of the parameter so that the feature amount obtained from the plurality of processing results approaches a preset reference value. 1).

(Configuration 6)
A target object (W) is reflected in each of the plurality of setting images (22),
The specified processing item is an item for determining the attribute of the object (W) appearing in the image output from the previous processing item,
The image processing device (1) according to configuration 3 or 4, wherein the plurality of correct answer values each indicate an attribute of the object (W) shown in the plurality of setting images (22).

(Configuration 7)
Configuration 6, wherein the designated processing item is an item for determining an attribute of the object based on a comparison result between a feature amount calculated from an image output from a previous processing item and a value of the parameter. The image processing device (1) described in (1).

(Configuration 8)
The specified processing item is an item for determining attributes of the object (W) using a learned model (140) generated by performing machine learning using the plurality of processed images (25). and
The image processing device (1) according to configuration 6, wherein the parameters define the learned model (140).

(Configuration 9)
Each of the plurality of setting images (22) shows an object (W) including a characteristic part (150),
The specified processing item is an item for detecting the position of the characteristic portion (150) in the image output from the previous processing item using a model image in which the characteristic portion (150) is reflected;
The image processing device (1) according to configuration 3 or 4, wherein the plurality of correct answer values each indicate the coordinates of the characteristic portion (150) shown in the plurality of setting images (22).

(Configuration 10)
Text appears in each of the plurality of setting images (22),
The specified processing item is an item for character recognition of the text appearing in the image output from the previous processing item,
The image processing device (1) according to configuration 3 or 4, wherein the plurality of correct answer values each indicate the text appearing in the plurality of setting images (22).

(Configuration 11)
A method for controlling an image processing device (1) that executes an image processing flow (30) in which a plurality of processing items (31 to 36) are arranged in an execution order, the method comprising:
The image processing flow is applied to the plurality of setting images (22) in response to the input of an instruction to set the value of a parameter used to execute a specified processing item among the plurality of processing items (31 to 36). generating each of a plurality of processed images (25) in which processing items up to the stage preceding the specified processing item have been executed by executing some or all of them;
and setting a value of the parameter using the plurality of processed images.

(Configuration 12)
A program (121) that causes a computer to execute the control method described in Configuration 11.

Although the embodiments of the present invention have been described, the embodiments disclosed this time should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and it is intended that all changes within the meaning and range equivalent to the claims are included.

1 Image processing device, 2 External storage, 3 Camera, 4 XY stage, 6 Display device, 7 Input device, 8 Defect, 11 Flow creation section, 12 Management section, 13 Generation section, 14 Setting section, 16 Flow execution section, 20 Storage unit, 21 Flow data, 22, 22a to 22d Setting image, 23 Management data, 24 Setting data, 25 Processed image, 30, 30A to 30C, 50 Image processing flow, 31 to 36, 31A to 31C, 32A to 32C, 33A to 33C, 34A to 34C, 35A to 35C, 36A to 36C, 37A to 39A, 41 Processing item, 40 Image collection flow, 51 Branch point, 60, 60A, 60B Screen, 61, 62, 67, 68 , 70, 72, 74, 78, 79, 86, 88 Button, 63, 81, 91 List, 63a, 73a, 81a, 82a, 91a Cursor, 64, 76, 77, 83, 84, 85, 95-97, 170 area, 64a frame line, 65 icon group, 66 button group, 69, 71, 80 tab, 73, 82 table, 75, 75A, 75B, 87 window, 94 optimal information, 106 memory card, 110 processor, 112 RAM, 114 display controller, 116 system controller, 118 I/O controller, 120 hard disk, 121 control program, 122 camera interface, 124 input interface, 126 PLC interface, 128 communication interface, 130 memory card interface, 140 learned model, 150 mark, 160 reference image, 180 printing surface, SYS image processing system, W work.

Claims (12)

An image processing apparatus that executes an image processing flow in which a plurality of processing items are arranged in an execution order,
By executing part or all of the image processing flow on a plurality of setting images in response to an input of a parameter setting instruction used for executing a specified processing item among the plurality of processing items, a generation unit that generates each of a plurality of processed images in which processing items up to the stage preceding the specified processing item are executed;
An image processing apparatus, comprising: a setting section that sets values of the parameters using the plurality of processed images. The setting section includes:
For each of the plurality of values that the parameter can take, each of the plurality of processing results is obtained by executing the specified processing item on the plurality of processed images using the value,
The image processing apparatus according to claim 1, wherein a value selected from the plurality of values based on the plurality of processing results is set as the value of the parameter. The setting section includes:
generating difference information indicating a difference between the plurality of processing results and a plurality of correct answer values respectively set in advance for the plurality of setting images;
displaying a correspondence relationship between each of the plurality of values and the difference information on a display device;
3. The image processing apparatus according to claim 2, wherein a value selected from among the plurality of values is set as the value of the parameter. The setting unit optimizes the values of the parameters so that the plurality of processing results approach a plurality of correct values respectively set in advance for the plurality of setting images. Image processing device. The image processing apparatus according to claim 2, wherein the setting unit optimizes the value of the parameter so that the feature amount obtained from the plurality of processing results approaches a preset reference value. Each of the plurality of setting images includes a target object;
The specified processing item is an item for determining the attribute of the object appearing in the image output from the previous processing item,
The image processing device according to claim 3 or 4, wherein the plurality of correct values each indicate an attribute of the object appearing in the plurality of setting images. The specified processing item is an item for determining an attribute of the object based on a comparison result between a feature amount calculated from an image output from a previous processing item and a value of the parameter. 6. The image processing device according to 6. The specified processing item is an item for determining an attribute of the object using a learned model generated by performing machine learning using the plurality of processed images,
The image processing device according to claim 6, wherein the parameters define the learned model. Each of the plurality of setting images depicts an object including a characteristic part,
The specified processing item is an item for detecting the position of the characteristic portion appearing in the image output from the previous processing item using a model image in which the characteristic portion is reflected;
5. The image processing apparatus according to claim 3, wherein the plurality of correct values each indicate coordinates of the characteristic portion shown in the plurality of setting images. Each of the plurality of setting images includes text;
The specified processing item is an item for character recognition of the text appearing in the image output from the previous processing item,
The image processing apparatus according to claim 3 , wherein the plurality of correct answer values each indicate the text appearing in the plurality of setting images. A method for controlling an image processing apparatus that executes an image processing flow in which a plurality of processing items are arranged in an execution order, the method comprising:
Executing part or all of the image processing flow on the plurality of setting images in response to input of a setting instruction for a value of a parameter used to execute a specified processing item among the plurality of processing items. generating a plurality of processed images in which processing items up to the stage preceding the specified processing item have been executed, respectively;
and setting a value of the parameter using the plurality of processed images. A program that causes a computer to execute the control method according to claim 11.

Images