CN112088395A - Image processing apparatus, image processing method, and image processing program - Google Patents
Image processing apparatus, image processing method, and image processing program Download PDFInfo
- Publication number
- CN112088395A CN112088395A CN201980030801.3A CN201980030801A CN112088395A CN 112088395 A CN112088395 A CN 112088395A CN 201980030801 A CN201980030801 A CN 201980030801A CN 112088395 A CN112088395 A CN 112088395A
- Authority
- CN
- China
- Prior art keywords
- image
- corrected
- good
- processing apparatus
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 153
- 238000003672 processing method Methods 0.000 title claims abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 63
- 230000007547 defect Effects 0.000 claims description 37
- 230000002950 deficient Effects 0.000 claims description 21
- 238000012937 correction Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 6
- 238000007689 inspection Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000010422 painting Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 239000003973 paint Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
Abstract
The invention provides an image processing apparatus, an image processing method and an image processing program capable of increasing good-quality images compared with the situation of simply accumulating good-quality images and defective-quality images. The image processing apparatus includes: a camera for capturing an image of an object; and a controller that executes a process flow including a plurality of steps, the controller having: an input unit that receives settings of a sequence of a plurality of steps; and an execution unit that sequentially executes a plurality of steps, the plurality of steps including an image editing step, the image editing step including: a first process of generating a corrected image in which an image is corrected; and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
Description
Technical Field
The present invention relates to an image processing apparatus, an image processing method, and an image processing program.
Background
Conventionally, an image processing apparatus is used which takes an image of a workpiece conveyed on a production line and determines whether the workpiece is a good product or a defective product based on the image. In addition to the determination of the quality of images, various functions are added to such an image processing apparatus. In addition, in order to select a function to be used by a user among a plurality of functions, a technique of configuring a process flow by arbitrarily combining a plurality of functions has been developed.
For example, an image processing apparatus described in patent document 1 below displays on a screen items corresponding to respective steps of imaging, extracting, measuring, and quality determination based on measurement results of a measurement target object. Among these items, an item to be used is selected on the screen, and the execution order of the selected item is set, thereby configuring a desired processing flow.
Documents of the prior art
Patent document
Patent document 1: japanese patent No. 4784269
Disclosure of Invention
Problems to be solved by the invention
In recent years, studies have been made to determine whether a workpiece is good or defective based on an image by using a determination model such as a Convolutional Neural Network (CNN). Here, since the determination model such as CNN is generated by teaching learning, it is necessary to prepare as many learning images as possible including images of workpieces classified as good products and images of workpieces classified as defective products.
However, in the initial stage of starting collection of images for learning, not only defective images but also good-quality images may not be sufficiently prepared. Therefore, even if the learning with teaching is performed using the determination model of the learning image, sufficient determination accuracy may not be obtained.
Accordingly, the present invention provides an image processing apparatus, an image processing method, and an image processing program capable of increasing the number of good-quality images as compared with a case where a good-quality image and a defective-quality image are simply accumulated.
Means for solving the problems
An image processing apparatus according to an aspect of the present invention includes: a camera for capturing an image of an object; and a controller that executes a process flow including a plurality of steps, the controller having: an input unit that receives settings of a sequence of a plurality of steps; and an execution unit that sequentially executes a plurality of steps, the plurality of steps including an image editing step, the image editing step including: a first process of generating a corrected image in which an image is corrected; and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
According to this aspect, the corrected image obtained by correcting the image of the object is generated in the image editing step, and the corrected image is added to the learning data set as the good product image, whereby the good product image can be increased as compared with a case where the good product image and the defective product image are simply accumulated.
In the above aspect, the first process may be executed when the result of the determination of the quality of the object based on the image by the determination model is a failure and the object includes a defect.
According to this aspect, the corrected image is generated when the determination model determines that the object is defective and the object actually includes a defect, and the good-product image can be increased.
In the above aspect, the image editing step may further include a third process of adding the image as a good-quality image to the learning data set when the result of the determination of the quality of the object based on the image by the determination model is a failure and the object does not have a defect.
According to this aspect, when it is determined by the determination model that there is a defect and the object does not substantially contain a defect, the original image can be added to the data set for learning as a good-quality image, and unnecessary execution of the image editing step can be omitted.
In the above aspect, the first process may be executed when the input unit receives a result of the determination of the acceptability of the object based on the image and the result is a defect.
According to this aspect, the corrected image is generated when the inspection person has determined that the inspection person has performed a good or bad judgment and the good product images can be increased.
In the above aspect, the second processing may include processing for adding the corrected image as the good-quality image to the learning data set when the result of the determination of the object as to whether or not the object is good is determined by the determination model based on the corrected image.
According to this aspect, when the corrected image is determined by the determination model and the determination result is a failure, the correction is performed again and the determination result is a good, the corrected image can be added to the learning data set as a good image, and an appropriate good image can be added.
In the scheme, the first processing may include a process of generating a modified image from the image processing model.
According to this aspect, the corrected image is automatically generated by the image processing model, and thus the cost for increasing the number of good-quality images can be reduced.
In the above aspect, the first processing may include processing for generating a corrected image based on the correction of the image received by the input unit.
According to this aspect, the image can be corrected by the inspector to generate a corrected image, thereby increasing the number of suitable good products.
In the above-described aspect, the second processing may include processing for adding the corrected image to the learning dataset while associating information indicating that the corrected image has been corrected.
According to this aspect, it is possible to specify which of the good product images included in the learning dataset is the corrected image, and the quality of the learning dataset can be easily confirmed.
In this case, the second processing may include processing for adding the corrected image to the learning data set in association with the image before correction of the corrected image.
According to this aspect, the image that is the generation source of the corrected image can be easily referred to, and whether or not the first process has been performed appropriately can be evaluated afterward.
In the above-described aspect, the input unit may receive a selection of whether to omit execution of the image editing step, and the execution unit may execute the plurality of steps in order by omitting execution of the image editing step according to the selection.
According to this configuration, the image editing step can be omitted and the processing flow can be executed, so that the operation for collecting the data set for learning and the operation during the actual operation of the image processing apparatus can be easily switched.
An image processing method according to another aspect of the present invention includes: receiving a setting of a sequence of a plurality of steps included in a process flow executed by a controller provided in an image processing apparatus; and executing the plurality of steps in sequence, wherein the plurality of steps comprises an image editing step comprising: a first process of generating a corrected image in which an image of an object is corrected; and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
According to this aspect, the corrected image obtained by correcting the image of the object is generated in the image editing step, and the corrected image is added to the learning data set as the good product image, whereby the good product image can be increased as compared with a case where the good product image and the defective product image are simply accumulated.
An image processing program according to another aspect of the present invention is an image processing program for causing a computer that receives a setting of an order of a plurality of steps included in a processing flow to function as an execution unit that executes the plurality of steps in the order, wherein the plurality of steps include an image editing step including: a first process of generating a corrected image in which an image of an object is corrected; and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
According to this aspect, the corrected image obtained by correcting the image of the object is generated in the image editing step, and the corrected image is added to the learning data set as the good product image, whereby the good product image can be increased as compared with a case where the good product image and the defective product image are simply accumulated.
ADVANTAGEOUS EFFECTS OF INVENTION
According to the present invention, it is possible to provide an image processing apparatus, an image processing method, and an image processing program capable of increasing the number of good-quality images as compared with a case where a good-quality image and a defective-quality image are simply accumulated.
Drawings
Fig. 1 is a diagram showing an example of an arrangement of an image processing apparatus according to an embodiment of the present invention.
Fig. 2 is a block diagram of the image processing apparatus according to the present embodiment.
Fig. 3 is an example of a process flow editing screen of the image processing apparatus according to the present embodiment.
Fig. 4 is an example of a setting screen of the image processing apparatus according to the present embodiment.
Fig. 5 is an example of an image captured by the image processing apparatus of the present embodiment.
Fig. 6 is an example of a corrected image generated by the image processing apparatus of the present embodiment.
Fig. 7 is a flowchart of a first example of the addition processing of the learning dataset executed by the image processing apparatus of the present embodiment.
Fig. 8 is a flowchart of a second example of the addition processing of the learning dataset executed by the image processing apparatus of the present embodiment.
Fig. 9 is a flowchart of a third example of the addition processing of the learning dataset executed by the image processing apparatus of the present embodiment.
Fig. 10 is an example of a process flow editing screen of the image processing apparatus according to the present embodiment.
Detailed Description
Hereinafter, an embodiment (hereinafter, referred to as "the present embodiment") according to one aspect of the present invention will be described with reference to the drawings. In the drawings, the same or similar structures are denoted by the same reference numerals.
[ structural example ]
An example of the configuration of the image processing apparatus 10 according to the present embodiment will be described with reference to fig. 1 and 2. Fig. 1 is a diagram showing an example of installation of an image processing apparatus 10 according to the present embodiment. The image processing apparatus 10 includes a camera 1, a controller 2, a monitor 3, and a console (console) 4.
The camera 1 captures an image of an object. The camera 1 can capture an image of the workpiece W conveyed on the line L, and the image may be a color image or a monochrome image. The workpiece W is an example of an object, but the object may be a part 7 (a mark, a character, a defect, or the like) of the workpiece W. The number of cameras 1 is not limited to one, and a plurality of cameras may be provided.
The controller 2 executes a process flow including a plurality of steps. The controller 2 can analyze the position, posture, size, and the like of the object by processing the image captured by the camera 1, and can determine whether the object is good or defective. Here, the determination of the quality of the object may be performed by a determination model generated by learning with a teaching. The decision model may comprise CNN, for example. However, the determination model may include any mechanical learning model. The process flow executed by the controller 2 includes an image editing step described later, which may include a process for collecting a data set for learning used for the judgment of the presence of teaching learning of the model.
The monitor 3 may display an image captured by the camera 1, or an analysis result obtained by the display controller 2, or an edit screen of the processing flow. The monitor 3 may include, for example, a liquid crystal display device.
The console 4 is an input unit of the controller 2, and may include a mouse, a keyboard, a touch panel, and the like. However, the form of the console 4 is not limited, and may be any form. The console 4 corresponds to an input unit that receives settings of the order of a plurality of steps executed by the controller 2.
In addition, the image processing apparatus 10 can be generally used for what is called a defect inspection, and in this case, the image processing apparatus 10 can determine the state of the workpiece W. Here, the state of the workpiece W may be the presence or absence of a defect or the type of a defect.
Fig. 2 is a block diagram of the image processing apparatus 10 according to the present embodiment. The controller 2 of the image Processing apparatus 10 includes a Central Processing Unit (CPU) 21, a flash Memory (flash Memory)22, a Random Access Memory (RAM) 23, a graphic controller 24, a camera Interface (Interface, I/F)25, an input I/F26, and an external I/F27.
The CPU 21 performs control, data calculation, and processing related to execution of the image processing program stored in the flash memory 22 and the RAM 23. The CPU 21 corresponds to an execution unit that executes a plurality of steps in a set order. The CPU 21 receives image data from the video camera 1 via the camera I/F25, receives data from the console 4 via the input I/F26, receives data from an external device via the external I/F, and displays the operation result on the monitor 3 via the graphic controller 24 or stores the operation result in the flash memory 22 or the RAM 23.
The flash memory 22 may store an image processing program or parameters required for processing, and the like. The RAM 23 can at least temporarily store image data captured by the camera 1, or an operation result or data in the middle of an operation obtained by the CPU 21.
The camera I/F25 may include a drive circuit of the camera 1, a buffer for receiving each image signal of R, G, B, an Analog/Digital (a/D) conversion circuit (not shown), and the like. The input I/F26 receives a signal from the console 4 and transmits the signal to the CPU 21. The external I/F27 may be an interface for communicating with an external device (not shown) (a personal computer (personal computer), a Programmable Logic Controller (PLC), or the like). The external I/F27 may be an interface for communicating with another image processing apparatus via a communication Network such as the Internet (Internet) or a Local Area Network (LAN). The graphics controller 24 may perform display control of the monitor 3.
The workpiece detection sensor 5 is provided on the line L and detects the arrival of the workpiece W. The workpiece detection sensor 5 may include a photoelectric sensor or a proximity sensor. Upon receiving a signal indicating that the workpiece W has been detected from the workpiece detection sensor 5, the CPU 21 drives the camera 1 using the camera I/F25 to photograph the workpiece W. When the image of the workpiece W is generated and stored in the RAM 23, the CPU 21 may execute a process of extracting or measuring a part 7 of the workpiece W captured in the image based on the image processing program stored in the flash memory 22, and determine whether the object is good or bad based on the measurement result. Further, the CPU 21 may save the image representing the processing result or the data of the determination result into the flash memory 22 or the RAM 23.
The controller 2 may be a device developed exclusively for the image processing apparatus 10, or may be a general-purpose computer on which an image processing program is installed. In this case, the image processing program may be supplied to the computer via a storage medium such as a Compact Disc-Read Only Memory (CD-ROM) or a communication line.
Fig. 3 is an example of a process flow editing screen of the image processing apparatus according to the present embodiment. The process flow editing screen is a screen for editing the process flow executed by the controller 2.
In the process flow editing screen of this example, the first step S1, the second step S2, and the third step S3 are set as the process flow. The first step S1 is a step described as "0 camera image input", and is a step of inputting the image of the object captured by the camera 1 to the controller 2. The first step S1 may be a step of inputting an image to the controller 2 in real time each time the camera 1 captures an image of the object transported on the line L, or may be a step of sequentially reading images of the object captured in advance by the camera 1 and stored in the RAM 23 or the like and inputting the images to the controller 2. The second step S2 is a step described as "1. image painting", and corresponds to an image editing step. The third step S3 is a step described as "2. exact matching" (step) for detecting different points with high accuracy by superimposing the registered good-product image on the image input in the first step S1. In this way, the plurality of steps included in the processing flow include an image editing step (second step S2).
The image editing step (second step S2) may include: a first process of generating a corrected image in which the image input in the first step S1 is corrected; and a second process of adding the corrected image as a good-quality image to a learning dataset used for generating a determination model for determining whether or not the object is good. By generating a corrected image in which the image of the object is corrected in the image editing step and adding the corrected image to the learning data set as a good-product image, the good-product image can be increased as compared with a case where a good-product image and a defective-product image are simply accumulated.
The first process may include a process of generating a corrected image based on the correction of the image received by the input unit such as the console 4. The inspector of the image can correct the image "1. image painting" displayed in the lower center of the process flow editing screen to generate a corrected image. For example, the inspector may erase the defect captured on the image by a pointing device (pointing device) such as a mouse, and rewrite the image as a good-quality image. In this way, the examiner corrects the image to generate a corrected image, thereby increasing an appropriate good product image.
The first processing may include a process of generating a corrected image by an image processing model. The image processing model may be any model, for example, a model in which a previously registered good product image is compared with an input image, and different portions are painted with the colors of peripheral pixels. In this way, the corrected image is automatically generated by the image processing model, and thus the cost for increasing the number of good-quality images can be reduced.
Fig. 4 is an example of a setting screen of the image processing apparatus 10 according to the present embodiment. On the setting screen of this example, "paint" is selected as "mode", the brightness setting area "is set to" 2 ", and the paint radius" is set to "1" for the setting item "parameter". By changing these parameters, the setting of the image editing tool can be changed.
On the setting screen of this example, the "C: "user" … "is set as" File "as the" File name ". The setting item "image" can set a folder for storing the corrected image and a file name of the corrected image.
Fig. 5 shows an example of an image Im1 captured by the image processing apparatus 10 according to the present embodiment. Image Im1 is a black and white image of workpiece W, and includes first defect D1, second defect D2, and third defect D3. In this figure, the first defect D1, the second defect D2, and the third defect D3 are surrounded by dashed circles.
Fig. 6 shows an example of a corrected image Im2 generated by the image processing apparatus 10 according to the present embodiment. The corrected image Im2 is an example of a corrected image generated by the first processing in the image editing step based on the image Im 1. The corrected image Im2 does not include the first defect D1, the second defect D2, and the third defect D3 included in the image Im 1. In this way, good-product images such as the corrected image Im2 can be generated from the defective-product images such as the image Im1, and the good-product images can be increased.
Fig. 7 is a flowchart of a first example of the addition processing of the learning dataset executed by the image processing apparatus 10 of the present embodiment. A first example of the addition processing of the data set for learning is processing for generating a corrected image by correcting a defective image based on the result of the determination of the presence or absence of the object by the examiner, and adding the corrected image to the data set for learning as a good image.
First, the image processing apparatus 10 inputs an image of the object captured by the camera 1 (S10). The inputted image is displayed on the monitor 3, and the examiner checks the image to determine whether the image is acceptable or not, and inputs the determination result through an input unit such as the console 4. Here, the image can be input in real time each time the object transported on the line L is captured by the camera 1. The image input may be performed by sequentially reading out images of the object captured in advance by the camera 1 and stored in the RAM 23 or the like.
If it is determined that the object has a defect (yes in S11), the image processing apparatus 10 executes a first process of smearing the defect of the object captured on the image to generate a corrected image (S12). Here, the correction image may be generated automatically by the image processing model or based on an input from the examiner.
In this way, the first process of generating the corrected image can be executed when the input unit receives the result of the determination of the acceptability of the object based on the input image and the determination result is a defect. In this way, when the inspection person determines that the inspection is defective or not, a corrected image is generated, and a good-product image can be increased.
Subsequently, the image processing apparatus 10 associates information indicating that the corrected image has been corrected with the image before correction (S13). Then, the corrected image is added as a good product image to the learning data set (S14). The image processing apparatus 10 may add the corrected image as a good product image to the learning data set, and may add the original image as a defective product image to the learning data set.
In this way, the second processing of adding the corrected image as the good product image to the data set for learning may include processing of adding the corrected image to the data set for learning in association with information indicating that the corrected image has been corrected. This makes it possible to specify which of the good product images included in the learning dataset is the corrected image, and to easily confirm the quality of the learning dataset.
The second processing may include processing for adding the corrected image to the learning data set in association with the image before correction of the corrected image. This makes it possible to easily refer to the image that is the source of generation of the corrected image, and to evaluate whether or not the first process for generating the corrected image has been performed appropriately afterward.
If it is determined that the object is not defective (no in S11), the image processing apparatus 10 adds the non-defective image to the learning data set as a non-defective image without correcting the input image (S14).
Subsequently, the image processing apparatus 10 determines whether to end the processing (S15). If the processing is not finished (no in S15), the image processing apparatus 10 repeats the processing after the image input (S10). On the other hand, when the processing is ended (yes in S15), the first example of the addition processing of the data set for learning ends.
Fig. 8 is a flowchart of a second example of the addition processing of the learning dataset executed by the image processing apparatus 10 of the present embodiment. A second example of the addition processing of the learning dataset is processing for correcting a defective image based on the result of the determination of the quality of the image of the object by the determination model to generate a corrected image, and adding the corrected image as a good image to the learning dataset.
First, the image processing apparatus 10 inputs an image of the object captured by the camera 1 (S20). The image processing apparatus 10 performs the image quality determination by using a determination model including a learned model generated using the learning dataset (S21). Here, the image can be input in real time each time the object transported on the line L is captured by the camera 1. The image input may be performed by sequentially reading out images of the object captured in advance by the camera 1 and stored in the RAM 23 or the like.
If the determination result obtained by the determination model is a failure (yes in S22) and the object includes a defect (yes in S23), the image processing apparatus 10 executes the first process of smearing the defect of the object captured on the image to generate a corrected image (S24). Here, the correction image may be generated automatically by the image processing model or based on an input from the examiner.
Subsequently, the image processing apparatus 10 associates information indicating that the corrected image has been corrected with the image before correction (S25). Then, the corrected image is added as a good product image to the learning data set (S26). The image processing apparatus 10 may add the corrected image as a good product image to the learning data set, and may add the original image as a defective product image to the learning data set.
In this way, the first process of generating the corrected image can be executed when the result of the determination of the quality of the object based on the image by the determination model is a failure and the object includes a defect. Thus, when the determination model determines that the defect is present, and the object actually includes the defect, the corrected image is generated, and the number of good-product images can be increased.
On the other hand, if the determination result obtained by the determination model is a failure (yes in S22), but the object does not have a defect (no in S23), the image processing apparatus 10 adds the original image as a good-quality image to the learning data set without generating a corrected image (S26).
In this way, the image editing step may include a third process of adding the image as a good-quality image to the learning data set when the determination model determines whether the object is defective or not based on the image and the object does not have a defect. In this way, when it is determined by the determination model that there is a defect and the object does not actually contain a defect, the original image can be added to the data set for learning as a good-quality image, and unnecessary execution of the image editing step can be omitted.
If the determination result obtained by the determination model is good (no in S22), the image processing apparatus 10 adds the good-quality image to the learning data set as a good-quality image without correcting the input image (S26).
Subsequently, the image processing apparatus 10 determines whether to end the processing (S27). If the processing is not finished (no in S27), the image processing apparatus 10 repeats the processing after the image input (S20). On the other hand, when the processing is ended (yes in S27), the second example of the addition processing of the data set for learning ends.
Fig. 9 is a flowchart of a third example of the addition processing of the learning dataset executed by the image processing apparatus 10 of the present embodiment. A third example of the addition processing of the learning dataset is processing for correcting a defective image based on the result of the determination of the quality of the image of the object by the determination model to generate a corrected image, and adding the corrected image as a good image to the learning dataset after the determination of the adoption by the determination model.
First, the image processing apparatus 10 inputs an image of the object captured by the camera 1 (S30). The image processing apparatus 10 performs the image quality determination by using a determination model including a learned model generated using the learning dataset (S31). Here, the image can be input in real time each time the object transported on the line L is captured by the camera 1. The image input may be performed by sequentially reading out images of the object captured in advance by the camera 1 and stored in the RAM 23 or the like.
If the determination result obtained by the determination model is a failure (yes in S32) and the object includes a defect (yes in S33), the image processing apparatus 10 executes the first process of smearing the defect of the object captured on the image to generate a corrected image (S34). Here, the correction image may be generated automatically by the image processing model or based on an input from the examiner.
Subsequently, the image processing apparatus 10 determines whether or not the object is good based on the corrected image by the determination model (S35). That is, the image processing apparatus 10 inputs the corrected image to the determination model, and checks whether or not the corrected image is determined as a good-quality image. If the determination result regarding the corrected image is not good (yes at S36), the image processing apparatus 10 executes the first process again to update the corrected image (S34). On the other hand, if the determination result regarding the corrected image is good (S36: No), information indicating that the corrected image has been corrected is associated with the image before correction (S37). Then, the corrected image is added as a good product image to the learning data set (S38). The image processing apparatus 10 may add the corrected image as a good product image to the learning data set, and may add the original image as a defective product image to the learning data set.
In this way, the second process of adding the corrected image as the good-quality image to the learning data set may include a process of adding the corrected image as the good-quality image to the learning data set when the result of the determination of the quality of the object based on the corrected image by the determination model is good. Thus, the corrected image can be determined by the determination model, and if the determination result is a failure, the corrected image is corrected again, and if the determination result is a good, the corrected image is added to the data set for learning as a good-quality image, so that an appropriate good-quality image can be added.
On the other hand, if the determination result obtained by the determination model is a failure (yes in S32) but the object does not have a defect (no in S33), the image processing apparatus 10 adds the original image as a good-quality image to the learning data set without generating a corrected image (S38).
If the determination result obtained by the determination model is good (no in S32), the image processing apparatus 10 adds the good-quality image to the learning data set as a good-quality image without correcting the input image (S38).
Subsequently, the image processing apparatus 10 determines whether to end the processing (S39). If the processing is not finished (no in S39), the image processing apparatus 10 repeats the processing after the image input (S30). On the other hand, when the processing is ended (yes in S39), the third example of the addition processing of the data set for learning ends.
Fig. 10 is an example of a process flow editing screen of the image processing apparatus 10 according to the present embodiment. An example of the process flow editing screen shown in the present drawing is a screen for editing the process flow executed by the controller 2.
On the process flow editing screen of this example, the first step S1, the second step S2a, and the third step S3 are set as the process flows. The first step S1 is a step described as "0 camera image input", and is a step of inputting the image of the object captured by the camera 1 to the controller 2. The second step S2a in which selection is omitted when executing is a step described as "1. image painting" in which an icon indicating selection is marked and which corresponds to an image editing step. The third step S3 is a step described as "2. exact match", and is a step of detecting a different point with high accuracy by superimposing the registered good-product image on the image input in the first step S1.
The input unit such as the console 4 can receive a selection of whether or not to omit execution of the image editing step (second step S2), and the execution unit such as the CPU 21 omits execution of the image editing step in accordance with the selection and executes a plurality of steps in a set order. Here, whether or not to omit the execution of the image editing step (second step S2) may be selected by a selection button B described as "measurement ON/OFF (ON/OFF)". When the omission of the second step S2 is selected by selecting the button B, the second step S2a whose selection is omitted is set, and after the execution of the first step S1, the execution unit of the CPU 21 or the like executes the third step S3 by omitting the execution of the second step.
In this way, the image editing step can be omitted and the processing flow can be executed, so that the operation for collecting the data set for learning and the operation during the actual operation of the image processing apparatus 10 can be easily switched.
The embodiments described above are for the convenience of understanding the present invention, and the present invention is not to be construed in a limiting sense. The elements included in the embodiments, their arrangement, materials, conditions, shapes, sizes, and the like are not limited to those illustrated in the drawings, and can be appropriately changed. Also, the structures shown in the different embodiments can be partially replaced or combined with each other.
[ notes ]
An image processing apparatus (10) comprising:
a camera (1) for capturing an image of an object; and
a controller (2) that executes a process flow including a plurality of steps,
the controller (2) has:
an input unit (4) for receiving the setting of the sequence of the plurality of steps; and
an execution unit (21) that executes the plurality of steps in the order described,
said plurality of steps comprising an image editing step,
the image editing step comprises:
a first process of generating a corrected image in which the image is corrected; and
and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
Claims (12)
1. An image processing apparatus comprising:
a camera for capturing an image of an object; and
a controller that executes a process flow including a plurality of steps,
the controller has:
an input unit that receives settings of the order of the plurality of steps; and
an execution unit configured to execute the plurality of steps in the order,
said plurality of steps comprising an image editing step,
the image editing step comprises:
a first process of generating a corrected image in which the image is corrected; and
and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
2. The image processing apparatus according to claim 1, wherein
The first processing is executed when the result of the determination of the quality of the object by the determination model based on the image is a failure and the object includes a defect.
3. The image processing apparatus according to claim 1 or 2, wherein
The image editing step further includes a third process,
the third processing is to add the image as a good-quality image to the learning data set when the determination model determines whether the object is defective based on the image and the object does not have a defect.
4. The image processing apparatus according to claim 1, wherein
The first processing is executed when the input unit receives a result of the determination of the quality of the object based on the image and the result is a failure.
5. The image processing apparatus according to any one of claims 1 to 4, wherein
The second processing includes processing for adding the corrected image as a good-quality image to the learning data set when the result of the determination of the object as to whether the object is good or not based on the corrected image by the determination model is good.
6. The image processing apparatus according to any one of claims 1 to 5, wherein
The first processing includes a process of generating the corrected image from an image processing model.
7. The image processing apparatus according to any one of claims 1 to 5, wherein
The first processing includes processing for generating the corrected image based on the correction of the image received by the input unit.
8. The image processing apparatus according to any one of claims 1 to 7, wherein
The second processing includes adding the corrected image to the learning dataset in association with information indicating that the corrected image has been corrected.
9. The image processing apparatus according to any one of claims 1 to 8, wherein
The second processing includes adding the corrected image to the learning data set in association with the image before the correction of the corrected image.
10. The image processing apparatus according to any one of claims 1 to 9, wherein
The input unit accepts a selection of whether or not execution of the image editing step is to be omitted,
the execution unit omits execution of the image editing step based on the selection, and executes the plurality of steps in the order.
11. An image processing method comprising:
receiving a setting of a sequence of a plurality of steps included in a process flow executed by a controller provided in an image processing apparatus; and
performing the plurality of steps in the order, in the image processing method,
said plurality of steps comprising an image editing step,
the image editing step comprises:
a first process of generating a corrected image in which an image of an object is corrected; and
and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
12. An image processing program for causing a computer that accepts setting of an order of a plurality of steps included in a processing flow to function as an execution unit that executes the plurality of steps in the order,
said plurality of steps comprising an image editing step,
the image editing step comprises:
a first process of generating a corrected image in which an image of an object is corrected; and
and a second process of adding the corrected image as a good-quality image to a data set for learning used for generating a determination model for determining whether or not the object is good.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2018109680A JP7148858B2 (en) | 2018-06-07 | 2018-06-07 | Image processing device, image processing method and image processing program |
| JP2018-109680 | 2018-06-07 | ||
| PCT/JP2019/020313 WO2019235223A1 (en) | 2018-06-07 | 2019-05-22 | Image processing device, image processing method, and image processing program |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112088395A true CN112088395A (en) | 2020-12-15 |
| CN112088395B CN112088395B (en) | 2024-01-16 |
Family
ID=68770186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980030801.3A Active CN112088395B (en) | 2018-06-07 | 2019-05-22 | Image processing apparatus, image processing method, and computer-readable storage medium |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP7148858B2 (en) |
| CN (1) | CN112088395B (en) |
| WO (1) | WO2019235223A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023184746A1 (en) * | 2022-03-29 | 2023-10-05 | 上海商汤智能科技有限公司 | Data processing method and apparatus, electronic device, and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022202365A1 (en) * | 2021-03-22 | 2022-09-29 | パナソニックＩｐマネジメント株式会社 | Inspection assistance system, inspection assistance method, and program |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH07287736A (en) * | 1994-04-19 | 1995-10-31 | N T T Data Tsushin Kk | Article identifying system |
| JPH11120362A (en) * | 1997-10-21 | 1999-04-30 | Omron Corp | Processor and method for image processing and machine-readable recording medium where criterion setting program is recorded |
| JP2004007234A (en) * | 2002-05-31 | 2004-01-08 | Make Softwear:Kk | Image editing device, and its method and program |
| JP2007128306A (en) * | 2005-11-04 | 2007-05-24 | Omron Corp | Image processing apparatus |
| JP2010072826A (en) * | 2008-09-17 | 2010-04-02 | Ricoh Co Ltd | Image processing apparatus, image processing method, program, and computer readable storage medium |
| JP2011043928A (en) * | 2009-08-19 | 2011-03-03 | Keyence Corp | Image processor, program creation support device, and image processing method |
| CN102005025A (en) * | 2009-08-31 | 2011-04-06 | 欧姆龙株式会社 | Image processing apparatus |
| CN102194227A (en) * | 2010-03-15 | 2011-09-21 | 欧姆龙株式会社 | Image attribute discrimination apparatus, attribute discrimination support apparatus, image attribute discrimination method and attribute discrimination support apparatus controlling method |
| CN103140872A (en) * | 2010-10-13 | 2013-06-05 | 欧姆龙株式会社 | Image processing apparatus and image processing method |
| JP2013140053A (en) * | 2011-12-28 | 2013-07-18 | Keyence Corp | Image processor and image processing method |
| JP2015143922A (en) * | 2014-01-31 | 2015-08-06 | オムロン株式会社 | Image processor, management system and management method |
| US20170358070A1 (en) * | 2016-06-13 | 2017-12-14 | Keyence Corporation | Image Processing Sensor, Image Processing Method, Image Processing Program, And Computer-Readable Recording Medium And Device Having Image Processing Program Recorded Therein |
| CN107895119A (en) * | 2017-12-28 | 2018-04-10 | 北京奇虎科技有限公司 | Program installation packet inspection method, device and electronic equipment |
| JP2018081629A (en) * | 2016-11-18 | 2018-05-24 | 住友電気工業株式会社 | Determination device, method for determination, and determination program |
-
2018
- 2018-06-07 JP JP2018109680A patent/JP7148858B2/en active Active
-
2019
- 2019-05-22 WO PCT/JP2019/020313 patent/WO2019235223A1/en active Application Filing
- 2019-05-22 CN CN201980030801.3A patent/CN112088395B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH07287736A (en) * | 1994-04-19 | 1995-10-31 | N T T Data Tsushin Kk | Article identifying system |
| JPH11120362A (en) * | 1997-10-21 | 1999-04-30 | Omron Corp | Processor and method for image processing and machine-readable recording medium where criterion setting program is recorded |
| JP2004007234A (en) * | 2002-05-31 | 2004-01-08 | Make Softwear:Kk | Image editing device, and its method and program |
| JP2007128306A (en) * | 2005-11-04 | 2007-05-24 | Omron Corp | Image processing apparatus |
| JP2010072826A (en) * | 2008-09-17 | 2010-04-02 | Ricoh Co Ltd | Image processing apparatus, image processing method, program, and computer readable storage medium |
| JP2011043928A (en) * | 2009-08-19 | 2011-03-03 | Keyence Corp | Image processor, program creation support device, and image processing method |
| CN102005025A (en) * | 2009-08-31 | 2011-04-06 | 欧姆龙株式会社 | Image processing apparatus |
| CN102194227A (en) * | 2010-03-15 | 2011-09-21 | 欧姆龙株式会社 | Image attribute discrimination apparatus, attribute discrimination support apparatus, image attribute discrimination method and attribute discrimination support apparatus controlling method |
| CN103140872A (en) * | 2010-10-13 | 2013-06-05 | 欧姆龙株式会社 | Image processing apparatus and image processing method |
| JP2013140053A (en) * | 2011-12-28 | 2013-07-18 | Keyence Corp | Image processor and image processing method |
| JP2015143922A (en) * | 2014-01-31 | 2015-08-06 | オムロン株式会社 | Image processor, management system and management method |
| US20170358070A1 (en) * | 2016-06-13 | 2017-12-14 | Keyence Corporation | Image Processing Sensor, Image Processing Method, Image Processing Program, And Computer-Readable Recording Medium And Device Having Image Processing Program Recorded Therein |
| JP2018081629A (en) * | 2016-11-18 | 2018-05-24 | 住友電気工業株式会社 | Determination device, method for determination, and determination program |
| CN107895119A (en) * | 2017-12-28 | 2018-04-10 | 北京奇虎科技有限公司 | Program installation packet inspection method, device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 林杰;李如意;: "基于深度学习的图像识别处理", 网络安全技术与应用, no. 11, pages 68 - 70 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023184746A1 (en) * | 2022-03-29 | 2023-10-05 | 上海商汤智能科技有限公司 | Data processing method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| JP7148858B2 (en) | 2022-10-06 |
| CN112088395B (en) | 2024-01-16 |
| JP2019212166A (en) | 2019-12-12 |
| WO2019235223A1 (en) | 2019-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7004145B2 (en) | Defect inspection equipment, defect inspection methods, and their programs | |
| US11301978B2 (en) | Defect inspection device, defect inspection method, and computer readable recording medium | |
| JP6869490B2 (en) | Defect inspection equipment, defect inspection methods, and their programs | |
| US10824906B2 (en) | Image processing device, non-transitory computer readable storage medium, and image processing system | |
| JP2019212073A (en) | Image discriminating apparatus and method thereof | |
| KR101117472B1 (en) | Apparatus and Method for Visual Inspection | |
| JP2000227316A (en) | Inspection device | |
| JP2014109479A (en) | Image processing apparatus, image processing method, and computer program | |
| CN112088395B (en) | Image processing apparatus, image processing method, and computer-readable storage medium | |
| CN113469991B (en) | Visual online detection method for laser welding spot of lithium battery tab | |
| WO2020012874A1 (en) | Image processing device, image processing method, and image processing program | |
| JP7215218B2 (en) | SPECTROSCOPY INSPECTION METHOD, IMAGE PROCESSING DEVICE, AND ROBOT SYSTEM | |
| JP2007198968A (en) | Image-classifying method and image-classifying apparatus | |
| JP2008032555A (en) | Inspection device for mounting part | |
| US6943826B1 (en) | Apparatus for debugging imaging devices and method of testing imaging devices | |
| WO2023162941A1 (en) | Inspection device, image processing method, and image processing program | |
| WO2023162940A1 (en) | Inspection device, inspection method, and inspection program | |
| US20230421891A1 (en) | Controller | |
| JP2002160161A (en) | Peeling efficiency evaluating method for blast device | |
| KR100274430B1 (en) | Examination apparatus and method for osd of automatic investigator | |
| CN116074646A (en) | Parameter adjustment support device and parameter adjustment support method | |
| JP2002071574A (en) | Defective image display method of image processing device, and image processing device | |
| JP3853007B2 (en) | Image processing device | |
| JP2012007979A (en) | Remote control inspection support device of automatic inspection device | |
| JP2023077058A (en) | Appearance inspection device and appearance inspection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |