WO2024090054A1 - 画像処理方法、プログラム、および、画像処理装置 - Google Patents

画像処理方法、プログラム、および、画像処理装置 Download PDF

Info

Publication number
WO2024090054A1
WO2024090054A1 PCT/JP2023/032840 JP2023032840W WO2024090054A1 WO 2024090054 A1 WO2024090054 A1 WO 2024090054A1 JP 2023032840 W JP2023032840 W JP 2023032840W WO 2024090054 A1 WO2024090054 A1 WO 2024090054A1
Authority
WO
WIPO (PCT)
Prior art keywords
small images
image
learning
images
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/032840
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勇斗 北川
太一 佐藤
久治 村田
正宏 石井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to CN202380073339.1A priority Critical patent/CN120077406A/zh
Priority to JP2024552868A priority patent/JPWO2024090054A1/ja
Publication of WO2024090054A1 publication Critical patent/WO2024090054A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/72—Data preparation, e.g. statistical preprocessing of image or video features

Definitions

  • This disclosure relates to an image processing method, a program, and an image processing device.
  • Patent Document 1 discloses a program that extracts multiple training images from an input image in order to train a classifier, classifies the training images into one or more sets, and displays the training images. The final training image is determined by the user selecting one of the displayed training images.
  • Machine learning for learning models such as classifiers requires many images for machine learning. If the composition of the machine learning images contains many images with high similarity, problems can arise, such as increased learning time and the data distribution differing from the distribution it should have, degrading the discrimination ability of the classifier. Therefore, it is desirable to easily select images that can be effectively used for machine learning, with fewer images for machine learning, and that can improve the performance of the learning model.
  • This disclosure provides an image processing method that makes it easier to select images that are effective for machine learning.
  • An image processing method is an image processing method executed by a computer, and includes an acquisition step of acquiring an original image showing an object, a selection step of selecting two or more small images that are effective for machine learning from among a plurality of small images generated by dividing the original image, based on a learning contribution degree indicating the degree of effectiveness in machine learning of each of the plurality of small images, and an output step of outputting the two or more small images in a display mode according to their respective learning contribution degrees.
  • a program according to one aspect of the present disclosure is a program for causing a computer to execute an image processing method according to one aspect of the present disclosure.
  • An image processing device includes an acquisition unit that acquires an original image showing an object, a selection unit that selects two or more small images that are effective for machine learning from among a plurality of small images generated by dividing the original image, based on a learning contribution level indicating the degree of effectiveness of each of the plurality of small images in machine learning, and an output unit that outputs the two or more small images in a display mode according to their respective learning contribution levels.
  • This disclosure provides an image processing method that makes it easier to select images that are effective for machine learning.
  • FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment.
  • FIG. 2 is a diagram for explaining a process in which the image processing device according to the embodiment determines the display mode of two or more small images.
  • FIG. 3 is a diagram for explaining normal and abnormal regions in an original image according to the embodiment.
  • FIG. 4 is a diagram for explaining a first example of an image output by the image processing device according to the embodiment.
  • FIG. 5 is a diagram for explaining a second example of an image output by the image processing device according to the embodiment.
  • FIG. 6 is a diagram for explaining a third example of an image output by the image processing device according to the embodiment.
  • FIG. 7 is a flowchart showing a processing procedure of the image processing device according to the embodiment.
  • each figure is a schematic diagram and is not necessarily a precise illustration.
  • the same reference numerals are used for substantially the same configuration, and duplicate explanations are omitted or simplified.
  • FIG. 1 is a block diagram showing the configuration of an image processing device 100 according to an embodiment.
  • the image processing device 100 is a device that displays an image (small image) based on an image (original image) generated by an imaging device such as a camera capturing an image of an object (workpiece).
  • the image processing device 100 is an automatic learning image selection device that selects small images (hereinafter also referred to as learning images) from a plurality of small images generated by dividing an original image, for use in machine learning (AI (Artificial Intelligence) learning) in a learning model that determines whether or not an object shown in the original image contains a defect.
  • AI Artificial Intelligence
  • a learning model is trained using various learning images obtained by photographing an object and information (annotation information) indicating whether each learning image is defective or normal.
  • training images that can effectively train the learning model, that is, images that can improve the performance of the learning model with a small number of images.
  • training images that cannot effectively train the learning model.
  • candidates for selecting small images of normal areas that do not contain defects but there is an issue in that it is unclear which candidate to select to be effective for machine learning.
  • the image processing device 100 therefore outputs learning images that can effectively train the learning model in a manner that is easy for the user to understand.
  • performance refers to, for example, the accuracy rate at which defects can be correctly extracted when an original image is input into a machine-learned learning model, or the accuracy rate at which it can correctly determine that there are no defects.
  • the image processing device 100 is, for example, a computer such as a personal computer or a tablet terminal.
  • the image processing device 100 is realized by a communication interface for communicating with the display device 200 and the input device 210, a non-volatile memory in which a program is stored, a volatile memory which is a temporary storage area for executing the program, an input/output port for sending and receiving signals, a processor for executing the program, and the like.
  • the communication interface may be realized by a connector to which a communication line is connected to enable wired communication, or may be realized by an antenna and wireless communication circuitry to enable wireless communication.
  • the image processing device 100 includes an information processing unit 110 and a memory unit 120.
  • the information processing unit 110 is a processing unit that performs various processes executed by the image processing device 100. For example, the information processing unit 110 outputs a plurality of small images obtained by performing image processing on the acquired original image to the display device 200, thereby displaying the multiple small images on the displayed image.
  • FIG. 2 is a diagram for explaining the process in which the image processing device 100 according to the embodiment determines the display mode of two or more small images.
  • the display mode is changed so that the outer edge of small images with a learning contribution degree equal to or greater than a predetermined degree is a thick line, a dashed line, or a dashed line.
  • ⁇ refers to, for example, a predetermined degree of contribution to learning or more.
  • the predetermined degree of contribution to learning may be determined arbitrarily.
  • the learning contribution of each of the multiple small images is determined, for example, based on the similarity between the multiple small images.
  • the similarity is calculated, for example, from the average value of the differences, such as the luminance difference or color difference, between each pixel at the same position in the two small images. For example, the similarity is calculated so that the larger the average value is, the lower the similarity is.
  • the learning contribution is determined from the calculated similarity.
  • the learning contribution is set, for example, so that the lower the similarity is, the higher the learning contribution is.
  • the information processing unit 110 is realized by one or more processors.
  • the information processing unit 110 includes an acquisition unit 111, a selection unit 112, an output unit 113, a reception unit 114, and a storage unit 120.
  • the acquisition unit 111 is a processing unit that acquires an original image in which an object is shown. Specifically, the acquisition unit 111 acquires an original image in which a first object is shown.
  • the target object is an object to be inspected by the learning model.
  • the acquisition unit 111 acquires an original image showing the target object, such as that shown in FIG. 2(a), from an imaging device that captures the target object, for example.
  • the object is, for example, an industrial product.
  • the object is an electronic component such as an IC (Integrated Circuit).
  • the target object does not have to be an electronic component; it can be any object, such as a circuit board.
  • the imaging device is a camera that generates an original image by capturing an image of an object.
  • the imaging device is realized, for example, by a CMOS (Complementary Metal Oxide Semiconductor) image sensor.
  • CMOS Complementary Metal Oxide Semiconductor
  • the acquisition unit 111 may also acquire the original image from a server device or the like via a communication interface provided in the image processing device 100.
  • the selection unit 112 is a processing unit that selects two or more small images that are effective for machine learning from among multiple small images generated by dividing the original image, based on the learning contribution degree that indicates the degree of effectiveness of each of the multiple small images in machine learning.
  • the selection unit 112 generates a plurality of small images by dividing the original image.
  • the way in which the original image is divided may be determined arbitrarily.
  • the number of the plurality of small images may be determined arbitrarily.
  • the plurality of small images may be rectangular, or may have any shape such as a triangle or a circle.
  • the sizes and shapes of the plurality of small images may be the same or different.
  • the selection unit 112 selects any one image from the multiple small images.
  • the image selected here may be determined arbitrarily. In the example shown in FIG. 2(c), first, the small image located at the top left corner of the multiple small images shown in FIG. 2(c) is selected from the multiple small images.
  • the selection unit 112 calculates the similarity between the selected small image and the multiple unselected small images. Furthermore, the selection unit 112 selects the image with the lowest similarity from the multiple unselected small images.
  • the selection unit 112 selects two or more small images that are effective for machine learning by repeating the process of selecting such small images and calculating the similarity (also called the selection process) a predetermined number of times. In other words, the selection unit 112 selects two or more small images by repeatedly executing the process of selecting one small image from among a plurality of small images excluding all of the small images already selected, based on the similarity between the plurality of small images excluding all of the small images already selected and all of the small images already selected.
  • the selection unit 112 selects two or more small images that are effective for machine learning from among a plurality of small images, based on the learning contribution (more specifically, the similarity) that indicates the degree of effectiveness of each of the plurality of small images in machine learning.
  • the predetermined number of times may be determined arbitrarily. For example, the predetermined number of times is determined based on a threshold. For example, the selection unit 112 selects two or more small images based on the similarity between multiple small images and the threshold of the similarity. For example, if the threshold is 0.2, the selection unit 112 repeats the selection process until there are no small images with a calculated similarity of 0.2 or less.
  • the predetermined number of times may be determined arbitrarily by, for example, the user.
  • the reception unit 114 may receive information indicating the predetermined number of times or information indicating a threshold value from the user via the input device 210.
  • the larger the threshold the more small images there are that are two or more. In other words, the larger the threshold, the more small images the selection unit 112 selects.
  • the threshold may be one or more.
  • the threshold includes a first threshold and a second threshold that is greater than the first threshold, and the selection unit 112 selects two or more small images from the multiple small images, including a first image with a similarity less than the first threshold and a second image with a similarity equal to or greater than the first threshold and less than the second threshold.
  • each of the two or more small images selected by the selection unit 112 is an image of a normal area that does not contain any defects of the object in the original image.
  • FIG. 3 is a diagram for explaining normal and abnormal regions in an original image according to an embodiment of the present invention. Specifically, FIG. 3 is a diagram showing multiple small images obtained by dividing the original image.
  • a normal area is an area in the original image that is free of defects such as scratches, chips, stains, or dust.
  • the small images included in the "normal area” are all of the multiple small images other than the four small images surrounded by thick lines.
  • an abnormal area is an area in the original image that has such defects.
  • the small images included in the "abnormal area” are all of the multiple small images other than the four small images surrounded by thick lines.
  • the selection unit 112 When selecting two or more small images, for example, the selection unit 112 does not select small images from abnormal areas that contain defects, but selects two or more small images from small images in normal areas that do not contain defects.
  • the original image acquired by the acquisition unit 111 is displayed on the display unit 200 by being output to the display unit 200 by the output unit 113.
  • the user operates the input device 210 to input the position of the defect in the original image.
  • the reception unit 114 receives the input.
  • the selection unit 112 selects two or more small images from among the small images of normal areas that do not contain defects based on the input received by the reception unit 114. At this time, for example, the selection unit 112 may add information indicating that the small images are normal (e.g., that there is no defect) or that they are abnormal (e.g., that there is a defect), that is, annotation information, to the multiple small images based on the input, and store them in the storage unit 120.
  • the output unit 113 is a processing unit that outputs the two or more small images selected by the selection unit 112 in a display mode corresponding to the respective degrees of contribution to learning. Specifically, the output unit 113 changes the display mode of the two or more small images selected by the selection unit 112 to a display mode corresponding to the respective degrees of contribution to learning, and outputs image information including the two or more small images with the changed display mode to the display device 200, thereby causing the display device 200 to display the two or more small images with the changed display mode.
  • outputting two or more small images means that an image containing two or more small images is output.
  • Multiple small images generated by dividing an original image containing two or more small images may be output, or the display mode of the parts of the original image corresponding to the two or more small images may be changed and output.
  • the display format may be determined arbitrarily.
  • the output unit 113 outputs a plurality of small images with different decorations around or inside each of the two or more small images based on the learning contribution of each of the two or more small images.
  • adding decoration includes, for example, adding a border around each of the two or more small images.
  • the output unit 113 determines at least one of the display modes of the border thickness, border color, and border shape based on the learning contribution of each of the two or more small images.
  • the border shape is the line type, and is a line shape such as a solid line, a dotted line, a dashed line, and a dashed dot line.
  • the output unit 113 adds a border to two or more small images so that the border is thicker for those with a higher learning contribution and thinner for those with a lower learning contribution.
  • FIG. 4 is a diagram for explaining a first example of an image output by the image processing device 100 according to the embodiment. Specifically, it is a diagram showing an example of image information output to the output unit 113 and displayed on the display device 200.
  • the display device 200 displays an original image in which the parts of the original image corresponding to small images with a predetermined learning contribution level or higher are surrounded by any of a solid line, a dashed line, and a dashed line.
  • the selection unit 112 selects, from among the multiple small images, a first image with a similarity level less than a first threshold, a second image with a similarity level equal to or greater than the first threshold and less than a second threshold, and a third image with a similarity level equal to or greater than the second threshold and less than a third threshold.
  • the output unit 113 changes the display mode of two or more small images so that the part of the original image corresponding to the first image (the "small image with the highest learning contribution level” shown in FIG. 4) is surrounded by a solid line, the part corresponding to the second image (the "small image with the second highest learning contribution level after the solid line rectangle” shown in FIG. 4) is surrounded by a dashed line, and the part corresponding to the third image (the "small image with the second highest learning contribution level after the dashed line rectangle" shown in FIG. 4) is surrounded by a dashed line.
  • the output unit 113 outputs the first image and the second image in different display modes.
  • the output unit 113 outputs an original image in which the first image and the second image are displayed in different display modes.
  • the output unit 113 outputs information indicating that the first image is a small image having a higher learning contribution than the second image.
  • the information is, for example, information indicating an explanation of the learning contribution (i.e., similarity) of two or more small images, such as the "small image with the highest learning contribution" shown in FIG. 4.
  • the output unit 113 outputs information about two or more small images in order of decreasing contribution to learning.
  • the output unit 113 outputs image information such that descriptions about two or more small images (e.g., "small image with the highest contribution to learning") are arranged in order of decreasing contribution to learning, starting from the top of the image displayed on the display device 200.
  • outputting information about two or more small images in order of the degree of learning contribution may include, for example, displaying the solid lines, dashed lines, and dashed dotted lines surrounding the two or more images shown in FIG. 4 in this order, sequentially changing over time.
  • information about two or more small images may include information for explaining the two or more small images, and the display mode of the two or more small images, such as a frame line.
  • "in order of highest contribution" may refer to a spatial order, such as from the top, or may refer to a temporal order.
  • decorating two or more small images includes, for example, at least one of correcting the hue, saturation, and brightness of each of the two or more small images.
  • the output unit 113 performs corrections on the two or more small images to make them closer to expanding colors such as warm colors, to increase the saturation, or to increase the brightness, thereby making the images more eye-catching.
  • the output unit 113 may change the display mode by adding a frame around each of the two or more small images and correcting the images, such as correcting the hue.
  • a small image among the multiple small images that is not selected by the selection unit 112 i.e., a small image other than the two or more small images
  • a small image among the multiple small images that is not selected by the selection unit 112 may be corrected, such as by lowering the brightness to make it less visible.
  • the reception unit 114 is a processing unit that receives user operations.
  • the reception unit 114 receives user operations, for example, via the input device 210.
  • the reception unit 114 receives input of position information indicating the position of an abnormal area (or a defect) contained in an original image. For example, the user looks at the original image or a small image displayed on the display device 200, and inputs the position of the abnormal area contained in the original image or the small image, or the small image containing a defect, using the input device 210.
  • the reception unit 114 receives the input, for example, as position information.
  • the reception unit 114 may receive a first instruction indicating the first threshold value or the second threshold value, and the output unit 113 may determine and output the display mode of two or more small images based on the first instruction received by the reception unit 114. In other words, the display mode of two or more images in the image information displayed on the display device 200 may be changed based on the first instruction.
  • FIG. 5 is a diagram for explaining a second example of an image output by the image processing device 100 according to the embodiment.
  • FIG. 6 is a diagram for explaining a third example of an image output by the image processing device 100 according to the embodiment.
  • the selection unit 112 selects, from among a plurality of small images, a first image having a similarity less than a first threshold, a second image having a similarity equal to or greater than the first threshold and less than a second threshold, and a third image having a similarity equal to or greater than the second threshold and less than a third threshold.
  • the first threshold is 0.2
  • the second threshold is 0.4
  • the third threshold is 0.6.
  • the output unit 113 first outputs image information in which a border has been added to the small image corresponding to the first image.
  • the output unit 113 outputs image information in which the small images corresponding to the first image and the second image are bordered.
  • the user selects a small image to be used for machine learning from among two or more small images. For example, if the user selects a threshold of 0.2, the first image is determined as the learning image to be used for machine learning. Also, for example, if the user selects a threshold of 0.4, the first image and the second image are determined as the learning images to be used for machine learning.
  • the receiving unit 114 receives a first instruction
  • the output unit 113 determines a learning image from among two or more small images based on the first instruction, and stores information indicating that the image is a learning image in the storage unit 120. For example, when the receiving unit 114 receives an instruction to perform machine learning, the output unit 113 selects a learning image based on the information, and inputs the selected learning image into the learning model, thereby performing machine learning in the learning model.
  • the image to be used for learning may be selected arbitrarily from among multiple small images.
  • the reception unit 114 receives a second instruction indicating which small images, starting from the small image with the highest learning contribution, among two or more small images ranked based on the degree of learning contribution, are to be used for machine learning.
  • the learning images may be determined.
  • the selection unit 112 may repeatedly execute the above selection process for all small images to calculate the similarity for all small images and calculate the learning contribution based on the calculated similarity. Note that when calculating the learning contribution for all small images, the selection unit 112 may select two or more small images for which the display mode is to be changed after calculating the learning contribution for all small images.
  • the reception unit 114 may receive a third instruction indicating the thickness of the border, and determine, among the two or more small images, a small image decorated with a border line thicker than the border indicated by the third instruction as the image to be used for machine learning.
  • the acquisition unit 111, the selection unit 112, the output unit 113, and the reception unit 114 may be realized, for example, by a common processor, or may each be realized by an independent processor.
  • the storage unit 120 is a storage device that stores programs executed by the processing units, such as the acquisition unit 111, the selection unit 112, the output unit 113, and the reception unit 114, to perform each process, information required for the process, and inspection images.
  • the storage unit 120 is realized, for example, by a HDD (Hard Disk Drive) and/or a semiconductor memory.
  • the display device 200 is a display that displays an image based on the control of the image processing device 100 (more specifically, the output unit 113).
  • the display device 200 displays, for example, a plurality of small images (i.e., original images) including two or more small images.
  • the display device 200 is realized by, for example, a display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel.
  • the input device 210 is a user interface that accepts user operations.
  • the input device 210 is realized by a mouse, a keyboard, a touch panel, and/or hardware buttons, etc.
  • the display device 200 and the input device 210 may be integrated into one device such as a touch panel display.
  • FIG. 7 is a flowchart showing the processing steps of the image processing device 100 according to the embodiment.
  • the acquisition unit 111 acquires an original image showing an object (S10).
  • the acquisition unit 111 acquires the original image from a camera (not shown) via a communication interface or the like provided in the image processing device 100.
  • the original image may be stored in, for example, the storage unit 120.
  • the acquisition unit 111 acquires the original image from, for example, the storage unit 120.
  • the selection unit 112 selects two or more small images that are effective for machine learning from among the multiple small images generated by dividing the original image, based on the learning contribution degree indicating the degree of effectiveness of each of the multiple small images in machine learning (S20). Specifically, the selection unit 112 generates multiple small images by dividing the original image acquired by the acquisition unit 111. Next, the selection unit 112 selects any one image from the multiple small images. In the above example, for example, the small image located at the top left corner shown in (c) of FIG. 2 is first selected from the multiple small images. Next, the selection unit 112 calculates the similarity between the selected small image and multiple small images that have not been selected.
  • the selection unit 112 calculates the similarity between all small images that have already been selected and small images that have not yet been selected.
  • the selection unit 112 selects two or more small images that are effective for machine learning by repeating such processing a predetermined number of times.
  • the predetermined number of times may be determined arbitrarily. For example, in the above example, the predetermined number of times is determined based on a threshold value.
  • the similarity between each small image may be calculated from the average value of the similarity between each small image.
  • the output unit 113 outputs the two or more small images selected by the selection unit 112 in a display mode corresponding to the degree of contribution to learning of each of the two or more small images (S30). Specifically, the output unit 113 causes the display device 200 to display the two or more small images selected by the selection unit 112 in a display mode corresponding to the degree of contribution to learning of each of the two or more small images.
  • the output unit 113 may output two or more small images selected by the selection unit 112 to the learning model, thereby causing the learning model to perform machine learning.
  • Technology 1 is an image processing method executed by a computer, and includes an acquisition step (S10) of acquiring an original image showing an object, a selection step (S20) of selecting two or more small images that are effective for machine learning from among a plurality of small images generated by dividing the original image, based on the learning contribution degree of each of the plurality of small images that indicates the degree of effectiveness in machine learning, and an output step (S30) of outputting the two or more small images in a display mode according to their respective learning contribution degrees.
  • Machine learning that uses images as input requires many images as learning data.
  • multiple images with similar image features such as the same shape and arrangement of objects in the images, are less effective for machine learning than multiple images with dissimilar image features. Therefore, by using multiple images with dissimilar image features for machine learning, machine learning can be performed effectively, for example, so that appropriate output can be obtained even with a small number of images for machine learning. Therefore, in an image processing method according to one aspect of the present disclosure, from multiple small images generated by dividing an original image, two or more small images that are effective for machine learning are output in a display mode corresponding to each learning contribution level based on the learning contribution level indicating the degree of effectiveness of each of the multiple small images in machine learning. According to this, since the small images can be displayed in a mode corresponding to the learning contribution level, it is easy for the user to select images that are effective for machine learning.
  • Technology 2 is an image processing method described in Technology 1, in which the learning contribution of each of the multiple small images is determined based on the similarity between the multiple small images, and in the selection step, two or more small images are selected based on the similarity between the multiple small images and a threshold value for the similarity.
  • the image processing method is a method for automatically selecting learning images using similarity, and selects small images that are effective for learning based on the similarity between small images.
  • small images that are not similar to each other, i.e., have low similarity can be automatically selected from among multiple candidates (i.e., multiple small images), so that two or more small images that are effective for machine learning can be appropriately selected.
  • the discrimination performance of the learning model can be improved with a smaller number of small images.
  • Technique 3 is the image processing method described in technique 2, in which each of the two or more small images is an image of a normal area that does not contain any defects of the object in the original image.
  • the image processing method according to one aspect of the present disclosure is particularly effective for images of normal regions.
  • Technique 4 is an image processing method described in Technique 2 or 3, in which the larger the threshold value, the greater the number of small images (two or more).
  • the threshold increases, the number of small images (two or more) displayed on the display device 200 increases.
  • the threshold can be set high to easily change the display mode of images that are effective for machine learning to one that is easy for the user to understand.
  • Technology 5 is an image processing method according to any one of techniques 2 to 4, in which, in the selection step, two or more small images are selected by repeatedly executing a process of selecting one small image from among a plurality of small images excluding all of the small images already selected, based on the similarity between the plurality of small images excluding all of the small images already selected and all of the small images selected.
  • the similarity between selected and unselected small images is calculated, and the small image that will be most effective for the next learning step is selected.
  • Technology 6 is an image processing method according to any one of techniques 2 to 5, in which the thresholds include a first threshold and a second threshold greater than the first threshold, and in the selection step, two or more small images including a first image having a similarity less than the first threshold and a second image having a similarity equal to or greater than the first threshold and less than the second threshold are selected from the plurality of small images, and in the output step, the first image and the second image are output in different display modes.
  • Technology 7 is an image processing method according to Technology 6, which further includes a receiving step of receiving a first instruction indicating a first threshold value or a second threshold value, and in an output step, a display mode of two or more small images is determined and output based on the first instruction received in the receiving step.
  • the output step first, as shown in FIG. 4, a combination of a threshold and a frame line corresponding to the threshold is output (displayed).
  • a selection of a threshold desired by the user is received from the user.
  • the output step second output step
  • the display mode of the small image for example, a frame line
  • the image shown in FIG. 5 is output in the second output step.
  • the small image selected in this way and below the threshold is used for machine learning of the learning model.
  • an objective evaluation image similarity value is used, which may differ from the similarity when viewed by a human.
  • the image to be used for machine learning is ultimately selected based on the threshold selected by the user, thereby filling in the gap between the judgments of a computer and a human.
  • Technology 8 is an image processing method according to any one of techniques 2 to 7, in which the thresholds include a first threshold and a second threshold greater than the first threshold, and in the selection step, two or more small images including a first image having a similarity less than the first threshold and a second image having a similarity equal to or greater than the first threshold and less than the second threshold are selected from the plurality of small images, and in the output step, information indicating that the first image is a small image having a higher contribution to learning than the second image is output.
  • the display device 200 displays a small image selected when the similarity threshold is small as a small image with a high degree of contribution to learning.
  • images selected based on a relatively small threshold value indicate images that are dissimilar to one another, and so users can easily select small images with the same features as images to be used in machine learning, that is, images with a high degree of contribution to learning.
  • the smaller the threshold value is the more dissimilar the small images selected are, and the more likely it is that multiple small images that have the same label (e.g., a predetermined feature such as brightness) and various different features for that label will be selected by the user as images for learning.
  • Technology 9 is an image processing method according to any one of techniques 1 to 8, further including a receiving step of receiving a second instruction indicating which small images, starting from the small image with the highest learning contribution, among two or more small images ranked based on the degree of learning contribution, are to be used for machine learning.
  • Technology 10 is an image processing method according to any one of techniques 1 to 9, in which, in the output step, a plurality of small images are output, each of which has different decorations around or inside the two or more small images based on the learning contribution of each of the two or more small images.
  • Technology 11 is an image processing method according to Technology 10, in which the decorating step includes adding a border around each of the two or more small images, and in the output step, at least one display mode of the border thickness, the border color, and the border shape is determined based on the learning contribution of each of the two or more small images.
  • Technology 12 is the image processing method described in Technology 10 or 11, in which the decoration includes at least one of correcting the hue, saturation, and brightness of each of the two or more small images.
  • Technology 13 is the image processing method described in Technology 11, in which, in the output step, two or more small images are bordered with a thicker border for images with a higher learning contribution and a thinner border for images with a lower learning contribution, and the image processing method further includes a receiving step of receiving a third instruction indicating the thickness of the border, and determining, as an image to be used for machine learning, a small image among the two or more small images that is decorated with a border line thicker than that indicated by the third instruction.
  • Technology 14 is an image processing method according to any one of techniques 1 to 13, in which the object is an industrial product.
  • Machine learning using images is used for various applications, such as inspecting industrial products such as parts of electrical equipment and identifying people. Unlike people, for example, identical industrial products are mechanically produced, so even images of the same industrial product that show different objects often have high similarity. In addition, unnecessary processing is rarely performed to facilitate manufacturing, and there may be many parts with high similarity even within a single image. For this reason, the image processing method according to one aspect of the present disclosure is particularly effective when dealing with images that tend to contain highly similar images, such as industrial products.
  • Technology 15 is an image processing method according to any one of techniques 1 to 14, in which, in the output step, information about two or more small images is output in order of the degree of contribution to learning.
  • Technology 16 is a program for causing a computer to execute the image processing method described in any one of Technologies 1 to 15.
  • Technology 17 is an image processing device 100 that includes an acquisition unit 111 that acquires an original image showing an object, a selection unit 112 that selects two or more small images that are effective for machine learning from among a plurality of small images generated by dividing the original image, based on the learning contribution degree that indicates the degree of effectiveness of each of the plurality of small images in machine learning, and an output unit 113 that outputs the two or more small images in a display mode according to their respective learning contribution degrees.
  • the image processing device 100 is realized as a single device, but it may be realized by multiple devices.
  • the components of the image processing device described in the above embodiment may be distributed in any manner among the multiple devices.
  • processing performed by a specific processing unit may be executed by another processing unit.
  • the order of multiple processes may be changed, and multiple processes may be executed in parallel.
  • each component may be realized by executing a software program suitable for each component.
  • Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.
  • a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.
  • each component may be realized by hardware.
  • Each component may be a circuit (or an integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit, or a dedicated circuit.
  • the general or specific aspects of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM.
  • the present disclosure may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.
  • the present disclosure may be realized as an image processing method executed by a computer such as an image processing device.
  • the present disclosure may also be realized as a program for causing a computer to execute the image processing method, or as a computer-readable non-transitory recording medium on which such a program is recorded.
  • this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art may conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the spirit of this disclosure.
  • the present disclosure is useful as an image processing device that presents images to a user.
  • REFERENCE SIGNS LIST 100 Image processing device 110 Information processing unit 111 Acquisition unit 112 Selection unit 113 Output unit 114 Reception unit 120 Storage unit 200 Display device 210 Input device

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
PCT/JP2023/032840 2022-10-28 2023-09-08 画像処理方法、プログラム、および、画像処理装置 Ceased WO2024090054A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202380073339.1A CN120077406A (zh) 2022-10-28 2023-09-08 图像处理方法、程序以及图像处理装置
JP2024552868A JPWO2024090054A1 (https=) 2022-10-28 2023-09-08

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-173631 2022-10-28
JP2022173631 2022-10-28

Publications (1)

Publication Number Publication Date
WO2024090054A1 true WO2024090054A1 (ja) 2024-05-02

Family

ID=90830408

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/032840 Ceased WO2024090054A1 (ja) 2022-10-28 2023-09-08 画像処理方法、プログラム、および、画像処理装置

Country Status (3)

Country Link
JP (1) JPWO2024090054A1 (https=)
CN (1) CN120077406A (https=)
WO (1) WO2024090054A1 (https=)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021060692A (ja) * 2019-10-03 2021-04-15 株式会社東芝 推論結果評価システム、推論結果評価装置及びその方法
JP2022099572A (ja) * 2020-12-23 2022-07-05 名古屋電機工業株式会社 情報処理装置、情報処理方法およびプログラム

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021060692A (ja) * 2019-10-03 2021-04-15 株式会社東芝 推論結果評価システム、推論結果評価装置及びその方法
JP2022099572A (ja) * 2020-12-23 2022-07-05 名古屋電機工業株式会社 情報処理装置、情報処理方法およびプログラム

Also Published As

Publication number Publication date
JPWO2024090054A1 (https=) 2024-05-02
CN120077406A (zh) 2025-05-30

Similar Documents

Publication Publication Date Title
US11386549B2 (en) Abnormality inspection device and abnormality inspection method
JP6792842B2 (ja) 外観検査装置、変換データ生成装置、及びプログラム
KR20180117532A (ko) 결함 검출 방법, 결함 검출을 위한 시스템 및 훈련 방법
JP7492240B2 (ja) 分析装置及び分析方法
JP2018005640A (ja) 分類器生成装置、画像検査装置、及び、プログラム
JP2018005639A (ja) 画像分類装置、画像検査装置、及び、プログラム
JP2013224833A (ja) 外観検査装置、外観検査方法及びコンピュータプログラム
JP2018152081A (ja) 取得された画像内のクラッタを決定するためのシステム及び方法
CN113012096A (zh) 显示屏子像素定位及亮度提取方法、设备以及存储介质
CN118488329A (zh) 像素坏点矫正方法、装置、电子设备及存储介质
JP2020071716A (ja) 異常判定方法、特徴量算出方法、外観検査装置
JP2005283197A (ja) 画面のスジ欠陥検出方法及び装置
AU2013350323A1 (en) Method for detection of faults on circuit boards
US11875547B2 (en) Image processing apparatus, image processing method, and storage medium
Wang et al. SCP-DETR: A efficient small-object-enhanced feature pyramid approach for PCB defect detection
WO2024090054A1 (ja) 画像処理方法、プログラム、および、画像処理装置
JP7658493B2 (ja) 検出支援装置、学習装置、検出支援方法、学習方法、及びプログラム
JP7302410B2 (ja) 画像認識装置、画像認識システム、画像認識方法及びプログラム
US20200264108A1 (en) Information processing apparatus, information processing method, and recording medium
CN119359647A (zh) 基于多通道扫描的缺陷检测方法、装置、设备及存储介质
Wang et al. How real is reality? A perceptually motivated system for quantifying visual realism in digital images
JP7533263B2 (ja) 画像検査装置、画像検査方法、及び学習済みモデル生成装置
CN117788444A (zh) Smt贴片偏移检测方法、装置及视觉检测系统
JP7714585B2 (ja) 自動アーチファクト検出
JP2006145228A (ja) ムラ欠陥検出方法及び装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23882267

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024552868

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202380073339.1

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 202380073339.1

Country of ref document: CN

122 Ep: pct application non-entry in european phase

Ref document number: 23882267

Country of ref document: EP

Kind code of ref document: A1