JP6387477B1 - Inspection device, inspection method, and inspection program - Google Patents

Abstract

An object of the present invention is to provide an inspection apparatus, an inspection method, and an inspection program capable of accurately detecting the presence / absence of an abnormality related to an inspection object even when there are various abnormalities that may occur. The inspection apparatus 10 includes a first dividing unit that divides the image of the inspection object 100 into a plurality of first partial images, and a second that divides the image into a plurality of second partial images different from the plurality of first partial images. A first classification unit that classifies the plurality of first partial images into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality; A second classification unit that classifies a plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality; and an abnormality A determination unit that determines whether there is an abnormality related to the inspection object 100 based on an overlap between the first partial image determined to be abnormal and the second partial image determined to include abnormality. .

Description

  The present invention relates to an inspection apparatus, an inspection method, and an inspection program.

  Conventionally, there has been a case where an image of an inspection object flowing through a production line is taken to detect the presence or absence of an abnormality related to the inspection object. The abnormality of the inspection object is, for example, a scratch or a dirt attached to the inspection object, or a foreign matter mixed in the inspection object.

  Regarding abnormality detection, for example, in Patent Document 1 below, a first processing unit that classifies signals to be inspected into normal and non-normal using a learned first neural network, and a learned second neural network, There is described an inspection apparatus including a second processing unit that extracts a partial signal including a region other than normal from an inspection target signal classified as non-normal by one processing unit and classifies the type of abnormality for the partial signal. .

JP 2012-26982 A

  According to the inspection apparatus described in Patent Literature 1, it is possible to detect the presence or absence of abnormality related to the inspection object by the first processing unit, and it is possible to classify the type of abnormality by the second processing unit. However, the detection accuracy of the presence or absence of abnormality depends on the classification accuracy of classifying the inspection target signal into normal and non-normal by the first processing unit, and when there are various possible abnormalities, the first processing unit The classification accuracy was not sufficiently high, and the detection accuracy was not necessarily high.

  Therefore, an object of the present invention is to provide an inspection apparatus, an inspection method, and an inspection program that can accurately detect the presence or absence of an abnormality related to an inspection object even when there are various abnormalities that may occur.

  An inspection apparatus according to an aspect of the present invention includes a first division unit that divides an image of an inspection object into a plurality of first partial images, and a plurality of second partial images that are different from the plurality of first partial images. A second division unit that divides the first partial image into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality. A first classification unit, and a second classification unit that classifies the plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality. The determination unit that determines whether there is an abnormality related to the inspection object based on an overlap between the first partial image that is determined to include abnormality and the second partial image that is determined to include abnormality And comprising.

  According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and each of the plurality of first partial images and the plurality of second partial images. When it is determined that there is an abnormality, the presence or absence of an abnormality related to the inspection object is determined based on the overlap of the partial images, thereby reducing misjudgment and various possible abnormalities. Even if it exists, the presence or absence of abnormality relating to the inspection object can be detected with high accuracy.

  In the above aspect, when the first partial image that is determined to include an abnormality and the second partial image that is determined to include an abnormality are superimposed, the determination unit includes an abnormality at the overlapping portion. May be determined to exist.

  According to this aspect, not only the presence / absence of an abnormality related to the inspection object but also a location where the abnormality exists can be identified with high accuracy.

  In the above aspect, the determination unit includes a case where the first partial image determined to include an abnormality and a second partial image determined to include no abnormality are superimposed or no abnormality is included. When the first partial image determined to be included and the second partial image determined to include an abnormality are superimposed, the first partial image determined to include the abnormality or the abnormality It may be determined that there is no abnormality in the portion of the second partial image that is determined to be included.

  According to this aspect, when it is determined that an abnormality is included in any one of the first partial image and the second partial image to be superimposed, the first partial image that is determined to include the abnormality. Alternatively, by determining that there is no abnormality at the location of the second partial image, erroneous determination of the presence of abnormality is prevented.

  In the above aspect, the first partial image that is determined to include an abnormality and the first partial image that is determined to include no abnormality are displayed so as to be distinguishable, and is determined to include an abnormality. You may further provide the display part which displays a 2nd partial image and the 2nd partial image determined not to contain abnormality so that distinction is possible.

  According to this aspect, it is possible to visually recognize in which part of the inspection object it is determined that the abnormality is included, and the classification result by the first classification unit and the second classification unit can be easily obtained. Can grasp.

  In the above aspect, the display unit may display the first partial image that is determined to include an abnormality and the second partial image that is determined to include an abnormality so as to be distinguishable.

  According to this aspect, the classification result by the first classification unit and the classification result by the second classification unit are shown to be visually distinguishable, so that the difference between the classification results by the first classification unit and the second classification unit can be determined. It can be easily grasped.

  In the above aspect, each of the plurality of second partial images may be smaller than each of the plurality of first partial images.

  According to this aspect, by determining whether or not an abnormality is included in the plurality of first partial images and the plurality of second partial images smaller than each of the plurality of first partial images, the size and shape of the abnormality Even if there are various cases, it is possible to accurately determine the presence / absence and location of an abnormality.

  In the above aspect, at least two of the plurality of second partial images may overlap with any one of the plurality of first partial images.

  According to this aspect, by determining whether or not an abnormality is included in the first partial image and at least two second partial images superimposed on the first partial image, the size and shape of the abnormality are various. Even if it is a case, the presence or absence and abnormality location of abnormality can be determined accurately.

  In the above aspect, the first dividing unit divides the image so that the plurality of first partial images do not overlap each other, and the second dividing unit divides the image so that the plurality of second partial images do not overlap each other. Also good.

  According to this aspect, it is possible to divide an image without excessively increasing the number of partial images, and to determine the presence or absence of an abnormality of the inspection object. The first classification unit, the second classification unit, and the determination The processing load on the part can be suppressed.

  In the above aspect, the first classifying unit includes a first learned model that has been learned using the first partial image that includes the abnormality and the first partial image that does not include the abnormality as learning data. The classification unit may include a second learned model in which the second partial image including the abnormality and the second partial image including no abnormality are learned as learning data.

  According to this aspect, by using the first learned model and the second learned model learned with different learning data, the first classification unit and the second classification unit classify the presence or absence of abnormality from different viewpoints. can do.

  An inspection method according to another aspect of the present invention includes a step in which a computer divides an image of an inspection object into a plurality of first partial images, and the plurality of second portions different from the plurality of first partial images. Dividing into a plurality of images; classifying the plurality of first partial images into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality; When a plurality of second partial images are classified into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality, and an abnormality is included Determining whether or not there is an abnormality related to the inspection object based on an overlap between the determined first partial image and the second partial image determined to contain an abnormality.

  According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and each of the plurality of first partial images and the plurality of second partial images. When it is determined that there is an abnormality, the presence or absence of an abnormality related to the inspection object is determined based on the overlap of the partial images, thereby reducing misjudgment and various possible abnormalities. Even if it exists, the presence or absence of abnormality relating to the inspection object can be detected with high accuracy.

  An inspection program according to another aspect of the present invention includes a computer, a first division unit that divides an image of an inspection object into a plurality of first partial images, and a plurality of first images different from the plurality of first partial images. A second dividing unit that divides the image into two partial images, and classifies the plurality of first partial images into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality. A first classification unit that classifies a plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality Inspection object based on the second classification part different from the part and the overlap of the first partial image determined to contain the abnormality and the second partial image determined to contain the abnormality It functions as a determination unit that determines whether or not there is an abnormality related to the.

  According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and each of the plurality of first partial images and the plurality of second partial images. When it is determined that there is an abnormality, the presence or absence of an abnormality related to the inspection object is determined based on the overlap of the partial images, thereby reducing misjudgment and various possible abnormalities. Even if it exists, the presence or absence of abnormality relating to the inspection object can be detected with high accuracy.

  According to the present invention, it is possible to provide an inspection apparatus, an inspection method, and an inspection program that can accurately detect the presence or absence of an abnormality related to an inspection target even if there are various abnormalities that may occur.

It is a figure which shows the network structure of the test | inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the physical structure of the inspection apparatus which concerns on embodiment of this invention. It is a functional block diagram of the inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the image of the test target object acquired by the test | inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the 1st partial image divided | segmented by the test | inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the 2nd partial image divided | segmented by the test | inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the 3rd partial image divided | segmented by the test | inspection apparatus which concerns on embodiment of this invention. It is a figure which shows the image of the test target object displayed on the display part of the test | inspection apparatus which concerns on embodiment of this invention. It is a flowchart of the 1st process performed by the test | inspection apparatus which concerns on embodiment of this invention. It is a flowchart of the 2nd process performed by the test | inspection apparatus which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in each figure, what attached | subjected the same code | symbol has the same or similar structure.

  FIG. 1 is a diagram showing a network configuration of an inspection apparatus 10 according to an embodiment of the present invention. The inspection apparatus 10 is a dedicated or general-purpose computer, and may be a general-purpose computer in which the inspection program according to the present embodiment is installed, for example. The inspection apparatus 10 is connected to the learning image database DB and the camera 30 via the communication network N. The communication network N is a wired or wireless communication network, and may be a LAN (Local Area Network) or the Internet, for example.

  The inspection apparatus 10 acquires an image of the inspection object 100 photographed by the camera 30 via the communication network N, and determines whether or not the inspection object 100 is abnormal based on the learned model. Here, the learned model is, for example, a learned neural network. The inspection device 10 learns a model using an image stored in the learning image database DB.

  The camera 30 captures an image of the inspection object 100. All or a part of the images photographed by the camera 30 is stored in the learning image database DB. The camera 30 may be a general digital camera that captures visible light, but may also be a camera that captures infrared rays, X-rays, or the like. It may be a camera for photographing.

  FIG. 2 is a diagram illustrating a physical configuration of the inspection apparatus 10 according to the embodiment of the present invention. The inspection apparatus 10 includes a CPU (Central Processing Unit) 10a corresponding to a hardware processor, a RAM (Random Access Memory) 10b corresponding to a memory, a ROM (Read Only Memory) 10c corresponding to a memory, and a communication interface 10d. , An input unit 10e and a display unit 10f. These components are connected to each other via a bus so that data can be transmitted and received.

  The CPU 10a is a control unit that performs control related to execution of a program stored in the RAM 10b or the ROM 10c, and calculates and processes data. The CPU 10a is an arithmetic device that executes a program (inspection program) related to inspection of an inspection object. The CPU 10a receives various input data from the input unit 10e and the communication interface 10d, and displays the calculation result of the input data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.

  The RAM 10b is a storage unit in which data can be rewritten, and is composed of, for example, a semiconductor storage element. The RAM 10b stores programs such as applications executed by the CPU 10a and data.

  The ROM 10c is a storage unit that can only read data, and is composed of, for example, a semiconductor storage element. The ROM 10c stores programs such as firmware and data, for example.

  The communication interface 10d is an interface that connects the inspection apparatus 10 to the communication network N.

  The input unit 10e receives data input from the user, and includes, for example, a keyboard, a mouse, and a touch panel.

  The display unit 10f visually displays the calculation result by the CPU 10a, and is configured by an LCD (Liquid Crystal Display), for example.

  The inspection program may be provided by being stored in a computer-readable storage medium such as the RAM 10b or the ROM 10c, or may be provided via the communication network N connected by the communication interface 10d. In the inspection apparatus 10, various functions described with reference to the following drawings are realized by the CPU 10 a executing the inspection program. In addition, these physical structures are illustrations, Comprising: It does not necessarily need to be an independent structure. For example, the inspection apparatus 10 may include an LSI (Large-Scale Integration) in which a CPU 10a, a RAM 10b, and a ROM 10c are integrated. Further, the inspection apparatus 10 may include an arithmetic circuit such as a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

  FIG. 3 is a functional block diagram of the inspection apparatus 10 according to the embodiment of the present invention. The inspection apparatus 10 includes an acquisition unit 11, a division unit 12, a classification unit 13, a determination unit 14, and a learning unit 16.

  The acquisition unit 11 acquires an image of the inspection object 100 from the camera 30 or the learning image database DB via the communication network N.

  The dividing unit 12 includes a first dividing unit 12a, a second dividing unit 12b, and a third dividing unit 12c. The first dividing unit 12a divides the image of the inspection object 100 into a plurality of first partial images. The second dividing unit 12b divides the image of the inspection object 100 into a plurality of second partial images different from the plurality of first partial images. The third dividing unit 12c divides the image of the inspection object 100 into a plurality of third partial images that are different from any of the plurality of first partial images and the plurality of second partial images.

  Each of the plurality of second partial images may be smaller than each of the plurality of first partial images. Similarly, each of the plurality of third partial images may be smaller than each of the plurality of second partial images. In addition, at least two of the plurality of second partial images may be superimposed on any one of the plurality of first partial images. Similarly, at least two of the plurality of third partial images may be superimposed on any one of the plurality of second partial images.

  The first dividing unit 12a may divide the image of the inspection object 100 so that the plurality of first partial images do not overlap each other. However, the first dividing unit 12a may divide the image of the inspection object 100 so that all or some of the plurality of first partial images overlap each other. The same applies to the second dividing unit 12b and the third dividing unit 12c.

  More specifically, the dividing unit 12 according to the present embodiment divides the image of the inspection object 100 into rectangular partial images. Here, the long side and short side of the first partial image are longer than the long side and short side of the second partial image, respectively, and the long side and short side of the second partial image are respectively long side and short side of the third partial image. It may be longer than the short side. But the division part 12 may divide | segment the image of the test object 100 into the partial images which have shapes other than a rectangle, for example, shapes, such as a polygon and an ellipse. The dividing unit 12 according to the present embodiment divides the image of the inspection object 100 into three types of partial images, but the dividing unit 12 divides the image of the inspection object 100 into at least two types of partial images. What is necessary is just to divide | segment into four or more types of partial images.

  The classification unit 13 includes a first classification unit 13a, a second classification unit 13b, and a third classification unit 13c. The first classification unit 13a classifies the plurality of first partial images into a first partial image that is determined to include abnormality and a first partial image that is determined to include no abnormality. The second classification unit 13b classifies the plurality of second partial images into a second partial image that is determined to include abnormality and a second partial image that is determined to include no abnormality. The third classification unit 13c classifies the plurality of third partial images into a third partial image that is determined to include an abnormality and a third partial image that is determined to include no abnormality.

  The first classification unit 13a may include a first learned model in which a first partial image including abnormality and a first partial image not including abnormality are learned as learning data. Further, the second classification unit 13b may include a second learned model in which a second partial image including an abnormality and a second partial image including no abnormality are learned as learning data. Similarly, the third classification unit 13c may include a third learned model in which a third partial image that includes an abnormality and a third partial image that does not include an abnormality are learned as learning data. Here, the first learned model, the second learned model, and the third learned model may be learned models obtained by learning a common model with different learning data, or different models may be different learning data. It may be a learned model learned in (1). The first learned model, the second learned model, and the third learned model may be, for example, a learned neural network, but may be any model such as a support vector machine.

  Note that the classification unit 13 may not include the learned model, or may classify whether or not the partial image includes abnormality by a predetermined method. For example, the classification unit 13 calculates an evaluation value based on a partial image, and compares the evaluation value with a threshold value to classify a partial image that includes an abnormality and a partial image that does not include the abnormality. It may be.

  The learning unit 16 acquires the first partial image, the second partial image, and the third partial image for learning from the learning image database DB, and the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. To learn each. Here, learning is to improve the classification accuracy of the classification unit 13 by inputting learning data to the classification unit 13 and correcting the configuration of the classification unit 13 based on the correctness of the output result. Say. Specifically, when the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c include a neural network, the learning unit 16 uses the first partial image and the second partial image for learning as learning data. And the third partial image can be input to the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c, respectively, so that the partial image including the abnormality and the partial image not including the abnormality can be correctly classified. Based on the value for evaluating whether or not, the weighting of the units included in the neural network is optimized so as to improve the evaluation value.

  The determination unit 14 determines whether or not there is an abnormality related to the inspection object 100 based on the overlap between the first partial image that is determined to include abnormality and the second partial image that is determined to include abnormality. Determine. The determination unit 14 may determine the presence / absence of an abnormality related to the inspection object 100 based on the overlap of at least two types of partial images, but the first partial image determined to include an abnormality and the abnormality Based on the overlap between the second partial image that is determined to be included and the third partial image that is determined to be abnormal, the presence / absence of an abnormality related to the inspection object 100 may be determined.

  When the first partial image that is determined to include an abnormality and the second partial image that is determined to include an abnormality are superimposed, the determination unit 14 has an abnormality at the overlapping portion. You may judge. The determination unit 14 may determine that an abnormality exists in the superimposed portion when a partial image including abnormality is superimposed on at least two types of partial images, but is determined to include abnormality. The first partial image, the second partial image that is determined to contain an abnormality, and the third partial image that is determined to contain an abnormality are superimposed on each other. You may determine that it exists.

  In addition, the determination unit 14 superimposes the first partial image determined to include an abnormality and the second partial image determined to include no abnormality or no abnormality. When the first partial image to be determined overlaps with the second partial image to be determined to include an abnormality, the first partial image to be determined to include the abnormality or the abnormality is included. It may be determined that there is no abnormality in the portion of the second partial image that is determined to be. The determination unit 14 includes an abnormality when at least two types of superimposed partial images include an abnormality in one of the partial images and the other partial image does not include the abnormality. It may be determined that there is no abnormality in the portion determined to be, but for any of the three or more types of partial images to be superimposed, any one type of partial image includes an abnormality, but other types of partial images If no abnormality is included in the case, it may be determined that there is no abnormality at a place where it is determined that the abnormality is included.

  The display unit 10f displays the first partial image that is determined to include an abnormality and the first partial image that is determined to include no abnormality so as to be distinguishable, and is determined to include an abnormality. The second partial image and the second partial image that is determined not to include an abnormality are displayed so as to be distinguishable, and an image of the inspection object 100 is displayed. In addition, the display unit 10f displays the third partial image that is determined to include an abnormality and the third partial image that is determined to include no abnormality so as to be distinguishable. May be displayed. Further, the display unit 10f displays the first partial image that is determined to include an abnormality and the second partial image that is determined to include an abnormality so as to be distinguishable. The first partial image that is determined to be present and the second partial image that is determined to include the abnormality may be displayed in a distinguishable manner from the third partial image that is determined to include the abnormality.

  FIG. 4 is a diagram showing an image P of the inspection object 100 acquired by the inspection apparatus 10 according to the embodiment of the present invention. The image P is an image obtained by photographing the inspection object 100 from a predetermined direction. In this example, the image P indicates that the inspection object 100 has a scratch S, and the inspection object 100 has an abnormality. In this example, a case where it is determined whether there is an abnormality in the inspection target object 100 using one image P will be described. However, the inspection apparatus 10 performs an inspection based on a plurality of images related to the inspection target object 100. You may determine whether the target object 100 has abnormality.

  FIG. 5 is a diagram showing the first partial image P1 divided by the inspection apparatus 10 according to the embodiment of the present invention. In the present example, the first dividing unit 12a divides the image P into 16 first partial images P1. The first partial images P1 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of first partial images P1 do not overlap each other, the presence or absence of abnormality of the inspection object 100 is determined without excessively increasing the number of the plurality of first partial images P1. And the processing load on the first classification unit 13a can be suppressed.

  The first classification unit 13a classifies the 16 first partial images P1 into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality. In the case of this example, the first classification unit 13a determines that one of the 16 first partial images P1 includes an abnormality, and adds the other 15 first partial images to the first partial image P1. Determines that no abnormality is included.

  The display unit 10f displays the first partial image determined to contain an abnormality and the first partial image determined to contain no abnormality so as to be distinguishable. In the case of this example, the display unit 10f highlights the first partial image P1a determined to include an abnormality by first type hatching, and determines that the abnormality is not included in the first partial image P1a. The partial image P1 is indicated by a broken line. Thereby, it is possible to visually recognize at which part of the inspection object 100 it is determined that the abnormality is included, and it is possible to easily grasp the classification result by the first classification unit 13a. . In addition, the aspect which displays so that the 1st partial image determined that abnormality is contained by the display part 10f and the 1st partial image determined that abnormality is not included is arbitrary, and abnormality is included. Alternatively, the hatched display of the first partial image P1a determined to be transparent may be subjected to a transparent process so that the scratch S can be visually recognized.

  FIG. 6 is a diagram showing the second partial image P2 divided by the inspection apparatus 10 according to the embodiment of the present invention. In this example, the second dividing unit 12b divides the image P into 25 second partial images P2. The second partial image P2 is a rectangular image and does not overlap each other. By dividing the image P so that the plurality of second partial images P2 do not overlap each other, the presence or absence of abnormality of the inspection object 100 is determined without excessively increasing the number of the plurality of second partial images P2. And the processing load on the second classification unit 13b can be suppressed.

  The second partial image P2 is smaller than the first partial image P1, and at least two of the second partial images P2 overlap with any one of the plurality of first partial images P1. This is a case where the size and shape of the abnormality are various by determining whether or not the abnormality is included in the first partial image P1 and at least two second partial images P2 superimposed on the first partial image P1. However, the presence / absence and presence of abnormality can be determined with high accuracy.

  The second classification unit 13b classifies the 25 second partial images P2 into a second partial image that is determined to include abnormality and a second partial image that is determined to include no abnormality. In the case of this example, the second classification unit 13b determines that an abnormality is included in three second partial images out of the 25 second partial images P2, and includes the other 22 second partial images. Determines that no abnormality is included.

  The display unit 10f displays the second partial image determined to include an abnormality and the second partial image determined to include no abnormality so as to be distinguishable. In the case of this example, the display unit 10f shows the second partial images P2a, P2b, and P2c, which are determined to include an abnormality, as highlighted by the second type of hatching, and determines that no abnormality is included. The second partial image P2 to be performed is indicated by a broken line. Thereby, it is possible to visually recognize in which part of the inspection object 100 it is determined that the abnormality is included, and it is possible to easily grasp the classification result by the second classification unit 13b. .

  In addition, the display unit 10f displays the first partial image P1a that is determined to include an abnormality and the second partial images P2a, P2b, and P2c that are determined to include an abnormality so as to be distinguishable. . Specifically, the display unit 10f indicates the first partial image P1a that is determined to include an abnormality with the first type of hatching, but the second partial image that is determined to include an abnormality. P2a, P2b, and P2c are indicated by the second type of hatching.

  The second partial images P2b and P2c that are determined to include an abnormality by the second classification unit 13b do not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality at the location of the second partial images P2b and P2c that are determined to include the abnormality. The presence or absence of abnormality can be correctly determined.

  FIG. 7 is a diagram showing a third partial image P3 divided by the inspection apparatus 10 according to the embodiment of the present invention. In this example, the third dividing unit 12c divides the image P into 64 third partial images P3. The third partial image P3 is a rectangular image and does not overlap each other. By dividing the image P so that the plurality of third partial images P3 do not overlap each other, the presence or absence of abnormality of the inspection object 100 is determined without excessively increasing the number of the plurality of third partial images P3. And the processing load of the third classification unit 13c can be suppressed.

  Further, the third partial image P3 is smaller than the first partial image P1 and the second partial image P2, and at least two of the third partial images P3 are superimposed on any one of the plurality of first partial images P1. And it superimposes with any one among several 2nd partial images P2. By determining whether or not an abnormality is included in the first partial image P1 and the second partial image P2 and at least two third partial images P3 superimposed on the first partial image P1 and the second partial image P2, Even when there are various sizes and shapes of abnormalities, it is possible to accurately determine the presence / absence and location of the abnormality.

  The third classification unit 13c classifies the 64 third partial images P3 into a third partial image that is determined to include abnormality and a third partial image that is determined to include no abnormality. In the case of this example, the third classification unit 13c determines that the abnormality is included in the four third partial images out of the 64 third partial images P3, and includes the other 60 third partial images. Determines that no abnormality is included.

  The display unit 10f displays the third partial image determined to include an abnormality and the third partial image determined to include no abnormality so as to be distinguishable. In the case of this example, the display unit 10f shows the third partial images P3a, P3b, P3c, and P3d that are determined to include an abnormality by highlighting them with the third type of hatching, and no abnormality is included. The third partial image P <b> 3 determined to be indicated by a broken line. Here, the display unit 10f includes the first partial image P1a that is determined to include abnormality, the second partial images P2a, P2b, and P2c that are determined to include abnormality, and the abnormality. The third partial images P3a, P3b, P3c, and P3d that are determined to be present are displayed in a distinguishable manner. Specifically, the display unit 10f indicates the first partial image P1a that is determined to include an abnormality with the first type of hatching, and the second partial image P2a that is determined to include the abnormality. P2b and P2c are indicated by the second type of hatching, but third partial images P3a, P3b, P3c, and P3d that are determined to include an abnormality are indicated by the third type of hatching.

  The third partial image P3d that is determined to include an abnormality by the third classification unit 13c does not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality in the portion of the third partial image P3d that is determined to include abnormality, The presence or absence of can be correctly determined.

  FIG. 8 is a diagram illustrating an image P of the inspection object 100 displayed on the display unit 10f of the inspection apparatus 10 according to the embodiment of the present invention. The display unit 10f includes a first partial image P1a that is determined to include an abnormality, second partial images P2a, P2b, and P2c that are determined to include an abnormality, and an abnormality. The determined third partial images P3a, P3b, P3c, and P3d are displayed in a distinguishable manner. The classification result by the first classification unit 13a, the classification result by the second classification unit 13b, and the classification result by the third classification unit 13c are shown to be visually distinguishable, so that the first classification unit 13a and the second classification unit It is possible to easily grasp the difference between the classification results by the unit 13b and the third classification unit 13c.

  The determination unit 14 of the inspection apparatus 10 includes the first partial image P1a that is determined to include abnormality, the second partial images P2a, P2b, and P2c that are determined to include abnormality, and the abnormality. Based on the overlap with the third partial images P <b> 3 a, P <b> 3 b, P <b> 3 c, and P <b> 3 d that are determined to be detected, it is determined whether there is an abnormality related to the inspection object 100.

  According to the inspection apparatus 10 according to the present embodiment, the image P is divided into a plurality of first partial images P1, a plurality of second partial images P2, and a plurality of third partial images P3. When it is determined that an abnormality is included in each of the second partial image and the third partial image, erroneous determination is reduced by determining whether there is an abnormality related to the inspection object based on the overlap of the partial images. Thus, even if there are various possible abnormalities, it is possible to accurately detect the presence / absence of an abnormality related to the inspection object.

  In the case of this example, the first partial image P1a determined to include an abnormality, the second partial image P2a determined to include an abnormality, and the first partial image P2a determined to include an abnormality. Since the three partial images P3a, P3b, and P3c are overlapped, the determination unit 14 determines that there is an abnormality in the overlapping portion R. In this way, by determining that there is an abnormality at the overlapping part of multiple types of partial images that are determined to contain an abnormality, not only the presence / absence of the abnormality related to the inspection object but also the location where the abnormality exists is accurate. Can be identified well.

  In addition, the determination unit 14 determines that the second partial images P2b and P2c that are determined to include an abnormality and the third partial image P3d that is determined to include an abnormality do not include any abnormality. The second partial images P2b and P2c that are determined to include an abnormality or the third partial image P3d that is determined to include an abnormality are superimposed on other types of partial images that are included. It is determined that there is no abnormality. In this way, it is determined that an abnormality is included in any of the first partial image, the second partial image, and the third partial image to be superimposed, and it is determined that no abnormality is included in other types of partial images. If it is determined that there is no abnormality in the location of the first partial image, the second partial image, or the third partial image that is determined to include the abnormality, an erroneous determination of the presence of the abnormality Is prevented.

  Furthermore, each of the plurality of second partial images P2 is smaller than each of the plurality of first partial images P1, and each of the plurality of third partial images P3 is smaller than each of the plurality of second partial images P2. Even when there are various sizes and shapes, it is possible to accurately determine the presence / absence and location of an abnormality.

  FIG. 9 is a flowchart of the first process executed by the inspection apparatus 10 according to the embodiment of the present invention. In the first process, the image P of the inspection object 100 is divided into a first partial image P1, a second partial image P2, and a third partial image P3, and the presence / absence of abnormality is determined. This is a process for determining the presence or absence of abnormality related to the inspection object 100 based on the overlap of the partial images determined to be.

  The inspection apparatus 10 first acquires an image P of the inspection object 100 (S10). Then, the image P is divided into a plurality of first partial images P1 by the first dividing unit 12a (S11), and the image P is divided into a plurality of second partial images P2 by the second dividing unit 12b (S12). Is divided into a plurality of third partial images P3 by the third dividing unit 12c (S13).

  Next, the first classification unit 13a classifies the plurality of first partial images P1 into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality. (S14) The second classification unit 13b then determines that the plurality of second partial images P2 are a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality. (S15), and the third classifying unit 13c determines that the plurality of third partial images P3 are determined to contain no abnormality and the third partial image that is judged to contain abnormality. Classification into partial images (S16).

  Thereafter, the display unit 10f includes a first partial image that is determined to include abnormality, a second partial image that is determined to include abnormality, and a third portion that is determined to include abnormality. The images are displayed in a distinguishable manner (S17).

  The determination unit 14 includes a first partial image that is determined to include abnormality, a second partial image that is determined to include abnormality, and a third portion that is determined to include abnormality. It is determined whether or not the image is superimposed (S18). When it is determined that these three types of partial images are to be superimposed (S18: Yes), the determination unit 14 determines that there is an abnormality at the overlapping portion (S19). On the other hand, when it is determined that these three types of partial images do not overlap (S18: No), that is, when it is determined that no abnormality is included in at least one type of partial image, the determination unit 14 determines that there is an abnormality. It is determined that there is no abnormality in the portion of the partial image determined to be included (S20). Thus, the first process ends.

  FIG. 10 is a flowchart of the second process executed by the inspection apparatus 10 according to the embodiment of the present invention. The second process is a process executed prior to the first process, and the partial image including the abnormality and the abnormality are included in the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. This is a process for learning the classification of partial images that are not included.

  The learning unit 16 of the inspection apparatus 10 acquires a first partial image for learning from the learning image database DB (S30). Similarly, the learning unit 16 acquires the second partial image for learning from the learning image database DB (S31), and acquires the third partial image for learning (S32). Here, the first partial image for learning includes a first partial image that includes abnormality and a first partial image that does not include abnormality, and each first partial image is tagged with the presence or absence of abnormality. Has been. Further, the second partial image for learning includes a second partial image that includes abnormality and a second partial image that does not include abnormality, and each second partial image is tagged with the presence or absence of abnormality. ing. Similarly, the third partial image for learning includes a third partial image that includes abnormality and a third partial image that does not include abnormality, and each third partial image is tagged with the presence or absence of abnormality. Has been.

  The learning unit 16 performs learning of the first classification unit 13a using the first partial image for learning (S33), and performs learning of the second classification unit 13b using the second partial image for learning ( S34), the learning of the third classification unit 13c is performed using the third partial image for learning (S35). Specifically, when the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c include a neural network, the learning unit 16 uses the first partial image, the second partial image, and the third part for learning. The images are input to the first classifying unit 13a, the second classifying unit 13b, and the third classifying unit 13c, respectively, and it is evaluated whether or not the partial image including the abnormality and the partial image including no abnormality are correctly classified. Based on the value, the weights of the units included in the neural network are optimized so as to improve the evaluation value.

  Thus, the second process ends, the first classification unit 13a includes the first learned model, the second classification unit 13b includes the second learned model, and the third classification unit 13c The third learned model will be included. Here, the first learned model, the second learned model, and the third learned model have different learning data (a first partial image for learning, a second partial image for learning, and a third part for learning). It may be a model having a different configuration learned using (image). Specifically, the first classification unit 13a includes a first learned neural network, the second classification unit 13b includes a second learned neural network having a configuration different from that of the first learned neural network, and a third classification. The unit 13c may include a third learned neural network having a different configuration from both the first learned neural network and the second learned neural network.

  According to the inspection apparatus 10 according to the present embodiment, the first classification unit 13a, the first learned model, the second learned model, and the third learned model that are learned with different learning data are used. The presence / absence of abnormality can be classified from different viewpoints in the second classification unit 13b and the third classification unit 13c.

  The embodiments described above are for facilitating the understanding of the present invention, and are not intended to limit the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those illustrated, and can be changed as appropriate. In addition, the structures shown in different embodiments can be partially replaced or combined.

Claims (11)

A first dividing unit that divides the entire image of the inspection object into a plurality of first partial images;
A second dividing unit that divides the entire image into a plurality of second partial images different from the plurality of first partial images;
A first classification unit that classifies the plurality of first partial images into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality;
A second classification unit that classifies the plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality;
Based on the overlap between the first partial image determined to include the abnormality and the second partial image determined to include the abnormality, the presence / absence of abnormality related to the inspection object is determined. A determination unit ;
The plurality of first partial images and the plurality of second partial images are rectangles each having a first side extending in a first direction and a second side extending along a second direction intersecting with the first direction. The first side and the second side of the plurality of first partial images are longer than the first side and the second side of the plurality of second partial images, respectively.
Inspection equipment. When the first partial image that is determined to include the abnormality and the second partial image that is determined to include the abnormality are superimposed, the determination unit has an abnormality at the overlapping portion. Determine that it exists,
The inspection apparatus according to claim 1. The determination unit includes a case where the first partial image determined to include the abnormality and a second partial image determined to not include the abnormality or the abnormality is not included The first partial image that is determined to include the abnormality and the second partial image that is determined to include the abnormality overlap the first partial image that is determined to include the abnormality or the abnormality It is determined that there is no abnormality in the portion of the second partial image that is determined to include
The inspection apparatus according to claim 1 or 2. The first partial image that is determined to contain the abnormality and the first partial image that is judged to contain no abnormality are displayed in a distinguishable manner, and the first partial image that is determined to contain the abnormality is displayed. A display unit for displaying the image, wherein the two partial images and the second partial image determined not to contain the abnormality are displayed in a distinguishable manner;
The inspection apparatus according to any one of claims 1 to 3. The display unit displays the first partial image determined to include the abnormality and the second partial image determined to include the abnormality in a distinguishable manner.
The inspection apparatus according to claim 4. Each of the plurality of second partial images is smaller than each of the plurality of first partial images.
The inspection apparatus according to any one of claims 1 to 5. At least two of the plurality of second partial images are superposed on any one of the plurality of first partial images;
The inspection apparatus according to any one of claims 1 to 6. The first dividing unit divides the image so that the plurality of first partial images do not overlap each other,
The second dividing unit divides the images so that the plurality of second partial images do not overlap each other.
The inspection apparatus according to any one of claims 1 to 7. The first classifying unit includes a first learned model that is learned as learning data using a first partial image that includes abnormality and a first partial image that does not include abnormality,
The second classification unit includes a second learned model in which a second partial image including abnormality and a second partial image not including abnormality are learned as learning data.
The inspection apparatus according to any one of claims 1 to 8. Computer
Dividing the entire image of the inspection object into a plurality of first partial images;
Dividing the entire image into a plurality of second partial images different from the plurality of first partial images;
Classifying the plurality of first partial images into a first partial image determined to include an abnormality and a first partial image determined to include no abnormality;
Classifying the plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality;
Based on the overlap between the first partial image determined to include the abnormality and the second partial image determined to include the abnormality, the presence / absence of abnormality related to the inspection object is determined. And including steps ,
The plurality of first partial images and the plurality of second partial images are rectangles each having a first side extending in a first direction and a second side extending along a second direction intersecting with the first direction. The first side and the second side of the plurality of first partial images are longer than the first side and the second side of the plurality of second partial images, respectively.
Test査方method. Computer
A first dividing unit that divides the entire image of the inspection object into a plurality of first partial images;
A second dividing unit that divides the entire image into a plurality of second partial images different from the plurality of first partial images;
A first classification unit that classifies the plurality of first partial images into a first partial image that is determined to include an abnormality and a first partial image that is determined to include no abnormality;
Different from the first classification unit that classifies the plurality of second partial images into a second partial image that is determined to include an abnormality and a second partial image that is determined to include no abnormality. A second classification unit, and a first partial image that is determined to include the abnormality, and a second partial image that is determined to include the abnormality. Function as a determination unit that determines the presence or absence of abnormality ,
The plurality of first partial images and the plurality of second partial images are rectangles each having a first side extending in a first direction and a second side extending along a second direction intersecting with the first direction. The first side and the second side of the plurality of first partial images are longer than the first side and the second side of the plurality of second partial images, respectively.
Inspection program.

Images