JP2019200558A - Image processing device and inspection system - Google Patents

Abstract

To provide an image processing device capable of easily generating learning data.SOLUTION: An image processing device for generating learning data which causes an artificial intelligence program to perform machine learning, the artificial intelligence program autonomously determining whether or not a feature corresponding to a feature of a target exists in an inspection target. The image processing device includes an annotation applying unit that selects, as the feature of the target, an arbitrary portion of the target in a target model image obtained by taking in the target as an image, and applies an annotation to the portion. The annotation applying unit applies an annotation to the target model image, whereby learning data is generated from the target model image.SELECTED DRAWING: Figure 2

Description

  The present invention generates learning data that causes machine learning to be performed by an artificial intelligence program that autonomously determines whether or not a feature that matches the feature of an object exists in the image processing device and the inspection system. The present invention relates to an image processing apparatus and an inspection system that inspects an inspection object by determining whether or not a characteristic that matches the characteristic of the object exists in the inspection object.

  When image processing is performed by software, it is necessary to select an area for processing. For example, Patent Document 1 proposes an image display device including a selection unit that selects an arbitrary position on image data displayed on an image display unit.

  In this type of image display device, as a user interface when the user selects an arbitrary position on the image data, for example, a so-called lasso display method that surrounds the area to be processed with an outline is adopted. There is.

JP-A-11-306374

  By the way, when learning the characteristics of an object using artificial intelligence technology and selecting an object based on the characteristics, for example, the artificial intelligence program uses the image data of the object. It is necessary to learn the characteristics of

  As described above, when the artificial intelligence program learns the feature of the target object using the image data of the target object, it is necessary to select a part having the feature of the target object displayed in the image data on the image data.

  On the other hand, in order to make the artificial intelligence program learn the characteristics of the target object, in general, a large amount of image data is required. Therefore, the task of selecting the characteristic part of the target object displayed in the image data is complicated. It is assumed that

  The present invention has been made in view of the above circumstances, and when an artificial intelligence program learns the characteristics of an object using image data, a portion having the characteristics of the object is easily selected on the image data. An object is to provide an image processing apparatus capable of generating learning data, and an inspection system including an artificial intelligence program capable of efficiently performing machine learning based on learning data generated by the image processing apparatus. To do.

  An image processing apparatus for solving the above-described problem is an image that generates learning data that causes an artificial intelligence program that autonomously determines whether or not a feature that matches the feature of an object exists in the inspection object to perform machine learning The processing apparatus includes an annotation adding unit that selects an arbitrary part of the target in the target model image in which the target is captured as an image as a feature of the target and adds an annotation. It is characterized in that learning data is generated from the object model image by giving an annotation to the object model image.

  According to this image processing apparatus, an arbitrary part of an object captured as an image in an object model image input in a large amount to an artificial intelligence program by a simple operation of giving an annotation as a feature of the object You can choose.

  Moreover, since the object model image can be used as learning data by giving an annotation to the object model image, the learning data can be easily generated by a simple operation of adding the annotation.

  An inspection system for solving the above-described problem is an inspection system for inspecting an inspection object by determining whether or not the inspection object has a feature that matches the feature of the object. A display unit for displaying the target object model image, and an annotation adding unit for selecting an arbitrary part of the target object captured as an image in the target object model image displayed on the display unit as a feature of the target object and giving an annotation And an object that has been learned as a learned model based on a learned model that has been machine-learned by learning data generated from the object model image by annotating the object model image by the annotation assigning unit. Artificial intelligence program that inspects the inspection object by autonomously judging whether the inspection object has a feature that matches the feature It is characterized in that it comprises, when.

  According to this inspection system, since the learning data can be easily generated by a simple operation of adding an annotation to the object model image, the artificial intelligence program can efficiently perform machine learning. Therefore, the workability at the time of inspecting the inspection object can be improved.

  Further, in this inspection system, the inspection result of the inspection object by the artificial intelligence program is displayed on the display unit based on the preset display standard, and the inspection object corresponding to the display standard changed by changing the display standard It is characterized by including a display reference changing unit for displaying the result of the inspection on the display unit.

  In this way, since the preset display standard can be changed by the display standard changing unit, the inspection result of the inspection object can be displayed on the display unit using the desired display standard.

  In addition, the display standard changing unit of the inspection system is characterized in that the upper and lower limits of the display standard can be set independently of each other, so that the operability when changing the display standard is improved. .

  According to the present invention, learning data can be easily generated by a simple operation of giving an annotation to an object model image. Furthermore, the artificial intelligence program can perform machine learning efficiently.

It is a figure explaining the outline of the target object and inspection object used with the inspection system concerning an embodiment of the invention. It is a block diagram explaining the outline of the inspection system concerning an embodiment of the invention. Similarly, it is a figure explaining the outline of the object model image which concerns on this Embodiment, and the outline of the process in an annotation provision part. Similarly, it is a block diagram explaining the outline of the process of the artificial intelligence program which concerns on this Embodiment. Similarly, it is a figure explaining the outline of a process of the display reference change part which concerns on this Embodiment. Similarly, it is a figure explaining the outline of a process of the display reference change part which concerns on this Embodiment.

  Next, an embodiment of the present invention will be described with reference to FIGS.

  Prior to the description of the outline of the inspection system according to the present embodiment, first, the outline of the object used in the inspection system according to the present embodiment and the inspection object will be described.

  FIG. 1A is a diagram illustrating an outline of an object, and FIG. 1B is a diagram illustrating an outline of an inspection object. As shown in FIG. 1 (a), the object W1 is a tissue T1 collected from a human body in the present embodiment, and this tissue T1 obtained as a pathological specimen by preparation is captured as an image. An object model image D1 is constructed.

  A large amount of the object model image D1 is generated in advance as a plurality of image data based on a plurality of tissues T1 collected from a plurality of human bodies.

  In the tissue T1 of the pathological specimen captured as an image in the object model image D1, in the present embodiment, there are a plurality of abnormal cells C that are not normal cells as features of the tissue T1.

  On the other hand, as shown in FIG. 1 (b), the inspection object W2 is a tissue T2 collected from a human body in the present embodiment, and the tissue T2 prepared as a pathological specimen by a preparation is captured as an image. Thus, the inspection object image D5 is formed.

  The tissue T2 captured as an image in the inspection object image D5 is an inspection target by the inspection system. For example, when a cancer diagnosis is performed, the tissue T2 is extracted from the affected part of the person receiving the diagnosis. The organization T2 is configured.

  In the present embodiment, a plurality of abnormal cells C that are not normal cells exist as features of the tissue T2 in the tissue T2 of the pathological specimen captured as an image in the inspection object image D5.

  Next, an outline of the inspection system according to the present embodiment will be described.

  FIG. 2 is a block diagram for explaining the outline of the inspection system according to the present embodiment. As illustrated, the inspection system 1 includes a computer 10, a scanner 11 connected to the computer 10, and a display 12 that is a display unit connected to the computer 10.

  The computer 10 includes an image processing device 20 to which an object model image D1 is input via a scanner 11 described later, an artificial intelligence program 21 connected to the image processing device 20, and an artificial intelligence program 21 and the image processing device 20. A display function setting unit 22 to be connected is provided.

  In the present embodiment, the image processing apparatus 20 includes an image acquisition unit 20a that acquires the object model image D1 and outputs the object model image D1 to the artificial intelligence program 21 and the display function setting unit 22.

  The object model image D1 acquired by the image acquisition unit 20a and output to the display function setting unit 22 is displayed on the display 12 in the present embodiment.

  Furthermore, the image processing apparatus 20 selects an annotation adding unit 20b which is a program for selecting an arbitrary part of the target in the image data captured in advance as an image in the target model image D1 as a feature of the target and giving an annotation. Prepare.

  The annotation giving unit 20b gives an annotation to the object model image D1 output from the image acquisition unit 20a to the artificial intelligence program 21, and is acquired by the image acquisition unit 20a and output to the display function setting unit 22 for display. An annotation is displayed on the object model image D1 displayed on the screen.

  FIG. 3 is a diagram for explaining the outline of processing in the annotation assigning unit 20b. In the present embodiment, as shown in the figure, it is determined that all of the plurality of abnormal cells C present as features of the tissue T1 of the pathological specimen captured as an image in the object model image D1 are cancer cells, Annotation D2 is assigned by the annotation assignment unit 20b.

  The annotation D2 is assigned to the object output from the image acquisition unit 20a to the artificial intelligence program 21 while the object model image D1 displayed on the display 12 is observed via the image acquisition unit 20a and the display function setting unit 22. The model image D1 is given by operating an external input device such as a mouse or a pointing device.

  When the annotation D2 is given, the annotation D2 to be given can be weighted by selecting a weighting command (not shown).

  The artificial intelligence program 21 is a program for inspecting the tissue T2 in the inspection object image D5 input through the scanner 11.

  FIG. 4 is a block diagram for explaining the outline of the process of the artificial intelligence program 21. As shown in the figure, the artificial intelligence program 21 generates learning data D3 from the object model image D1 by giving annotations D2 to a plurality of abnormal cells C existing as features of the tissue T1 in the object model image D1. Is done.

  Further, the artificial intelligence program 21 generates a learned model D4 by performing machine learning with the generated learning data D3.

  As a method for performing machine learning, various algorithms such as a neural network, a random forest, and an SVM (support vector machine) are appropriately used.

  As described above, the artificial intelligence program 21 detects, based on the learned model D4, abnormal cells C that match the plurality of abnormal cells C in the tissue T1 in the object model image D1 learned as the learned model D4. Whether or not the object image D5 exists in the tissue T2 is autonomously determined, and the tissue T2 is inspected.

  As shown in FIG. 2, the display function setting unit 22 is connected to the image acquisition unit 20a and the artificial intelligence program 21 and connected to the display 12, the display unit setting unit 22b connected to the output unit 22a, and the display reference setting unit 22b. A display reference changing unit 22c connected to the display reference setting unit 22b is provided.

  The output unit 22a outputs the object model image D1 output from the image acquisition unit 20a of the image processing device 20 to the display 12, and displays the object model image D1 on the display 12.

  Further, the output unit 22a outputs the result of the examination of the tissue T2 by the artificial intelligence program 21 to the display 12 as result data, and causes the display 12 to display the result data.

  In the display reference setting unit 22b, a display reference for displaying the result of the examination of the tissue T2 by the artificial intelligence program 21 on the display 12 as result data is set in advance.

  This display standard is determined based on, for example, the ratio of abnormal cells C that match the plurality of abnormal cells C in the tissue T1 in the object model image D1 in the tissue T2 in the inspection object image D5.

  For example, if the proportion of abnormal cells C that match a plurality of abnormal cells C in the tissue T1 in the tissue T2 in the inspection object image D5 is determined to be 60%, the artificial intelligence that matches at a rate of 60% The result of the examination of the tissue T2 by the program 21 is displayed on the display 12 as result data.

  In this case, in the result data, the test object that has been determined to be a cancer cell in the tissue T1 in the object model image D1 and matched with the abnormal cell C to which the annotation D2 has been given, at a rate of 60%. A marker is added to the abnormal cell C present in the tissue T2 in the image D5.

  5 and 6 are diagrams for explaining the outline of the processing of the display reference changing unit 22c. The display reference changing unit 22c changes the display reference set in the display reference setting unit 22b.

  As shown in the figure, the display reference changing unit 22c is independent of the Min slider 22ca and the Min slider 22ca that set the lower limit of the matching ratio between the abnormal cells C of the tissue T1 and the abnormal cells C of the tissue T2. A Max slider 22cb is provided for setting an upper limit of the rate of coincidence between the plurality of abnormal cells C in T1 and the abnormal cells C in the tissue T2.

  This display reference changing unit 22c is displayed on the display 12 together with the result data D6. As shown in FIG. 5, when the Min slider 22ca is set to 80% and the Max slider 22cb is set to 100%, The result of the examination of the tissue T2 by the artificial intelligence program 21 that matches at a rate of 80% or more is displayed on the display 12 as result data D6.

  That is, an abnormal cell C that is determined to be a cancer cell in the tissue T1 in the object model image D1 and is annotated with the annotation D2 at a rate of 80% or more is a tissue in the inspection object image D5. It is determined that it exists in T2, and the marker M is given to the abnormal cell C existing in the tissue T2 in the inspection object image D5.

  On the other hand, as shown in FIG. 6, when the Min slider 22ca is set to 60% and the Max slider 22cb is set to 80%, the organization by the artificial intelligence program 21 that matches at a rate of 60% to 80%. The result of the inspection at T2 is displayed on the display 12 as result data D6.

  That is, the abnormal cell C that is determined to be a cancer cell in the tissue T1 in the target object model image D1 and is annotated with the annotation D2 at a rate of 60% to 80% is the inspection target image D5. The marker M is given to the abnormal cell C existing in the tissue T2 in the inspection object image D5.

  In this way, the display reference changing unit 22c changes the matching ratio between the plurality of abnormal cells C in the tissue T1 and the abnormal cells C in the tissue T2 to a desired ratio by changing a preset display reference. As a result, the result data D6 corresponding to the changed percentage of matching can be displayed on the display 12.

  In the present embodiment, the scanner 11 captures the object model image D1 and outputs it to the image processing apparatus 20, and also captures the inspection object image D5 and outputs it to the artificial intelligence program 21.

  In the present embodiment, the display 12 displays the object model image D1 output from the image acquisition unit 20a of the image processing device 20, and the result of the examination of the tissue T2 in the examination object image D5 by the artificial intelligence program 21. Is displayed as result data D6.

  Next, the operation of the inspection system 1 according to the present embodiment will be described.

  First, the user captures a plurality of object model images D1 with the scanner 11, and causes the image processing apparatus 20 to output the captured object model images D1.

  The plurality of object model images D1 output to the image processing apparatus 20 are output to the display function setting unit 22 and displayed on the display 12, and are also output to the artificial intelligence program 21.

  While observing a plurality of object model images D1 displayed on the display 12, the user operates an external input device such as a mouse or a pointing device to output a plurality of object model images D1 output to the artificial intelligence program 21. Is given an annotation D2.

  In the present embodiment, each of the plurality of abnormal cells C existing as a feature of the tissue T1 of the pathological specimen captured as an image in the object model image D1 is determined whether or not it is a cancer cell. Annotation D2 is given to the abnormal cell C determined to be a cell.

  As described above, the annotation D2 is given to the object model image D1 output to the artificial intelligence program 21, so that the artificial intelligence program 21 generates learning data D3 from the object model image D1.

  When the learning data D3 is generated, the artificial intelligence program 21 generates a learned model D4 by performing machine learning on the generated learning data D3. Thereby, the preparation for inspecting the tissue T2 in the inspection object image D5 by the inspection system 1 is completed.

  Subsequently, the user captures the inspection object image D5 with the scanner 11 and causes the artificial intelligence program 21 to output the captured inspection object image D5.

  Based on the learned model D4, the artificial intelligence program 21 detects abnormal cells C that match a plurality of abnormal cells C in the tissue T1 in the object model image D1 learned as the learned model D4 in the inspection object image D5. The organization T2 is examined by autonomously judging whether or not it exists in the organization T2.

  The result of the examination of the tissue T2 by the artificial intelligence program 21 is displayed on the display 12 as result data D6 based on a display standard preset in the display standard setting unit 22b.

  For example, if the display standard defines that the proportion of abnormal cells C that coincide with a plurality of abnormal cells C in the tissue T1 in the tissue T2 in the inspection object image D5 is 60%, the proportion is 60%. The result of the examination of the tissue T2 by the artificial intelligence program 21 that matches is displayed on the display 12 as result data.

  On the other hand, by changing the preset display standard by the display standard changing unit 22c, the ratio of the coincidence between the abnormal cells C in the tissue T1 and the abnormal cells C in the tissue T2 is changed to a desired ratio and changed. As a result, the result data D6 corresponding to the matching ratio can be displayed on the display 12.

  Therefore, for example, when the result of the examination of the tissue T2 by the artificial intelligence program 21 that matches at a rate of 80% or more is displayed on the display 12 as the result data D6, even if it is an abnormal cell C but not a cancer cell It is assumed that the marker M is attached and the result data D6 is displayed on the display 12.

  In such a case, the display standard is changed so that the result of the examination of the tissue T2 by the artificial intelligence program 21 that matches at a rate of 60% to 80% is displayed on the display 12 as the result data D6. What is necessary is just to make it display on the display 12 the result data D6 to which the marker M is not attached | subjected to the abnormal cell C which is not a cell.

  On the other hand, when the result data D6 displayed on the display 12 is viewed and the artificial intelligence program 21 determines that the percentage of coincidence that the abnormal cell C of the tissue T2 is a cancer cell is too high or too low. The result data D6 is regarded as the object model image D1, and the annotation D2 is given again to be used for the learning of the artificial intelligence program 21 again.

  As described above, the inspection system 1 according to the present embodiment uses the image processing apparatus 20 as an image on the object model image D1 that is input to the artificial intelligence program 21 by a simple operation of giving the annotation D2. The abnormal cells C of the tissue T2 that have been taken up in advance can be selected as the characteristics of the tissue T2.

  Moreover, since the object model image D1 can be used as the learning data D3 by adding the annotation D2 to the object model image D1, the learning data D3 can be easily generated by a simple operation of adding the annotation D2. can do.

  Since the inspection system 1 according to the present embodiment can easily generate the learning data D3 by a simple operation of assigning the annotation D2 to the object model image D1, the artificial intelligence program 21 efficiently performs machine learning. It can be performed.

  Therefore, the workability at the time of inspecting the tissue T2 in the inspection object image D5 can be improved.

  In addition, when the examination of the tissue T2 by the artificial intelligence program 21 is displayed on the display 12 as the result data D6, the preset display standard can be changed by the display standard changing unit 22c. Data D6 can be displayed on the display 12.

  The display reference changing unit 22c includes a Min slider 22ca that sets a lower limit of the display reference, and a Max slider 22cb that sets an upper limit of the display reference independently of the Min slider 22ca, and can be operated independently of each other. The operability when changing the display standard is improved.

  In addition, this invention is not limited to the said embodiment, A various change is possible in the range which does not deviate from the meaning of invention. In the above-described embodiment, the case where the object W1 and the inspection object W2 are the tissue T1 and the tissue T2 respectively collected from the human body has been described. For example, the surface of the resin product or the semiconductor substrate, contaminated impurities, Any object that can be inspected by the inspection system 1 such as a defective molding portion can be the object W1 and the inspection object W2.

  In the above-described embodiment, the case where the annotation D2 is given by operating an external input device such as a mouse or a pointing device has been described. For example, a touch panel display is used to touch the screen. In this case, the annotation D2 may be added.

1 Inspection System 10 Computer 12 Display (Display Unit)
20 Image processing apparatus 20b Annotation giving unit 21 Artificial intelligence program 22 Display function setting unit 22b Display reference setting unit 22c Display reference changing unit D1 Object model image D2 Annotation D3 Learning data D4 Trained model D5 Inspection object image D6 Result data T1 Organization T2 Organization W1 Object W2 Inspection object

Claims (4)

In an image processing device that generates learning data that causes an artificial intelligence program to autonomously determine whether or not a feature that matches the feature of the target exists in the test target, machine learning,
An annotation adding unit that selects an arbitrary portion of the object in the object model image in which the object is captured as an image and adds an annotation as a feature of the object;
An image processing apparatus, wherein the learning data is generated from the object model image by adding the annotation to the object model image by the annotation adding unit. In the inspection system for inspecting the inspection object by determining whether the inspection object has a feature that matches the characteristic of the object,
A display unit for displaying an object model image in which the object is captured as an image;
An annotation giving unit for selecting an arbitrary part of the target object captured as an image in the target object model image displayed on the display unit as a feature of the target object and giving an annotation;
The annotation giving unit learns the learned model based on the learned model that has been machine-learned by the learning data generated from the object model image by giving the annotation to the object model image. An artificial intelligence program that autonomously determines whether or not a feature that matches the feature of the object exists in the inspection object and inspects the inspection object;
An inspection system comprising: The result of the inspection of the inspection object by the artificial intelligence program based on a preset display standard is displayed on the display unit,
The inspection system according to claim 2, further comprising: a display reference changing unit that causes the display unit to display a result of the inspection of the inspection object corresponding to the display reference changed by changing the display reference. The display reference changing unit
The inspection system according to claim 3, wherein an upper limit and a lower limit of the display standard are configured to be set independently of each other.

Images