KR20230125999A - Immunofluorescence assay result reading apparatus using artificial intelligence and Reading and method thereof - Google Patents

Abstract

The present invention relates to an immunofluorescence test result reading device using artificial intelligence and a reading method using the same.
According to the present invention, a learning unit for learning a negatively read image and a positively read image by a pre-built machine learning algorithm, in an enlarged state of a reading slide with fluorescently treated bodily fluid or tissue collected from a patient An image acquisition unit that acquires a captured image, and a data extraction unit that classifies the acquired image in pixel units, obtains RGB values for each of the classified pixels, and extracts feature data according to the obtained RGB values. and a reading unit receiving a reading result for the antibody by inputting the obtained feature data to the machine learning algorithm.
As described above, according to the present invention, since a machine learning algorithm is used to automatically provide a reading result for an image taken of an IFA test slide, it is possible to objectify the subjective judgment of an observer or the reading of a reaction value that varies depending on the skill level, and the judgment result Acquisition time can be shortened.

Description

Immunofluorescence assay result reading apparatus using artificial intelligence and reading method thereof

The present invention relates to an immunofluorescence test result reading device using artificial intelligence and a reading method using the same, and more particularly, to provide a reading result for a reading slide attached with fluorescently treated bodily fluid or tissue using a machine learning algorithm It relates to an immunofluorescence test result reading device using artificial intelligence and a reading method using the same.

Laboratory methods such as culture, antibody assay, antigen test, and nucleic acid test are used to diagnose infectious diseases. Immunofluorescence is one of the most widely used assays for antigen or antibody assays.

There are two main types of immunofluorescence: direct immunofluorescence and indirect immunofluorescence. Direct immunofluorescence is a principle of immobilizing a specimen containing an antigen on a slide and measuring it using an immune response. An antibody that reacts to the fixed sample is added, and the antibody is labeled with a fluorescent dye. After washing to remove unbound antibody, the fluorescence of the antibody bound to the antigen is observed under a fluorescence microscope for reading. Showing fluorescence means that an antigen that specifically reacts with the antibody used for the test is attached to the slide. Indirect immunofluorescence is used to test for the presence of antibodies. A specific antigen of interest is immobilized on a slide and the serum to be tested is treated. If there is a specific antibody in the serum, the antibody reacts with the antigen immobilized on the slide to form a complex. After washing to remove unreacted antibody, a secondary antibody labeled with a fluorescent dye is added to measure the immobilized antibody reacting with the antigen. The slide subjected to the immunofluorescence test is observed and examined under a fluorescence microscope, and showing fluorescence means that antibodies specific to the antigen used for the test are present in the serum.

Currently, most laboratories adopt a method of visually reading slides inspected by immunofluorescence using a fluorescence microscope and semi-quantitatively reporting the results. The biggest limitation of this method is the fundamental limitation that there is room for subjective judgment by the examiner to intervene when visually read, and the result must be reported non-quantitatively. Also, reading takes a lot of time.

The background technology of the present invention is disclosed in Republic of Korea Patent Registration No. 10-1550225 (2015.09.07. Notice).

The technical problem to be achieved by the present invention is to provide an immunofluorescence test result reading device using artificial intelligence that provides reading results for a reading slide attached with fluorescently treated bodily fluid or tissue using a machine learning algorithm and a reading method using the same. has a purpose to

According to an embodiment of the present invention for achieving this technical problem, in an immunofluorescence test result reading device using artificial intelligence, learning to learn a negatively read image and a positively read image by a pre-built machine learning algorithm An image acquisition unit that acquires an image captured from a patient and photographed in an enlarged state of a reading slide to which fluorescently treated bodily fluid or tissue is attached, classifies the acquired image in pixel units, and classifies each of the classified pixels It includes a data extraction unit that obtains RGB values of and extracts feature data according to the acquired RGB values, and a reader that inputs the acquired feature data to the machine learning algorithm to receive a reading result for the antibody.

When the antibody is expressed in a first color when the antibody is positive, and when the antibody is expressed in a second color when the antibody is negative, the characteristic data is the ratio occupied by pixels having RGB values corresponding to the first color. , and a ratio occupied by pixels having RGB values corresponding to the second color.

The machine learning algorithm may be trained to read a positive or negative antibody reaction according to the ratio occupied by the first color and the ratio occupied by the second color of the fluorescently treated bodily fluid or tissue.

The feature data may include an average RGB value of the input image.

The machine learning algorithm may be trained to read a positive or negative antibody reaction according to the average RGB value of the fluorescently treated bodily fluid or tissue.

In addition, according to an embodiment of the present invention, in a reading method using an immunofluorescence test result reading device, learning a negatively read image and a positively read image by a pre-built machine learning algorithm, from a patient Acquiring an image photographed in an enlarged state of a reading slide having collected and fluorescently treated bodily fluid or tissue attached thereto, classifying the obtained image in pixel units, and obtaining respective RGB values for the classified pixels; , extracting feature data according to the acquired RGB values, and receiving a reading result for the antibody by inputting the obtained feature data to the machine learning algorithm.

As described above, according to the present invention, since a machine learning algorithm is used to automatically provide a reading result for an image taken of an IFA test slide, it is possible to objectify the subjective judgment of an observer or the reading of a response value that varies depending on the skill level, and the judgment result Acquisition time can be shortened.

1 is a block diagram of an immunofluorescence test result reading device according to an embodiment of the present invention.
2 is a flowchart illustrating a reading method using an immunofluorescence test result reading device according to an embodiment of the present invention.
3 (a) is a diagram showing an image read as a negative reaction to the antibody, and (b) is a diagram showing an image read as a positive reaction to the antibody.
FIG. 4A is a flowchart illustrating a method of extracting feature data using the first method in step S230 shown in FIG. 2 .
FIG. 4B is a flowchart illustrating a method of extracting feature data using the second method in step S230 shown in FIG. 2 .

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of explanation.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

Hereinafter, an immunofluorescence test result reading apparatus according to an embodiment of the present invention will be described with reference to FIG. 1 .

1 is a block diagram of an immunofluorescence test result reading device according to an embodiment of the present invention.

As shown in FIG. 1, the immunofluorescence test result reading apparatus 100 according to an embodiment of the present invention includes a learning unit 110, an image acquisition unit 120, a data extraction unit 130, and a reading unit 140. includes

First, the learning unit 110 inputs a pre-read image to a pre-built machine learning algorithm to learn it. Here, the pre-read image represents an image captured by enlarging the read slide to which the fluorescently treated bodily fluid or tissue is attached, and is an image that has been read negatively or positively for the antibody by the examiner.

That is, the learning unit 110 acquires images read as negative and images read as positive, and inputs the acquired images to a pre-built machine learning algorithm to conduct supervised learning.

Here, the machine learning algorithm learns a reading result of the received image using a ratio of colors that are differently represented according to a positive reaction or a negative reaction to the antibody. To elaborate, antibodies produced in cells in an immune response cause chemical interactions when combined with antigens.

Therefore, when the antibody is present in the patient's body fluid or tissue stained by immunofluorescence, green color is developed by chemical interaction, and when the antibody is not present, red color is developed. Therefore, the machine learning algorithm learns to read the antibody as a positive or negative reaction according to the ratio of green and red to the total area of the input image. And, the machine learning algorithm uses the ratio occupied by the green series or the ratio occupied by the red series in the image by repetitive learning to determine whether it is positive or negative.

In addition, the machine learning algorithm acquires the RGB value of each pixel included in the input image, and learns to read the positive or negative reaction of the antibody using the obtained average value of RGB. In addition, the machine learning algorithm obtains determination ranges for RGB average values of images determined as positive and RGB average values of images determined as negative due to repetitive learning, respectively.

The image acquisition unit 120 acquires an image taken in an enlarged state of the reading slide on which the immunofluorescence test was performed. In other words, the examiner collects bodily fluid or tissue from the patient. Then, the examiner performs the immunofluorescence method directly or indirectly on the collected body fluid or tissue. In the direct method, a fluorescent antibody and an antigen or a fluorescent antigen and an antibody are combined, and in an indirect method, an antigen and an antibody are mutually coupled, and then a fluorescent antibody or a fluorescent antigen is coupled.

Next, the examiner fixes the immunofluorescence-treated body fluid or tissue on a reading slide and places it on a microscope. Then, the photographing device obtains an image obtained by photographing an enlarged image of the specimen using a microscope. Then, the photographing device transfers the obtained image to the image acquisition unit 120 .

The data extractor 130 extracts feature data using the acquired image. In other words, the data extraction unit 130 classifies the image received from the image acquisition unit 120 in units of pixels, and acquires RGB values of each classified pixel. Then, the data extractor 130 extracts feature data using the acquired RGB values.

Here, the feature data includes a ratio of pixels having RGB values corresponding to green and red colors and an average RGB value of all pixels constituting the image.

To explain this again, the data extractor 130 extracts feature data by two methods.

Describing the first feature data extraction method, the data extraction unit 130 extracts pixels having RGB values corresponding to a green color series or pixels having RGB values corresponding to a red color series. Next, the data extraction unit 130 calculates the ratio of pixels having RGB values corresponding to the extracted green color series and the ratio of pixels having RGB values corresponding to the extracted red color series relative to the entire area of the image. to obtain feature data.

And, in the second method of extracting feature data, the data extractor 130 acquires the total sum of the obtained RGB values of each pixel, and then calculates an average value of the obtained total sum to obtain feature data.

Finally, the reading unit 140 receives a reading result from the machine learning algorithm on which learning has been completed. That is, the reader 140 inputs the feature data extracted from the acquired image to the machine learning algorithm. Then, the machine learning algorithm outputs a positive or negative reading result for the antibody using the feature data.

Hereinafter, a reading method using the immunofluorescence test result reading device according to an embodiment of the present invention will be described in more detail with reference to FIGS. 2 to 4B.

Figure 2 is a flow chart for explaining a reading method using an immunofluorescence test result reading device according to an embodiment of the present invention, Figure 3 (a) is a diagram showing an image read as a negative reaction to the antibody, (b ) is a diagram showing an image read as a positive reaction for the antibody.

As shown in FIG. 2, the immunofluorescence test result reading apparatus according to an embodiment of the present invention uses the learning unit 110 to input a read image to a machine learning algorithm to read positive and negative reactions. Machine learning is performed (S210). Here, the read image indicates an image read as a negative reaction and an image read as a positive reaction.

As shown in (a) of FIG. 3 , only a red-stained background color is output when an antibody reaction does not occur in a patient's body fluid or tissue. On the other hand, as shown in (b) of FIG. 3 , when an antibody is present in a patient's body fluid or tissue and chemically reacts with the stained antigen, the antibody is output in green.

Therefore, the learning unit 110 generates a negative training dataset by randomly extracting from a plurality of images read as negative, and generates a positive training dataset by randomly extracting from a plurality of images read as positive. Then, the learning unit 110 inputs the generated voice training dataset and positive training dataset to a pre-built machine learning algorithm. Then, the machine learning algorithm learns to read the input image as negative or positive using the input negative training dataset and the positive training dataset.

When learning for reading is completed as in S210, the image acquisition unit 120 acquires a microscope image to be read from the photographing device (S220).

To paraphrase, the examiner places a reading slide with fluorescently treated bodily fluid or tissue attached to it under the microscope. Then, the photographing device photographs the enlarged reading slide with a microscope, and transmits the photographed image to the image acquisition unit 120 .

Next, the data extraction unit 130 classifies the image to be read received through the image acquisition unit 120 in units of pixels, and obtains RGB values of the classified pixels (S230).

In other words, a photographed image includes a plurality of pixels according to resolution. And each pixel represents a color by a combination of R (red), G (green), and B (blue). Therefore, the data extractor 130 acquires RGB values for each pixel and extracts feature data according to the obtained RGB values (S240).

Hereinafter, step S240 will be described in more detail using FIGS. 4A and 4B.

Figure 4a is a flow chart for explaining a method for extracting feature data using the first method in step S240 shown in Figure 2, Figure 4b is a feature using the second method in step S240 shown in Figure 2 It is a flowchart to explain how to extract data.

According to an embodiment of the present invention, the data extraction unit 120 extracts feature data by two methods.

First, in the first extraction method, as shown in FIG. 4A, the data extraction unit 130 acquires RGB values of pixels included in the image to be read (S241).

Then, the data extraction unit 130 calculates a ratio occupied by pixels having red-based RGB values among pixels included in the image. Also, the data extraction unit 130 calculates a ratio occupied by pixels having green RGB values among pixels included in the image (S242).

Here, the RGB value of the red series can be determined through the RGB values of the learned image when the antibody reacts negatively, and similarly, the RGB value of the green series is the RGB value of the learned image when the antibody reacts positively. can be determined through

That is, the data extractor 130 calculates a ratio occupied by pixels having RGB values corresponding to a green series and a ratio occupied by pixels having RGB values corresponding to a red series. Here, the total number of pixels constituting the image to be read varies depending on the resolution (dpi) of the image. In this way, among the pixels included in the image, the ratio occupied by pixels having red or green RGB values is used as specific data and transmitted to the reader 140 to be described later.

Meanwhile, looking at the second method of extracting feature data according to an embodiment of the present invention, as shown in FIG. 4B, the data extractor 130 obtains RGB values of pixels included in the image to be read (S241). .

Next, the data extraction unit 120 calculates an average value using the RGB values of each pixel constituting the entire image (S243).

In other words, the data extraction unit 130 obtains the total sum of RGB values of each pixel constituting the entire image. Then, the data extraction unit 130 calculates an average value by dividing the sum of the obtained RGB values by the number of pixels constituting the entire image.

As in the first method, the average RGB values of the pixels included in the image are used as specific data and transmitted to the reader 140.

When feature data extraction is completed by two methods in step S240, the reading unit 140 inputs the obtained feature data to a machine learning algorithm and receives a reading result (S250).

That is, the reader 140 inputs the obtained image and feature data extracted from the image to the machine learning algorithm. Then, the machine learning algorithm outputs a reading result for the acquired image using the input feature data. Here, the feature data may be a ratio of pixels having RGB values corresponding to green and red colors, or may be an average value of RGB values of the entire image.

Accordingly, the machine learning algorithm reads the antibody reading result as positive or negative according to the input feature data, and transfers the read result to the reading unit 140 . Then, the reading unit 140 outputs the received reading result.

As described above, the immunofluorescence test result reading device according to the present invention automatically provides reading results for images taken of IFA test slides using a machine learning algorithm, thereby objectifying the reading of response values that vary depending on the observer's subjective judgment or skill level. and shorten the time to obtain the judgment result.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. will be. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the claims below.

100: immunofluorescence test result reading device
110: learning unit
120: image acquisition unit
130: data extraction unit
140: reading unit

Claims (10)

In the immunofluorescence test result reading device using artificial intelligence,
A learning unit for learning a negatively read image and a positively read image by a pre-built machine learning algorithm;
An image acquisition unit for obtaining an image taken in an enlarged state of a reading slide collected from a patient and attached with fluorescently treated bodily fluid or tissue;
A data extractor configured to classify the acquired image in pixel units, acquire RGB values for each of the classified pixels, and extract feature data according to the acquired RGB values; and
and an immunofluorescence test result reading device comprising a reading unit receiving a reading result for an antibody by inputting the obtained feature data to the machine learning algorithm. According to claim 1,
When the antibody is expressed in a first color when the antibody is positive and the second color is expressed when the antibody is negative,
The feature data,
A ratio occupied by pixels having RGB values corresponding to the first color, and
Immunofluorescence test result reading device comprising a ratio occupied by pixels having RGB values corresponding to the second color. According to claim 2,
The machine learning algorithm is
An immunofluorescence test result reading device that is trained to read a positive or negative antibody reaction according to a ratio occupied by a first color and a ratio occupied by a second color of a fluorescently treated bodily fluid or tissue. According to claim 1,
The feature data,
Immunofluorescence test result reading device including the average RGB value of the input image. According to claim 4,
The machine learning algorithm is
An immunofluorescence test result reading device that is trained to read positive or negative antibody reactions according to the average RGB values of fluorescently treated bodily fluids or tissues. In the immunofluorescence test result reading method using an immunofluorescence test result reading device,
Learning the machine learning algorithm by inputting the negatively read image and the positively read image to a pre-built machine learning algorithm;
An image acquisition step of acquiring an image taken from a photographing device in an enlarged state of a reading slide collected from a patient and attached with fluorescently treated bodily fluid or tissue;
Classifying the obtained image in units of pixels, acquiring RGB values for each of the classified pixels, and extracting feature data according to the obtained RGB values; and
An immunofluorescence test result reading method comprising the step of inputting the obtained feature data to the machine learning algorithm and receiving a reading result for the antibody from the machine learning algorithm. According to claim 1,
When the antibody is expressed in a first color when the antibody is positive and the second color is expressed when the antibody is negative,
The feature data,
A ratio occupied by pixels having RGB values corresponding to the first color, and
Immunofluorescence test result reading method comprising a ratio occupied by pixels having RGB values corresponding to the second color. According to claim 7,
The machine learning algorithm is
An immunofluorescence test result reading method in which a positive or negative antibody reaction is read according to a ratio occupied by a first color and a ratio occupied by a second color of a fluorescently treated bodily fluid or tissue. According to claim 6,
The feature data,
Immunofluorescence test result reading method including the average RGB value of the input image. According to claim 9,
The machine learning algorithm,
An immunofluorescence test result reading method in which a positive or negative antibody reaction is learned to be read according to the average RGB values of fluorescently treated bodily fluids or tissues.

Images