WO2021025458A1

WO2021025458A1 - Device for analyzing mobile in-vitro diagnostic kit by using multimedia information

Info

Publication number: WO2021025458A1
Application number: PCT/KR2020/010329
Authority: WO
Inventors: 장현재; 김석호
Original assignee: 주식회사 에프앤디파트너스
Priority date: 2019-08-05
Filing date: 2020-08-05
Publication date: 2021-02-11
Also published as: KR102091832B1

Abstract

The present invention relates to a device for analyzing a mobile in-vitro diagnostic kit by using multimedia information, and more specifically, to a device for analyzing a mobile in-vitro diagnostic kit by using multimedia information, wherein the device: uses a deep learning model to evaluate the multimedia information, captured through a camera, about the in-vitro diagnostic kit to determine at least one in-vitro diagnostic method among a shape change reading method or a color change reading method; uses the deep learning model to calculate a label and an output value for each position of a sample in which changes did not occur; reads a label and an output value for each position of a sample in which changes did occur; and displays the read result value or transmits the same to a user terminal.

Description

Portable in vitro diagnostic kit analysis device using multimedia information

The present invention relates to a portable in vitro diagnostic kit analysis apparatus using multimedia information, and more particularly, a change in shape or color change using a deep learning model for multimedia information of an in vitro diagnostic kit photographed through a camera. Determining any one or more in vitro diagnostic methods among the reading methods, calculating labels and output values for each sample location where no change has occurred using a deep learning model, and reading the labels and output values for each sample location where changes have occurred It relates to a portable in vitro diagnostic kit analysis device using multimedia information for displaying or transmitting to a user terminal.

Recently, a technology called machine learning or machine learning has been applied to various fields ranging from software technology to finance and economy, and is positioned as a key technology leading the rapid development of computer vision and image processing in particular.

In addition, in recent years, machine learning technology is widely used in the medical diagnosis field including medical image analysis and the overall medical image analysis field such as extraction and segmentation of organs or cancer parts from medical images, image matching, and image search.

This machine learning technology is a field of artificial intelligence (AI), which refers to algorithms and related fields that enable new data to be analyzed by learning patterns or characteristics from given data.

And, as a machine learning technique called deep learning has recently emerged as a core technology, interest in related technologies and applications is increasing.

The deep learning technique is a model of an artificial neural network that mimics the nervous system of an organism. If the existing artificial neural network model consists of a connection of thin layers of neuron models, the deep learning technique stacks the layers of the neuron model deeply. It is a technology that applies a model that increases the learning ability of neural networks by raising.

The concept of deep learning as a multi-layered artificial neural network was proposed in the 1970s, but it has been stagnant due to the complexity of learning calculations, and the performance has been improved through recent studies, and related studies have been conducted such as voice recognition and image recognition. The demand is growing rapidly with outstanding results in the field.

For example, in order to increase the efficiency of image reading and improve the productivity of the diagnosis process in analyzing dozens of medical image slices per patient during MRI examination, there is a need for a medical image diagnosis assistance system that can be used by machine learning based on actual data. .

In addition, the data-based artificial intelligence system generated by applying all data used by doctors for diagnosis in the medical field, that is, various clinical information other than medical images, has improved diagnostic performance compared to medical machine learning algorithms learned only with medical images. Can be expected.

On the other hand, the in vitro diagnosis related to the present invention is a medical technology capable of confirming the infection or treatment effect of a disease using substances derived from the human body such as blood, manure, body fluids, saliva, such as a urine test or a blood test.

Representative in vitro diagnostic kits include blood glucose meters, pregnancy diagnostics, and urine test kits.

Recently, various in vitro diagnostic kits have been developed for diseases that have been difficult to diagnose such as cancer and dementia, and research is being conducted for commercialization.

Accordingly, even diseases that are difficult to diagnose now are expected to be diagnosed more easily and without burden in the future.

The above in vitro diagnosis will be described in detail.

The most representative diagnosis of the in vitro diagnosis is an immunochemical diagnosis, and the immunochemical diagnosis is a method based on an antigen-antibody reaction.

Harmful substances that enter the body, such as bacteria or viruses, are called antigens, and the body produces antibodies to remove the antigen. When the antigen and the antibody meet, an immune reaction occurs.

As shown in FIG. 1, there are various types of immunochemical diagnosis, and the most commonly used methods are enzyme immunoassay and aggregation.

The enzyme immunoassay method is widely used in pregnancy diagnosis and urine test sticks, and changes color in response to body fluids.

The agglutination method is a method used in a blood type test sheet, and the blood aggregates and changes its shape.

As such, there is an urgent need to develop a new concept diagnosis and read reader that can inspect and monitor the health status of the public anytime, anywhere.In analyzing and determining signals detected by light-emitting elements, optical elements, and microprocessors, near the judgment measured value There are problems in that it cannot be activated due to poor accuracy and very expensive for individuals to purchase, and difficulties in use as non-professionals.

Accordingly, for an efficient personalized health care service, the development of a highly sensitive and highly selective sensor that can effectively diagnose and manage symptoms of health abnormalities, and is easy to collect, such as urine or sweat, has no pain, and is easy for personal use. It can be said that the development of a convenient system is urgent.

Accordingly, in the present invention, it is intended to provide a user with portability and to provide a diagnosis result with high accuracy.

(Patent Document 0001) Korean Laid-Open Patent Publication No. 10-2015-0026166 (2015.03.11)

(Patent Document 0002) Republic of Korea Patent Publication No. 10-2018-0057220 (2018.05.30)

Accordingly, the present invention has been proposed in consideration of the problems of the prior art as described above, and a first object of the present invention is a method of reading changes in shape using a deep learning model for multimedia information of an in vitro diagnostic kit photographed through a camera. Or, determine any one or more in vitro diagnostic methods among the color change reading methods, calculate the label and output value for each sample location where no change has occurred using a deep learning model, and read the label and output value for each sample location where the change has occurred. The result is to be displayed or transmitted to the user terminal.

In order to achieve the problem to be solved by the present invention, a portable in vitro diagnostic kit analysis apparatus using multimedia information according to an embodiment of the present invention,

When acquiring multimedia information of an in vitro diagnostic kit photographed through a camera, an in vitro diagnostic method to determine at least one of the in vitro diagnostic methods of the shape change reading method or the color change reading method using a deep learning model With the determination unit 100,

A sample location recognition unit 200 for recognizing the location of one or more samples in the multimedia information,

A labeling unit 300 for assigning a label for each recognized sample location,

When the in vitro diagnosis method is a method of reading changes in shape, a deep learning model is used to calculate the label and output value for each sample location where no change has occurred, and the shape change to read the label and output value for each sample location where the change has occurred. A reading unit 400,

When the in vitro diagnostic method is a color change reading method, a deep learning model is used to calculate the label and output value for each sample location where no change has occurred, and the color change to read the label and output value for each sample location where the change has occurred. It includes a reading unit 500.

Portable in vitro diagnostic kit analysis apparatus using multimedia information according to the present invention,

Using a deep learning model, the multimedia information of the in vitro diagnostic kit captured by the camera is used to determine any one or more of the in vitro diagnosis method, either the shape change reading method or the color change reading method, and the change is changed using the deep learning model. Calculate the label and output value for each sample location that has not occurred, and display the read result value by reading the label and output value for each sample location where the change has occurred, or send it to the user terminal so that anyone can easily be outside the body regardless of time. It provides convenience to check the results of the diagnostic kit.

In addition, by using a deep learning model to determine the diagnosis result of the in vitro diagnostic kit, new images can be relearned, thereby further improving the discrimination performance of the in vitro diagnostic kit.

In other words, after learning through artificial intelligence learning in advance, it has the effect of continuously improving diagnosis accuracy by continuously learning new images.

1 is a diagram showing the types of general immunochemical diagnosis.

Figure 2 is a configuration diagram of a portable in vitro diagnostic kit analysis apparatus using multimedia information according to an embodiment of the present invention.

3 is a diagram illustrating an example of an image in which position recognition and labeling are completed when a blood test is performed by a portable in vitro diagnostic kit analyzing apparatus using multimedia information according to an embodiment of the present invention.

4 is a diagram illustrating an example of an image in which location recognition and labeling are completed when a urine test of a portable in vitro diagnostic kit analysis device using multimedia information according to an embodiment of the present invention is performed.

5 is a result table showing output values and label values for each sample location during a blood test of a portable in vitro diagnostic kit analyzing apparatus using multimedia information according to an embodiment of the present invention.

6 is an exemplary view of a result table showing output values and label values for each sample location during a urine test of a portable in vitro diagnostic kit analyzing apparatus using multimedia information according to an embodiment of the present invention.

7 is a diagram showing change result table information for a shape referenced when a deep learning model of a shape change reading unit 400 of a portable in vitro diagnostic kit analysis apparatus using multimedia information according to an embodiment of the present invention performs basic learning. Example diagram.

8 is an exemplary view showing metadata added when a deep learning model of a color change reading unit 500 of a portable in vitro diagnostic kit analyzing apparatus using multimedia information according to an embodiment of the present invention performs basic learning.

9 is an exemplary diagram of a result color table of 10 kinds of kits used for a urine test.

FIG. 10 is a diagram showing a color containing a urine test result and a specified value for basic learning by a deep learning model of a color change reading unit 500 of a portable in vitro diagnostic kit analysis apparatus using multimedia information according to an embodiment of the present invention. An example diagram showing the information on the change result table.

Figure 11 is the effectiveness of the basic learning of the deep learning model used in the shape change reading unit 400 and the color change reading unit 500 of the portable in vitro diagnostic kit analysis device using multimedia information according to an embodiment of the present invention. An exemplary diagram showing the Confusion Matrix used to determine

100: in vitro diagnosis method judgment unit

200: sample location recognition unit

300: Whether to label

400: shape change reading unit

500: color change reading unit

600: read result output processing unit

The following content merely illustrates the principles of the present invention. Therefore, a person skilled in the art can implement the principles of the present invention and invent various devices included in the concept and scope of the present invention, although not clearly described or illustrated herein.

In addition, all conditional terms and examples listed in this specification are, in principle, intended to be clearly intended only for the purpose of making the concept of the present invention understood, and should be understood as not limiting to the embodiments and states specifically listed as described above. do.

Portable in vitro diagnostic kit analysis apparatus using multimedia information according to an embodiment of the present invention,

A labeling unit 300 for assigning a label for each recognized sample location,

When the in vitro diagnostic method is a color change reading method, a deep learning model is used to calculate the label and output value for each sample location where no change has occurred, and the color change to read the label and output value for each sample location where the change has occurred. Characterized in that it is configured to include a reading unit (500).

In addition, the portable in vitro diagnostic kit analysis device using the multimedia information,

It characterized in that the configuration further comprises a read result output processing unit 600 for displaying the result value read through the shape change reading unit 400 and the color change reading unit 500 or transmitting it to the user terminal. .

In addition, the deep learning model used in the shape change reading unit 400 and the color change reading unit 500,

Using a pre-learned Convolutional Neural Network (CNN) algorithm, the CNN (Convolutional Neural Network) algorithm uses the change result table information for shape and the change result table information for color to perform basic learning. , In order to determine the effectiveness at which basic learning is completed, sensitivity and specificity are measured through the Confusion Matrix to determine the effectiveness.

In addition, the read result output processing unit 600,

The multimedia information of the in vitro diagnostic kit is labeled and stored in the user labeling information storage module, and the result values read through the shape change reading unit 400 and the color change reading unit 500 are stored in the corresponding user labeling information storage module. It is characterized in that the stored labeling information is referred to and transmitted to the corresponding user terminal 2000.

In addition, the in vitro diagnostic method determination unit 100,

In order to obtain multimedia information of the in vitro diagnostic kit, it is a device that is connected to a camera to receive a direct input image, or to receive and input from a wireless network or an Internet network.

In addition, the shape change reading unit 400,

The output value of the sample with the shape change is characterized in that it is a learning result value learned in advance using the deep learning model.

In addition, the color change reading unit 500,

The output value of the sample with color change is characterized in that it is a learning result value learned in advance using a deep learning model.

Hereinafter, it will be described in detail through an embodiment of a portable in vitro diagnostic kit analysis apparatus using multimedia information according to the present invention.

2 is a block diagram of a portable in vitro diagnostic kit analyzing apparatus using multimedia information according to an embodiment of the present invention.

As shown in Figure 2, the present inventors portable in vitro diagnostic kit analysis device using multimedia information is in vitro diagnostic method determination unit 100, sample location recognition unit 200, labeling unit 300, shape change reading unit ( 400), it is configured to include a color change reading unit 500.

When configured as described above, the analysis apparatus of the present invention provides the advantage of being able to analyze with only an in vitro diagnostic kit without a colorimetric table without the need to visually check through a colorimetric table, which is a conventional general method, using a deep learning model.

For example, with only an in vitro diagnostic kit called a urine test strip, you can easily check your current health status.

Specifically, the in vitro diagnosis method determination unit 100 reads a change in shape or color change by using a deep learning model when acquiring multimedia information of an in vitro diagnosis kit photographed through a camera. It performs a function of determining any one or more in vitro diagnostic methods among the methods.

The deep learning model described above uses a pre-trained convolutional neural network (CNN) algorithm, and the convolutional neural network (CNN) algorithm creates a feature map by extracting main features from a multimedia image. It is a neural network algorithm that determines an image by giving a weight to a map.

Therefore, a deep learning model that determines at least one in vitro diagnosis method among a shape change reading method or a color change reading method is to learn from images of in vitro diagnosis kits.

For example, by learning whether it is an in vitro diagnostic kit that changes shape according to the appearance of the in vitro diagnostic kit, or an in vitro diagnostic kit that changes color, the deep learning model can change the shape change reading method or color according to the input multimedia image. Any one or more of the in vitro diagnostic methods of the change reading method of the patient are determined.

The shape change in vitro diagnostic kit means, for example, a pregnancy test kit, a blood test kit, and the like, and the color change in vitro diagnostic kit means, for example, a urine test kit.

In addition, the specimen location recognition unit 200 performs a function of recognizing the location of one or more specimens in the multimedia information.

For example, in the case of a urine test kit, about 10 samples are formed, and the process of recognizing the location of each sample is required, and except for the location of the sample, it is excluded because it is unnecessary for comparing shapes or colors. It is to let.

In addition, the labeling unit 300 performs a function of assigning a label for each recognized sample location.

For example, as shown in FIG. 3, when an actual blood type test image is captured, a location is recognized for an image part required for the image, and a sample-specific number (labeling) is assigned to the recognized location.

In the example, the location of the Anti-A sample is given as ① number, the location of the Anti-B sample is given as ② number, the location of the Anti-D sample is given as ③ number, and the location of the control sample is assigned ④ number.

As another example, as shown in FIG. 4, when an actual urine test image is taken, the position of the image part required for the image is recognized, and the number of each sample is assigned to the recognized position. The location is given as ① number, the location of sample 2 is assigned the number ②,..., and the location of sample 10 is assigned the number ⑩.

In addition, the shape change reading unit 400 calculates a label and output value for each sample location where no change has occurred using a deep learning model when the in vitro diagnosis method is a shape change reading method, and the sample location where the change occurs. It performs a function to read the star label and output value.

For example, the shape change reading method is a detection that checks when agglutination occurs when blood, body fluid, etc. meets a sample, and outputs the result by determining when aggregation has occurred or does not occur through a deep learning model. .

In this case, the deep learning model sets an image in which coagulation occurs and an image that does not occur as a class to perform learning, and compares the image with the currently input image and outputs the result.

In other words, when the position image of each sample is input into the deep learning model, '1' is output when the shape changes for each position, and '0' is otherwise output.

The output value and the label value for each sample location are as shown in FIG. 5, and the final result is output by comparing it with the blood type result table information. In the case of FIG. 5, it can be seen that it is'RH+O'.

In addition, the color change reading unit 500 calculates the label and output value for each sample location where no change has occurred by using a deep learning model when the in vitro diagnosis method is a color change reading method, and the sample location where the change occurs. It performs a function to read the star label and output value.

In this case, the deep learning model sets an image in which color change occurs and an image that does not occur as a class to perform learning, and compares the image with the currently input image to output a result.

The output value and the label value for each sample location are as shown in FIG. 6, and the final result is output by comparing it with the urine test result table information.

Using a pre-learned Convolutional Neural Network (CNN) algorithm, the CNN (Convolutional Neural Network) algorithm allows basic learning to proceed by using the change result table information for shape and color change result table information. do.

The deep learning model of the shape change reading unit 400 is initially trained with images in which the shape of the in vitro diagnostic kit changes.

If the input image is different from the original in vitro diagnostic kit image, the correct result is output.

For example, in the case of blood type, when blood is put in an in vitro diagnostic kit, it reacts with the kit and agglutination occurs.

According to each sample, it is possible to confirm the occurrence of aggregation and to confirm the blood.

ANTI-A and ANTI-B react to types A and B, and no reaction occurs to type O.

ANTI-D is a sample related to RH blood type. If an agglutination reaction occurs, it is RH+, and if it does not, it is RH-.

The CONTROL sample is a sample that checks whether there are any other abnormalities in the blood. If this sample aggregates, the blood type cannot be confirmed.

At this time, the deep learning model of the shape change reading unit 400 performs basic learning by using the change result table information for the shape as shown in FIG. 7.

The deep learning model of the color change reading unit 500 performs learning with colors of color tables as a result of the in vitro diagnosis kit.

Check the color of the input image, compare it with the learned color, and output the result.

For example, the deep learning model of the color change reading unit 500 confirms that the color changes due to an enzyme immune reaction when body fluids, urine, etc. meet the sample, and compares the color with the color of the change result table information to the color. Determine the value and print the result.

In this case, according to an additional aspect, the deep learning model of the color change reading unit 500 performs learning by adding metadata as shown in FIG. 8.

As shown in FIG. 8, metadata is stored in the form of a label value, a location, and a value (RGB), and is color and location data for an image.

It detects the color value of the input image through learning and outputs the top 5 primary result data, selects the data matching the position among the output values as the secondary result data, and the final result is the value of the color table and the second The result data is compared and the grade suitable for the disease name is output as the final result.

As described above, basic learning is conducted by using the change result table information for the color.

The in vitro diagnosis result of the urine test has a color change, and the result color table of the 10 kinds of kits is shown in FIG. 9.

In general, by comparing the color that appears after the urine is applied to the urine test stick with the color table of the result of FIG. 9, the state of itself can be visually confirmed.

There are 10 types that can be checked by a urine test: white blood cells, occult blood, nitrite, protein, acidity, specific gravity, ketone bodies, bilirubin, glucose, and urbilinogen.

The color that changes according to each sample is different, and you can check your body condition by checking this.

However, in the present invention, the deep learning model of the color change reading unit 500 performs basic learning using the change result table information for the color, and then determines the color most similar to the currently input image and outputs a result value. .

In particular, the deep learning model of the color change reading unit 500 of the present invention, as shown in FIG. 10, refers to the urine test result and the color change result table information including the specified value, and then basic learning, The color most similar to the input video image is determined, and the result value is outputted as a label for each sample location and a designated value for each sample location as shown in FIG.

As shown in the example of FIG. 10, ① label is'LN',..., ⑩ label outputs information such as'G100'.

Also, if necessary, it checks the output value and outputs it to the user to see which part is abnormal.

On the other hand, the deep learning model used in the shape change reading unit 400 and the color change reading unit 500,

In order to determine the effectiveness at which basic learning is completed, sensitivity and specificity are measured through a confusion matrix to determine the effectiveness.

Specifically, as shown in Fig. 11, in order to determine the effectiveness of completing basic learning, the sensitivity and specificity are measured through the Confusion Matrix to determine the effectiveness, but continuous images are required to reduce the error. Re-learning is required.

In order to relearn the values and colors of the images output in the final result table, change them into metadata and relearn.

Sensitivity refers to the rate at which a disease is actually present and the test result determines that there is a disease, and the specificity refers to the rate at which it is determined that the test result does not contain a disease.

Sensitivity and specificity are essential criteria when developing an in vitro diagnostic kit, and the higher the value, the higher the effectiveness.

The sensitivity is measured with reference to Equation 1 below.

(Equation 1)

The specificity is measured with reference to Equation 2 below.

(Equation 2)

The A, B, C, D refers to the alphabet shown in Fig. 11, for example, in the case of A, it means test -positive, confirming -positive, and in the case of B, the test -positive, confirming -negative it means.

On the other hand, according to an additional aspect, the present inventors portable in vitro diagnostic kit analysis apparatus using multimedia information,

That is, the read result output processing unit 600 displays the result value read through the shape change reading unit 400 and the color change reading unit 500, or transmits it to the user terminal, for example, multimedia information. If the portable in vitro diagnostic kit analysis device used constitutes a display panel, the read result value is displayed, and if it does not exist, it means that it is transmitted to the user terminal using a wired or wireless network.

On the other hand, the read result output processing unit 600,

When multimedia information is captured through a camera, the multimedia information of the obtained in vitro diagnostic kit is labeled and stored in the user labeling information storage module.

Thereafter, the result values read through the shape change reading unit 400 and the color change reading unit 500 are transmitted to a corresponding user terminal with reference to the labeling information stored in the user labeling information storage module.

That is, in addition to the device of the present invention, an external terminal assigns labeling in advance to transmit the user's diagnosis result value to the user terminal, and transmits and processes the read result value to the corresponding user terminal based on the assigned labeling information.

On the other hand, according to an additional aspect, the in vitro diagnostic method determination unit 100,

That is, direct input images may be received through the camera by interlocking with the camera, and various image images may be acquired using a wireless network or an Internet network.

According to the present invention, the multimedia information of the in vitro diagnostic kit photographed through the camera is determined by using a deep learning model to determine at least one in vitro diagnosis method among a change reading method for shape or a change reading method for color, and a deep learning model Calculate the label and output value for each sample location where no change has occurred, and display the read result value by reading the label and output value for each sample location where change has occurred, or send it to the user terminal so that you can stick with it anytime, anywhere. It provides convenience that anyone can easily check the results of the in vitro diagnostic kit without receiving it.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be implemented by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

The portable in vitro diagnostic kit analysis apparatus using multimedia information according to the present invention is one of a method of reading changes in shape or a change in color by using a deep learning model for multimedia information of the in vitro diagnostic kit photographed through a camera. Determine the above in vitro diagnosis method, calculate the label and output value for each sample location where no change has occurred using a deep learning model, read the label and output value for each sample location where the change has occurred, and display the read result, or Since it provides the convenience that anyone can easily check the results of the in vitro diagnostic kit anytime, anywhere by sending it to a user terminal, regardless of time, it has high industrial applicability.

Claims

In the portable in vitro diagnostic kit analysis device using multimedia information,

When acquiring multimedia information of an in vitro diagnostic kit photographed through a camera, an in vitro diagnostic method to determine at least one of the in vitro diagnostic methods of the shape change reading method or the color change reading method using a deep learning model With the determination unit 100,

A sample location recognition unit 200 for recognizing the location of one or more samples in the multimedia information,

A labeling unit 300 for assigning a label for each recognized sample location,

When the in vitro diagnosis method is a method of reading changes in shape, a deep learning model is used to calculate the label and output value for each sample location where no change has occurred, and the shape change to read the label and output value for each sample location where the change has occurred. A reading unit 400,

When the in vitro diagnostic method is a color change reading method, a deep learning model is used to calculate the label and output value for each sample location where no change has occurred, and the color change to read the label and output value for each sample location where the change has occurred. Portable in vitro diagnostic kit analysis device using multimedia information, characterized in that configured to include a reading unit (500).
The method of claim 1,

Portable in vitro diagnostic kit analysis device using the multimedia information,

And a read result output processing unit 600 for displaying the result value read through the shape change reading unit 400 and the color change reading unit 500 or transmitting it to the user terminal. Portable in vitro diagnostic kit analysis device using multimedia information.
The method of claim 1,

The deep learning model used in the shape change reading unit 400 and the color change reading unit 500,

Using a pre-learned Convolutional Neural Network (CNN) algorithm, the CNN (Convolutional Neural Network) algorithm uses the change result table information for shape and the change result table information for color to perform basic learning. , Portable in vitro diagnostic kit analysis device using multimedia information, characterized in that to determine the effectiveness by measuring sensitivity and specificity through a confusion matrix in order to determine the effectiveness at which basic learning is completed.
The method of claim 2,

The read result output processing unit 600,

The multimedia information of the in vitro diagnostic kit is labeled and stored in the user labeling information storage module, and the result values read through the shape change reading unit 400 and the color change reading unit 500 are stored in the corresponding user labeling information storage module. Portable in vitro diagnostic kit analysis apparatus using multimedia information, characterized in that transmitting the stored labeling information to the corresponding user terminal (2000).
The method of claim 1,

In vitro diagnosis method determination unit 100,

Portable in vitro diagnostic kit analysis device using multimedia information, characterized in that the device is connected to a camera to receive a direct input image or received from a wireless network or an Internet network to obtain multimedia information of the in vitro diagnostic kit.
The method of claim 1,

The shape change reading unit 400,

A portable in vitro diagnostic kit analysis device using multimedia information, characterized in that the output value of the sample in which the shape change has occurred is a learning result value learned in advance using a deep learning model.
The method of claim 1,

The color change reading unit 500,

A portable in vitro diagnostic kit analysis device using multimedia information, characterized in that the output value of the sample in which the color change occurs is a learning result value learned in advance using a deep learning model.