RO128001A0 - Mobile automatic neural network-based system for classifying focal hepatic formations - Google Patents

Abstract

The invention relates to a mobile automatic system based on neural networks for classifying focal hepatic formations and tumours. According to the invention, the system comprises a mobile unit (4) for the acquisition of patients' input data which are then converted into logic values and introduced into an input layer of 15 neurons (10) together with some sequences of values resulting from the analysis of the time variation of maximal colour intensity in some zones of interest comprising a tumour formation and a non-tumour zone, in a video record of the echographic investigation with contrast agent, by means of a matrix which contains a variable number of input neurons (17), and a central computerized system (6) which contains an information taking over interface (7) a data base (8) and an assembly of neural networks (9) comprising two consecutive neural networks of 15 input neurons (10) and two neurons (15 or 16), respectively, and a matrix (17) of input neurons which takes over data from a video record of the echographic investigation with contrast agent, an intermediate layer (11) and two intermediate layers (18 and 22), respectively, and an output layer (26) of five neurons processing the information, to offer the doctor an immediate response by means of the same mobile device, the said response being saved, as well, into the data base (8), the doctor introducing then a value (27) of classification to re-train the system offering new weight factors for some input values (13, 20 and 24).

Description

MOBILE AUTOMATIC SYSTEM BASED ON NEURONAL NETWORKS FOR 9

CLASSIFICATION OF FOCAL HEPATIC FORMATIONS

I

b. specifying the technical field

The invention relates to an automated mobile system based on neural networks for the classification of focal liver formations, hepatic turnor, and, in particular, to an automated system and method, based on neural network and computers for increasing the accuracy of classification of discovered liver tumors. by ultrasound methods.

The system consists of a mobile device for recording and transmitting data to a central computer, which performs the automatic classification of focal liver formations and in particular hepatocellular carcinoma, after processing with the help of learning techniques based on neural networks the information provided by a ultrasound equipped with contrast agent evaluation techniques, together with clinical and paraclinical data of the investigated patient.

c. presentation of the state of the art

Ultrasonography has the benefit of evaluating the blood supply to the liver and the presence of vascular turnar invasion. New technologies, in particular the use of contrast agents in ultrasound examination, can improve the accuracy of this method in the classification of focal liver formations. Current diagnostic guidelines recommend Contrast Enhanced UltraSonography (CEUS), along with other imaging techniques (computed tomography or contrast-enhanced magnetic resonance imaging) among the techniques that can be used to correctly classify liver rotator cuff formations. The CEUS technique involves injecting an intravenous bolus of contrast substance made up of microbubbles with a diameter between 2 and 7 microns. Under the action of the frequencies generated by an ultrasound transducer, the microbubbles vibrate, producing an amplified signal which is received, by special techniques, by the same ultrasound transducer.

Movies can be recorded in real time on the magnetic media storage unit with which the ultrasound is equipped. These can be analyzed later by comparative measurement of the average intensity of the contrast in two areas of interest drawn by the operator on the ultrasound image, one corresponding to the turnor area and one c \ -2 Ο 1 2 - Ο Ο 3 9 2 - 1 * 1 -μ - DAY »of the normal parenchyma. Thus, two intensity-time curves are drawn, from which a series of quantifiable parameters can be extracted, such as the maximum intensity, the area under the curve, the time until the maximum intensity is reached. Each aspect of the two curves is considered pathognomonic for a certain type of liver tumor, the analysis of an experienced operator can successfully classify these formations.

Neural networks represent the translation into computer language of the principles of functioning of the human central nervous system. In this way, the basic components of the brain, namely neurons and neural synapses are reproduced through dedicated software applications, creating networks that have the ability to retrieve and process information to solve complex problems, in addition to the ability to learn and learn. make decisions.

Neurons, the basic unit of these networks, are placed in visible or "hidden" layers (hidden layers), being interconnected through synapses, which ensure the transmission of information between each layer. Each variable that is entered in the system is taken over by a neuron, which assigns a value, a "weight" (weighf), respectively the degree of importance in the problem. These values can be both positive and negative in the dynamics, thus ensuring at all times a correct evaluation of the variables. All the information taken is then transmitted through cross-links to the next layer, which contains fewer neurons but receive more information than the first, thus making logical associations between the data of the problem. They in turn can communicate with other "hidden" decision-making layers, ultimately resulting in a solution or a logical assessment of the situation presented.

In order to be able to produce the most accurate results, neural networks must be trained, either by a human operator (supervised training) or by intrinsic systems (self / unsupervised learning networks). Redundant data are eliminated by these processes, the networks being able as a result of this process to correctly evaluate each variable entered, in order to make the correct decision at the end of the logical cycle.

Currently, there are several systems and methods based on the application of neural networks in the diagnosis and classification of various human preparations, with a predilection in the field of pathological anatomy (such as, for example, RO 106931 B1, which describes a system and an automatic method on basis of neural network for the classification of cytological specimens), as well as strictly statistical diagnostic methods, such as the one proposed by Gyorfi (patent RO 00121981 - “Diagnostic method, assisted 2 ^ - 2 0 1 2 - 0 0 3 9 2 -PI -06- 2012 computer, chest pain ”). However, they either do not have immediate clinical applicability or do not integrate concepts applicable to a condition as complex as liver cancer.

The results were variable, depending on the quality of the parameters introduced and the efficiency of the procedures used in the training.

The neural networks used so far in medical studies and especially in the classification of malignancies have some disadvantages:

Wrong selection of input variables, either too numerous or of no clinical significance;

A small number of cases involved in the training process, which led to the choice of too specific variables, the cases being "ideally" included in the diagnostic model pursued;

Erroneous estimation of the error rate, resulting from improper validation;

Stopping the learning process after a certain number of cases, the diagnostic system thus lacking the plasticity and adaptability that should have been defining.

d. presentation of the technical problem

The technical problem solved by the invention is the improvement of the diagnostic method assisted by an expert system based on neural networks of liver cancer, also able to classify other types of malignant or benign liver tumors.

e. disclosing the invention as claimed

The automatic system based on neural networks for the classification of focal liver formations according to the invention consists of a mobile unit (4) for entering a patient's data converted into logical values (10), together with the ranges of values resulting from the analysis of time variation of the maximum color intensity in the areas of interest comprising the tumor formation and a non-tumor area, from the video recording of the contrast-enhanced ultrasound investigation (17) and a central computer system (6) containing an information retrieval interface (7) ), a database of ^ 2012-00392-0 f -06- 2012 data (8) and a set of neural networks (9), consisting of two consecutive neural networks, with 15 (benchmark 10) and two input neurons, respectively ( 15 or 16 and 17), one (11) and two intermediate layers (18, 22), respectively, and an output layer with five neurons (26), which processes the information, providing the doctor with an immediate response via the same mobile device, the answer is also saved in the database (8), the doctor entering a value of the classification (27) which re-trains the system offering new weights for the values entered (13, 20, 24).

f. presenting the advantages of the invention

Data is entered in real time from any location where there is Internet access, allowing virtually any use of the system in almost any existing clinical context.

All parameters are stored in a dynamic database, from where they can be used later for successive trainings, once the diagnoses are validated by a clinician. Thus, one of the main drawbacks of classical neural networks is eliminated, namely the stopping of the active learning process at a certain level.

Validation can be done at any time by human operators, thus eliminating any possibility of erroneous estimation of the error rate.

The number of cases introduced into the system is practically unlimited, there are always reference medical centers capable of providing new results for the improvement of the system.

The data are entered in real time using the dedicated device. The recording and processing of data is carried out centrally, within the dedicated system for this purpose, at a distance.

The quality of the introduced variables results from the follow-up of the existing guidelines and statistics, with pre-existing clinical validation, being possible their dynamic readjustment, thus eliminating the inconvenience related to their wrong selection.

The use of imaging parameters resulting from the introduction of the ranges of values recorded during the analysis of contrast-enhanced ultrasound films.

To the parameters already described above are added three new variables, important in classifying the type of liver tumor. Thus, the system uses the ratio between the maximum intensities of the two reference areas, which differ depending on the type of liver tumor. The diagnosis is also based on the appearance of the load with contrast agent (centripetal loading is specific to hepatic hemangioma), as well as on ίΧ- 2 Ο 1 2 - Ο Ο 3 9 2 - d 4 -06- 2012 decrease or keeping the intensity constantly high in the tumor area compared to the control area (presence or absence of tumor wash-out).

It can be used to accurately determine the origin of a liver formation in the context of a classic consultation, as well as in telemedicine applications, and can be an expert reference system for remote centers. It also has didactic utility, being able to be used in the training of resident or trainee doctors, as well as in practical applications made by students of a medical university.

The modularity of the system allows it to provide a result even in the absence of one or more investigations, with the appreciation of the degree of safety. The system is thus flexible and eliminates the subjectivity of a human operator.

An embodiment of the invention is given below in connection with FIG. 1,2,3 and 4, which show:

Figure 1, a block diagram of the mobile automated system based on neural networks for the classification of focal liver formations and the prognosis of liver cancer. Figure 2, representation of the neural network of the type used in the system.

Figure 3, presentation of the interface A

Figure 4, presentation of the B interface

h. presentation of at least one embodiment of the invention

Hepatocellular carcinoma is a malignant tumor developed in the liver, currently ranked third in mortality worldwide, being the fifth incidence. Most often, the diagnosis of hepatocellular carcinoma is suspected after the observation on a classic ultrasound of a liver tumor, in a patient with a history of liver damage, and its objectification is done by a contrast-enhanced imaging method.

Ultrasound technique based on contrast agents was introduced in the diagnosis of primary liver cancer to improve the accuracy of locating a liver formation following a classic ultrasound investigation. It is based on the intravenous injection of a quantity of contrast agent, which once reached in the liver can be ultrasound monitored by a special probe and plotted

Ο 1 2 - 0 Ο 5 9 2 - (Μ -06- 2012 as ο image of increasing intensity as it accumulates in the blood vessels in the field of vision of the ultrasound probe. The investigation takes place over 2 to 5 minutes, during which the contrast substance crosses the arterial vascular compartments and the hepatic portal, thus distinguishing three distinct phases: arterial phase, venous phase and late phase.The tumor will capture the contrast agent differently compared to the surrounding parenchyma due to the newly formed vascularization This is visually recorded by the observer and the different evolution during the three phases, specific to each type of liver tumor, is interpreted.Eltrasound recordings can be saved on a magnetic medium during recording for later viewing. can conceive the so-called “intensity-time curves”, respectively determinations made in each frame of the recording, of the intensity ii averages for one or more areas of interest selected by the human operator. An area that overlaps the liver tumor, and a reference area outside the area considered to be the tumor, are usually chosen. The result is two rows of values, which are represented graphically by providing contrast-filled graphs, where the abscissa represents the recorded time interval, and the ordered maximum value in each frame. A number of parameters are thus calculated, such as the maximum intensity, the time until the maximum intensity is reached, the average transit time of the contrast agent or the area under the curve. The tracing of the areas of interest and the obtaining of the strings of values can be done by the ultrasound, or it can be done later within the interface of the mobile device.

The system uses the clinical data obtained by the attending physician during the anamnesis and the objective clinical examination, the data obtained from laboratory investigations, as well as a series of imaging parameters resulting from the simple ultrasonographic examination and contrast agent. All parameters are recorded in real time by any clinician, the system itself being of the "cloud-computing" type, data entry is done through a remote graphical interface, using the dedicated mobile device, permanently connected to the Internet.

In the interface of the mobile unit, the doctor enters the relevant clinical information to establish the origin of the liver formation, to which are added the data from imaging investigations (standard ultrasound or contrast agent) in the form of numerical parameters. The module sends the data via a permanent Internet connection (mobile data subscription or existing wireless network) to a central computer system, where they are entered into a database, from where the network ^ -2012-00392-0 I -06-2012 neuronal processes information and gives the doctor a specific answer.

The system uses a series of parameters obtained in the ultrasonographic investigation based on contrast agent injection techniques, to improve the decision-making capacity of a neural network for the purpose of differential diagnosis of a liver tumor mass. Clinical and paraclinical data are of particular importance in the diagnosis of any condition, but in the case of liver tumors the diagnosis of certainty must be based on current treatment guidelines on at least one imaging investigation with contrast-enhanced Di (EFSUMB Study Group et al. Guidelines and Good Clinical Practice Recommendations for Contrast Enhanced Ultrasound (CEUS) -Update 2008. Ultraschall in Med 2008; 29: 28-44. DOI 10.1055 / S-2007-963785).

The neural network is a set of decision-making structures called neurons, whose connections are modulated during three consecutive stages, namely training, validation and testing. The layers formed by them are of three types: input, decisional and output. In the input layer, the data in numeric format are entered into the system, where each parameter is assigned a weight, which ultimately counts for the final result, following the nonlinear calculation based on those weights. During the training process, the weights are adjusted accordingly so that the result is close to that expected. Neural networks applied in medicine have been used in medical applications in the past, in the process of diagnosis, in predicting risks or in assessing the prognosis of diseases, including cancer.

According to the invention, an operator 0 records with the probe 1 in the ultrasound scanner 2 the ultrasound film, which is transferred by means of a mobile transfer device 3 to the mobile device 4. Through the graphical interface 5, the operator 0 enters the data from categories I ... V of the table 1 in interface A and processes the recorded ultrasound film, using interface B, obtaining the parameters in Table 1, category VI, according to the method described below. It also introduces a possible classification according to category VII in table 1.

The mobile system 4 sends the data via a secure wireless connection to the central unit 6, a personal computer connected to the Internet, with a unique address, containing the data reception application 7, the database 8 (whose structure is described in table 1 ) and the neural network 9 .described in detail in Figure 2. The database contains numerical values that encode the information entered by the doctor in 7

2012-0 0 392-0 | '06 - 2012 interface A and extracted from interface B.

The neural data processing department. Figure 2, consists of a cascade of two neural networks. The values encoded in the database from categories I ... V in table 1 are represented by the general reference 10-15 neurons in the input layer. The hidden layer 11 contains 15 neurons that assign a weight to each parameter (denoted by the general reference 12), to which a bias is added (general reference 13) and which pass through a sigmoid function (general reference 14). Thus, two output values are obtained, respectively 15 and 16, corresponding to the classification as malignant and benign of the liver formation, respectively. Imaging data collected through the B interface are added (general reference 17), and are introduced as input neurons in a new hidden layer (18), where they are assigned a weight (19) and a bias (20), passing through a sigmoid function (21) in a hidden layer (22) where they are re-evaluated, being given a new weight (23), bias (24) and passing through a linearization function (25). In the output layer (26), five possible values are obtained, corresponding to a class of liver tumor. The result is transmitted to the mobile unit (4) in interface A, the data also being entered in the database (8). The classification introduced by the doctor (category VII in table 1, respectively benchmark 27) is used for the re-training of the neural network, giving a new weight in layers 11, 18 and 22.

A neural network is a system of decision-making structures called neurons, elements of information retrieval or processing. connections are established between them, these assemblies being organized in consecutive layers: one input layer, one or more decision layers and one output layer. In the input layer, the data in numeric format is entered into the network. In the intermediate layer or layers, each parameter is given a weight (also called weight), which ultimately counts for the final result, following the nonlinear calculation based on those weights. These neurons have attached a transfer function, which operates from the sum of all products parameter x weight, producing a result. The weights are modulated during three consecutive stages, namely training, validation and testing. During the training process, the weights are adjusted accordingly so that the result is close to that expected.

To carry out the present invention, the preferred embodiment uses a set of two consecutive neural networks, which operate dependent on each other. The first network (assembly 10 ... 16) consists of an output layer, an input layer, and a hidden intermediate layer. It provides a classification of malignant formation 8 cv2012-00392-Q | -06- 2012 and benign, starting from the 15 input neurons (10), corresponding to the 15 parameters presented in table 1, the information taken from the database (benchmark 8 in figure 1). The classification of the type of formation is made numerically, in the range [O; 1]. The range [0; 0,5) corresponds to the classification as benign (15), the range [O; 1] corresponds to the classification as malignant (16). The second neural network contains two neurons in the input layer (15 or 16, 17), two hidden layers (18 and 22) and an output layer with five neurons (26). A classification is obtained in layer 26 according to the range to which the value obtained belongs, as follows: [-0.5; 0.5) = focal steatosis; [0,5,1,5) = hepatocellular carcinoma; [1,5,2,5) = hypervascular metastasis; [2,5,3,5) = hypovascular metastasis [3,5; 4,5) = hepatic hemangioma.

Table 1. Structure of the data entry application and the corresponding database.

variable Variable type Special mentions Value recorded in the database /. General data Sex binary Male / Female 1/0 Age Default values e <30 years, 30-60,> 60 years. 0/1/2 Origin of origin binary Urban / Rural 1/0 II. Risk factors Alcohol consumption binary Abuse / normal 1/0 Liver viruses stock B + C / B / C 2/1/0 ///. Background person significant pathologies Cirrhosis of the liver Default values e None / A / B / C / D 0/1/2/3/4/5 IV. Tumor markers Alpha-fetoprotein value binary Over limits / Normal 1/0 Alloy tumor markers binary Over limits / Normal 1/0 V. Standard ultrasound hepatomegaly binary Present / Absent 1/0 Dimension Value Value in millimeters 1 ... n Number Value Number 1 ... n Malignant portal vein thrombosis binary Present / Absent 1/0 VI. Ultrasound with contrast agent Maximum intensity of stock Value expressed 1 ... n

C <- 2 Ο 1 2 - Ο Ο 3 9 2 - 0 | Ό6- 2012

signal from the tumor area relative to the parenchyma centripetal loading binary Yes No 1/0 Terminal phase appearance Default values e Total washout / Partial washout / washout absent 2/1/0 ARE YOU COMING. Certainty diagnosis Classification introduced by the doctor Default value HCC / Hypovascularized tumor / Hypervascularized tumor / Hem angioma / Focal steatosis 1/2/3/4/5

Ο a description of how the method works is given below:

The doctor examines the patient by the usual methods. An ultrasound examination identifies a liver tumor. Consecutively, the ultrasound examination with contrast agent is performed, according to the classical protocols.

Upon completion of the contrast investigation, the recording is uploaded to the mobile device using a mobile recording device (eg a USB stick).

The doctor uses the mobile device to enter the clinical and paraclinical data recorded during the objective examination and results of laboratory tests, then processes the obtained film, extracting the areas of interest and calculating the intensity-time curves. The mobile device extracts the relevant data and adds it to the data set entered by the doctor for transmission.

The data is transmitted remotely to the central system, where it is processed.

The neural network takes over the significant data and in a few seconds a result is offered.

The doctor receives a classification of the liver tumor he is facing, accompanied by a probability of correctness and possible alternatives, a staging, with the existing therapeutic possibilities and a possible prognosis for the patient.

The doctor introduces a personal diagnosis, which will later be used in a new neural network training procedure, if the diagnosis is confirmed over time.

Claims (1)

Semi-automatic neural network-based system for the classification of identified focal liver formations characterized in that it includes a mobile unit (4) for entering a patient's data converted into logical values (10), together with the value ranges resulting from the analysis of variation in time of maximum color intensity in areas of interest comprising the tumor formation and a non-tumor area, from the video recording of the ultrasound investigation with contrast agent, (17) and a central computer system (6) containing an information retrieval interface ( 7), a database (8) and a set of neural networks (9), consisting of two consecutive neural networks, with 15 (benchmark 10) and two input neurons (15 or 16 and 17), respectively, one (11) ) and two intermediate layers (18, 22), respectively, and an output layer with five neurons (26), which processes the information, giving the doctor an immediate response via the same mobile device, the answer be nd also saved in the database (8), the doctor entering a value of the classification (27) which re-trains the system offering new weights for the values entered (13, 20, 24).