WO2022216173A1 - Artificial intelligence-based diagnosis of pathologies - Google Patents

Abstract

The invention relates to the field of medicine and can be used as a medical assistant for making diagnoses using artificial intelligence technology. In a method for diagnosing pathologies using artificial intelligence technology, implemented by a computing device, an image of a biopsy specimen is obtained containing a pathology. The obtained biopsy image is analyzed at a low resolution to obtain a thumbnail image which is searched for regions containing tissues, and said regions are clustered into larger objects. Patches are read at a high resolution. The patches obtained in the previous step are run through a segmenting artificial neural network (ANN) to obtain a multichannel mask containing classes of pathologies. The mask containing classes of pathologies is vectorized for representation in the form of polygons. The polygons are grouped according to biopsy specimens, from which they are extracted to obtain a provisional diagnosis for each biopsy specimen. The following are determined for each biopsy specimen: the total area of damage, the length of damage, the type of pathology and the Gleason score to allow more accurate and rapid diagnosis by a doctor with the aid of digitized histological slides.

Description

DIAGNOSTICS OF PATHOLOGIES ON THE BASIS OF ARTIFICIAL

INTELLIGENCE

FIELD OF TECHNOLOGY

[001] The present technical solution generally relates to the medical field of technology, and in particular to methods and systems for diagnosing pathologies based on artificial intelligence technology, and can be used as a doctor's assistant for diagnosing based on artificial intelligence technology.

BACKGROUND OF THE INVENTION

[002] Currently, companies are known from the prior art, both having developed analogues and in the process of developing them.

[003] For example, the prior art application US20200364587A1 "Systems and methods for processing images to classify the processed images for digital pathology" (publication date: 2020-11-19, copyright holder: PAIGE Al INC). This solution discloses systems and methods for obtaining a target image corresponding to a target sample, the target sample includes a patient tissue sample, applying a machine learning model to the target image to determine at least one characteristic of the target sample and / or at least one characteristic of the target image. The machine learning model was generated by processing a plurality of training images to predict at least one feature, the training images contain images of human tissues and/or images that are algorithmically generated, and output at least one feature of the target sample and/or at least one feature target image.

[004] Also known from the prior art is patent US10614285B2 “Computing technologies for image operations” (publication date: 2020-04-07, copyright holder: PROSCIA INC). The method includes: obtaining an image with a tissue through the processor; image quantification through a processor based on: segmentation through an image processor into a plurality of segments; identifying through the processor a plurality of histological elements in the segments; forming through the processor a network graph containing a plurality of nodes, in which the histological elements correspond to the nodes; measuring with the help of the processor characteristics of the network graph; performing, through the feature-based image transformation processor; determining by the processor a non-parametric feature of the image based on the transformation; saving through the processor of non-parametric characteristics in the database.

[005] However, existing commercial systems do not allow for the accuracy of digital diagnostics.

ESSENCE OF THE TECHNICAL SOLUTION

[006] The technical result achieved by solving the above technical problem is to increase the accuracy and speed of the doctor's diagnosis by means of digitized histological glasses.

[007] An additional technical result is the reduction of errors associated with the human factor by automatically marking the slide, instantly determining the pathological areas of the tissue in the images and the type of pathology, as well as making a diagnosis.

[008] The specified technical result is achieved by implementing a method for diagnosing pathologies based on artificial intelligence technology, performed by at least one computing device, in which at least one biopsy image containing a pathology is obtained; searching for areas containing tissue (as an optimization, the search can occur on a lower resolution slide), as well as clustering the areas into larger objects; read patches in high resolution; passing the patches obtained in the previous steps through a segmenting artificial neural network (ANN) to obtain a multi-channel mask with pathology classes; carry out the vectorization of the mask with classes of pathologies for presentation in the form of polygons; group polygons according to the biopsy specimens from which they were extracted for obtaining a preliminary diagnosis for each of the biopsy specimens; determine for each biopsy the total area of the lesion, the length of the lesion, the type of pathology and the Gleason sum.

[009] In some implementations of the technical solution, patches are read with overlap to reduce prediction artifacts due to edge effects.

[0010] In some embodiments of the technical solution, the multi-channel mask has the number of channels equal to the number of detected pathologies.

[0011] In some implementations of the technical solution, vectorization is carried out by representing in the form of coordinates for each class.

[0012] In some embodiments of the technical solution, when performing vectorization for each class, pathological contours are searched.

[0013] In some embodiments of the technical solution, the search is carried out using the OpenCV library.

[0014] In some implementations of the technical solution, polygons obtained from neighboring patches are combined to eliminate joints between them in the final rendering.

[0015] In some implementations of the technical solution, the ANN is modified into an encoder.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The features and advantages of the present technical solution will become apparent from the following detailed description and the accompanying drawings, in which:

[0017] In FIG. 1 shows an example of the implementation of a method for diagnosing pathologies based on artificial intelligence technology. [0018] In FIG. Figure 2 shows a variant of the implementation of a graphical user interface that displays slides with the ability to edit the markup of the resulting image from pathologies. [0019] In FIG. 3 shows an implementation of a graphical user interface displaying slides with pathological diagnoses on the left side of the interface.

[0020] In FIG. 4 shows an example of the implementation of a system for diagnosing pathologies based on artificial intelligence technology in the form of a block diagram.

[0021] In FIG. 5 shows an example implementation of a table with Cohen's Kappa metrics (Cohen's Carr).

DETAILED DESCRIPTION OF THE INVENTION

[0022] Below will be discussed in detail the terms and their definitions used in the description of the technical solution.

[0023] In this invention, the system refers to a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of operations (actions, instructions), centralized and distributed databases, smart contracts.

[0024] A command processing device refers to an electronic unit or an integrated circuit (microprocessor) executing machine instructions (programs), a smart contract, an Ethereum virtual machine (EVM), or the like. An instruction processing device reads and executes machine instructions (programs) from one or more data storage devices. The role of a storage device can be, but not limited to, hard disk drives (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical drives.

[0025] A program is a sequence of instructions intended to be executed by a computer control device or command processing device.

[0026] Biopsy - is a diagnostic method of research, which consists in excising the tissues of a certain organ or taking a suspension of cells, carried out in a living organism, with the aim of subsequent microscopic examination, carried out after processing the drug with special dyes.

[0027] Slides are digitized glasses (biomaterial of the patient), according to which the pathology is determined and the diagnosis is made.

[0028] A biopsy is a material obtained by biopsy. In most cases, a biopsy is a direct research method that allows you to accurately diagnose a particular disease. The biopsy method has a particularly important information for the recognition of tumors.

[0029] Histological examination - is the study of a specific tissue area of interest under a microscope. These tissue areas are dehydrated using specially prepared solutions. The tissues become fat-soluble, and then they are moved into pre-prepared molds and soaked in paraffin, taking the form of solid cubes. After that, using a special knife with an ultra-thin blade, small sections are made with a thickness of no more than 3 micrometers.

[0030] In a specific implementation, slides (where histological examination is used) will be described as an example of images with pathology, i.e. digitized slides, however, it is obvious to any person skilled in the art that any known types of pathological images (eg, MRI, X-ray, etc.) can be used. [0031] In a specific embodiment, when examining areas of tissues or organs located near the surface of the skin, a puncture biopsy is used using specific thin needles that are inserted directly into the area under study. The tissue column located in the lumen of the needle is sent to the laboratory for research, where this technical solution is used in the study of the biopsy.

[0032] This technical solution can be implemented as a computing system with a microservice architecture, consisting of three main parts.

[0033] The first component is an artificial neural network (hereinafter referred to as ANN), which generates a slide layout and a diagnosis and, for example, is a modified Resnet-34. In some variants implementations as such a network can be used by Unet, LinkNet. Similarly, ResNet-50, ResNet-101, HRNetV2-W18, HRNetV2-W32, HRNetV2-W48 can be used as an artificial neural convolutional network.

[0034] In a specific implementation example, the ANN detects the following pathologies, but is not limited to: a. Atrophy;

B. Adenocarcinoma; i. Gleason 3 ii. Gleason 4 iii. Gleason 5 c. foamy cell carcinoma; d. ductal cancer; e. PIN (Prostatic intraepithelial neoplasia); f. chronic inflammation;

[0035] The second component is a module for viewing an image with a pathology, for example, slides (with and without markup) and a diagnosis, is configured to edit the markup of the resulting image from pathologies, for example, a slide and a diagnosis, manually marking a slide and making a diagnosis (without participation INS) as shown in Fig. 2. An image in some implementations can be received from a client to a server, from a database of images with pathologies, or from any other external system. In order for a technical solution to produce a result in the form of a definition of a pathology on a slide and form a conclusion, it is preliminarily trained. One of the stages of this training is the preparation by doctors of a data set (a set of slides, i.e. digitized glasses), which is the so-called manual markup. This preparation may include the marking of these slides by doctors in a separate program or within a technical solution by area (areas of tissue damage / biomaterial of the patient). Upon completion of the learning process of the artificial neural network, the technical solution can automatically mark up the slides loaded into it. The doctor can correct the automatic markup of the system if he does not agree with it, and also send his own version and the version of the system to receive a third b other doctor's opinions. The performed automatic markup is a selected area on a slide with a pathology (label). In some implementations, the ability to change the label and add a description to the area.

[001] Another component of the system is a graphical user interface ("GUI") with a personal account (hereinafter - LC) and a database (hereinafter - DB), which can be stored on the server. Registration is required to use the web interface. LC allows you to create "working areas" (hereinafter - RO) (some analogue of a medical institution or laboratory). RO is needed to isolate the work of doctors in various medical institutions. institutions, in particular for the protection of personal data. It is possible to add users to the workspace with different levels of access to information, for example:

- owner (full access to all RO slides)

- doctor (only has access to his own slides)

- supervisor (has access to his own slides and slides assigned to the cases of all RO doctors)

- administrator (any of the users shown above can be an administrator, can change user access levels).

[002] Within the workspace are "localizations" (in a specific implementation, the prostate). Having entered the localization, they find two blocks (“Cases” (can be interpreted as a patient) and “Slides” (downloaded slides and not added to the Case). It is possible to view slides from both the case and from the Slides block. Each case can contain one or more slides (usually belonging to one patient) For automatic processing (marking up and making a diagnosis), the corresponding button is pressed The entire case is sent for processing using ANN If desired, you can mark up a slide and make a diagnosis in manual mode, for which each slide separately.

[003] This technical solution is one or more interacting artificial neural networks. Each neural network works on a certain slide scale and is aimed at finding its features. This approach takes place because that some signs appear only at the macro level, while others can only be recognized at the cellular level. In a specific implementation example, the slide is processed by one or more ANNs with different levels of detail, and upon completion of the analysis and collection of all found features, the machine learning model decides whether each area belongs to a certain class of pathologies.

[004] Previously, in the first step, the model is assembled on previously trained scales. Assembling the model means loading the weights into the well-known ANN architecture. In this technical solution, these are the weights of the previously trained machine learning model. In this implementation, the analyzed slide is read in low resolution to obtain a thumbnail. The miniature searches for areas containing tissue, as well as clustering areas into larger objects, since one slide can contain a series of sections of a paraffin block with one or more biopsy specimens (columns). The slide is read in grayscale and a median filter is applied to it, and then the extreme values are cut off by a threshold. Thus, there is a rejection of "peaks" in the image. Rejection of the "plateau" can occur using the Laplace transform. As a result of these actions, a binary mask for the slide is obtained, in which 0 is the background value, 1 is the value for tissues. Thus, areas are selected for a more detailed analysis using ANN.

[005] High resolution patches are read from the selected areas. A patch is a fixed-size image (usually a square shape) cut out of a slide. The patches are read with overlap to reduce prediction artifacts due to edge effects. These patches are passed through a segmenting ANN to obtain a multi-channel mask. A normal image has 3 channels: Red, Green, Blue or RGB for short. The multi-channel mask will have the number of channels equal to the number of classes, that is, the number of detected pathologies. As an example of an ANN, models known from the prior art, for example, from information sources [2], [3], can be used. A segmented ANN is an ANN that assigns a label to each pixel in an image. Pixels with the same labels have common visual characteristics. The thus obtained mask with classes is vectorized for a more compact representation in the form of polygons. The number of classes is taken from the number of detected pathologies and healthy tissue. One for each pathology plus one more for each Gleason grade. In a specific implementation example, 9 classes + 1 = 10 are obtained. A multi-band mask is a bitmap representation. One of the disadvantages of this representation is the speed of further processing. The vectorization process is a representation in the form of coordinates for each class. For each of the channels, that is, for each class, the contours are searched, and hence the coordinates. The search can be carried out using the OpenCV library. Polygons obtained from adjacent patches are merged to eliminate joints between them in the final rendering. The polygons are grouped by the biopsies (bars) from which they are extracted to obtain a preliminary diagnosis for each of the biopsies (bars), as shown in FIG. 3, both on the server side and on the client side. For example, a doctor, for some reason, changes the contour of the biopsy specimen (column), accordingly, on the client side (in a web browser), the polygons belong to the new boundaries of the column and form a new conclusion on the biopsy specimen (column)

[006] For each biopsy specimen (bar), such characteristics as the total area of the lesion, the length of the lesion, the most common and most aggressive types of cancer (if any) and the so-called Gleason sum are calculated. The Gleason sum is one of the signs of determining the degree of pathology for making a conclusion. The Gleason score is determined by adding the scores (on a 5-point scale) of the two most characteristic areas of the tumor biopsy. Gleason scores can range from 2 to 10, with 2 being the least aggressive tumor and 10 being the most aggressive. The Gleason sum is sometimes called the Gleason scale. To make a diagnosis for one patient, the doctor needs 10 to 12 of these patient tissue samples. They are also called columns. The Gleason score (Gleason sum) is used for histological assessment of the differentiation of prostate cancer (small values of the scale correspond to highly differentiated forms, and high values correspond to poorly differentiated). High values on the Gleason scale are associated with a poor prognosis of the outcome of the disease. A biopsy of the prostate is carried out, then the two most characteristic areas of the biopsy are evaluated on a five-point scale. One point means the highest degree of differentiation, and 5 means the lowest. The "Gleason sum" obtained as a result of adding these estimates varies from 2 (1 + 1) to 10 (5 + 5) points. Poorly differentiated tumors (that is, those with a high Gleason score) tend to be more aggressive (spread and metastasize faster), but respond better to chemotherapy and radiation therapy than well-differentiated ones. Together with other assessment methods, the Gleason score helps classify prostate carcinoma, assess prognosis, and select optimal therapy.

[007] All the information received (polygons and a preliminary diagnosis, which can be compiled on the client side) are output to a file and / or to a database (DB) for further visualization through a graphical user interface.

[008] In a specific implementation example, the ResNet-34 model from the official PyTorch repository is taken as the basis. [1] The model is assembled unchanged in order to load the weights of the ImageNet dataset. Then this ANN with layers pretrained on ImageNet is modified into the so-called encoder by the following actions.

[009] The last two layers avgpool and fc are removed, since they are not used for segmentation. AvgPool2d - goes through the sliding window and takes the average value, and thereby reduces the size of the data. Fc is a fully connected layer that is responsible for classification, which is not needed in the problem being solved.

[0010] Next, the maxpool layer at the beginning is cut to save more contextual information. MaxPool2d goes through the sliding window and takes the maximum value, and thus reduces the size of the data. This item was selected empirically in the implementation example and allowed to increase the accuracy on the data set used.

[0011] The encoder converts the incoming images into a fixed length vector. The encoder is a set of Conv2d layers, ReLU, BatchNorm, AvgPool2d, MaxPool. Different models use different combinations with different parameters of these layers.

[0012] The final machine learning model is a modification of U-Net - SkipNet [2][3], which uses a modification of the ResNet-34 model as an encoder. The output is the Conv2d layer, which converts the features it contains into n_classes. The number of classes is taken from the number of detected pathologies plus healthy tissue. One for each pathology and one more for each Gleason grade. Total 9 classes + 1 = 10. In a specific implementation example, n_classes = the number of detected classes +1 (for an empty background). The final number of parameters in the model in a specific implementation example is ~ 21.5M.

[0013] The available slides are divided into training, validation and test datasets. To obtain a more balanced sample during training, a distribution was used in which data portions were selected with equal probability for each class.

[0014] Fragments of 256x256 in size with a 1 μm pixel are read from the slide, and then fed into the ANN. The choice of this approach is due to the maximization of information from images that are portioned into the model. Also, to reduce overfitting, training took place on randomly selected areas, and not on solid slides.

[0015] To provide greater data variability, augmentations were applied such as:

• Reflections of pictures vertically, horizontally. 100% cast chance

• Image quality reduction through jpeg codec compression. The probability of application is 50%.

• Elastic distortion of the image with the parameters scale = 1 , sigma = 50. For augmentations that change the size/shape of the selected patch, linear interpolation and mirroring of boundary areas were used. The probability of application is 50%.

• Adding changes in brightness. The probability of application is 50%.

• Mixing channels. The probability of application is 50%. • Change Hue and Saturation. The probability of application is 50%.

• Adding multiplicative noise with parameters low = 0.5, high = 1.5. The probability of application is 50%.

[0016] Rectified Adam [4] is used as an optimizer with weight_decay = 1e-2. The loss function is a measure of how well the ANN predicts the expected outcome. During the learning process, this function is minimized and the ANN parameters are updated to improve accuracy. Rectified Adam is a loss function minimization algorithm. During the training process, Rectified Adam slightly changes the parameters of the ANN so that it works more accurately. weight_decay - regularization method that adds a small penalty to the loss function according to the formula: loss=loss+weight_decay*L2_norm_weights loss - loss function weight_decay - a parameter that we set from outside

[0017] L2_norm_weights - L2 norm of model weights. This method is used to reduce the effect of overfitting the model.

[0018] To change the learning rate, the OneCycleLR scheduler from the official pytorch repository was used with the following parameters:

• Optimizer = Rectified Adam (weight_decay = 1e-2).

• Upper bounds on learning rate in a cycle for each group of parameters = 7e-4. The value was chosen empirically.

• Total number of steps in the cycle = length of the training dataset.

• Number of epochs = 1023.

• Percentage of a cycle (in number of steps) spent on increasing the learning rate = 0.3.

• Annealing strategy = cos.

• Cycle impulse = True.

• Basic moment = 0.85.

• Maximum torque = 0.95.

• Dividing factor = 25.

• Last division factor = 1e4. [0019] The loss function used was CrossEntropyLoss, whose behavior was adjusted so that unlabeled areas of the slide do not contribute to training.

[0020] During the learning process, the main metrics by which results were tracked were Accuracy[5] and Cohen's kappa[6]. The Accuracy metric tracks how well the ANN predicts a particular class, and Cohen's kappa tracks the consistency of ANN label and contour prediction with the doctor's label and contour.

[0021] The retention of learning outcomes occurs in terms of Cohen's kappa metric. The three best results that the ANN produces are stored. The weights of the model that produced the highest scores on Cohen's kappa metric are saved. On these three saved models, re-validation takes place, but already on the validation and test data sets, that is, on those slides on which the ANN was not learned. As a result, we calculate the following error matrix.

[0022] Cohen's kappa = 0.813

[0023] Cohen's kappa is a metric that indicates the consistency of the opinions of two experts, in our case these are the labels that the doctors noted and the labels that the ANN predicted. It is more correct to use it for unbalanced datasets, such as in this solution, since it is more resistant to class imbalance than the Accuracy metric.

[0024] An explanation of the metrics table is shown below and in FIG. 5:

[0025] The left column indicates in brackets the total number of contours for each of the studied classes. Above are the classes that ANN predicts. The following examples consider the upper left cell.

[0026] Each cell has 5 values (listed from top to bottom):

• Percentage of row-class contours that were classified as a column-class. For each contour, only those classes are taken that occupy more than 10% of its area, and the dominant one is taken among them. The sum of such values across rows is always less than or equal to 1;

• The absolute number of such circuits; • The absolute number of contours that crossed the threshold for classified pixels in 10% of the contour area (regardless of whether this class is dominant or not);

• The absolute number of contours with at least one pixel classified as belonging to this class;

• The proportion of such pixels in relation to the total area of contours of the given class.

[0027] The labels are arranged so that there are correct hits on the diagonal, and misses in the remaining cells.

[0028] The created technical solution can be practically implemented using a computing device. The user (doctor) can open the solution via the link on his PC, then registers in the LC of the solution, then he can upload digitized slides (histological images) from any folder on the PC and get the result from the solution in the form of a formed conclusion with a diagnosis.

[0029] Information confirming the possibility of implementing the appointment and achieving a technical result - internal testing of the product with the help of a team of doctors.

[0030] Referring to FIG. 4, this technical solution can be implemented as a computer system 400 for performing pathology diagnostics based on artificial intelligence technology, which contains one or more of the following components:

• a processing component 401 comprising at least one processor 402,

• memory 403,

• multimedia component 405,

• audio component 406,

• interface 407 input / output (I / O),

• sensor component 408,

• component 409 data.

[0031] The processing component 401 mainly manages all operations of the system 400, such as processing user data or a pathology search request, and managing display, phone call, data transmission, camera operation, and operation mobile communication device records. Processing component 401 may include one or more processors 402 executing instructions for completing all or part of the steps from the above methods. In addition, the processing component 401 may include one or more modules for convenient interaction between other processing modules 401 and other modules. For example, the processing component 401 may include a multimedia module for convenient, lightweight interaction between the multimedia component 405 and the processing component 401.

[0032] The memory 403 is configured to store various types of data to support the operation of the system 400, such as a user profile database. Examples of such data include instructions from any application or method, contact data, address book data, messages, images, videos, etc., all of which run on system 400. Memory 403 may be implemented as any type of volatile memory, non-volatile memory, or a combination thereof, e.g., static random access memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory device (ROM), magnetic memory, flash memory, magnetic disk or optical disk, and others, without being limited.

[0033] The media component 405 includes a screen that provides an output interface between the system 400, which may be installed on the user's mobile communications device, and the user. In some implementations, the screen may be a liquid crystal display (LCD) or a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input from a user. The touchpad includes one or more touch sensors in terms of gestures, touching and sliding on the touchpad. The touch sensor can not only sense the subject's touch boundary or swipe gesture, but also to determine the duration of time and pressure associated with the operation mode of touch and slide. In some embodiments, media component 405 includes one front camera and/or one rear camera. When the system 400 is in an operating mode, such as shooting mode or video mode, the front camera and/or rear camera can receive media data from outside. Each front camera and rear camera can be one fixed lens optics system or can have focal length or optical zoom.

[0034] The audio component 406 is configured to output and/or input an audio signal. For example, the audio component 406 includes one microphone (MIC) that is configured to receive an external audio signal when the system 400 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode. The received audio signal may be further stored in the memory 403 or routed through the communication component 409 . In some embodiments, the audio component 406 also includes a single speaker configured to output an audio signal.

[0035] An input/output (I/O) interface 407 provides an interface between the processing component 401 and any peripheral interface module. The above peripheral interface module may be a keyboard, steering wheel, button, etc. These buttons may include, but are not limited to, a start button, a volume button, a home button, and a lock button.

[0036] The touch component 408 includes one or more sensors and is configured to provide various aspects of assessing the state of system 400. system 400, the presence or absence of contact between a subject and system 400, as well as the orientation or acceleration/deceleration and temperature change of system 400. Sensor component 408 includes a proximity sensor configured to detect the presence of a nearby object when there is no physical contact. The sensor component 408 includes an optical sensor (eg, CMOS or CCD image sensor) configured for use in rendering an application. In some embodiments, the sensor component 408 includes an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor. [0037] The communication component 409 is configured to facilitate wired or wireless communication between the system 400 and other devices. System 400 may access a wireless network based on a communication standard such as WiFi, 2G, 3G, 5G, or combinations thereof. In one exemplary embodiment, the communication component 409 receives a broadcast signal or broadcast related information from an external broadcast control system via a broadcast channel. In one embodiment, communication component 409 includes a Near Field Communication (NFC) module to facilitate near field communications. For example, the NFC module may be based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0038] In an exemplary embodiment, system 400 may be implemented by one or more Application-Specific Integrated Circuits (ASICs), a Digital Signal Processor (DSP), a Digital Signal Processor (DSP), a Programmable Logic Unit (PLU), a logic chip programmable in operating conditions (FPGA), controller, microcontroller, microprocessor or other electronic components, and can be configured to implement a method 500 for performing pathology diagnostics based on artificial intelligence technology.

[0039] In an exemplary embodiment, the non-volatile computer-readable medium includes a memory 403 that includes instructions, where the instructions are executed by the processor 401 of the system 400 to implement the above-described methods for performing pathology diagnosis based on artificial intelligence technology. For example, a non-volatile computer-readable medium can be ROM, random access memory (RAM), compact disk, magnetic tape, floppy disks, optical storage devices, and the like.

[0040] Computing system 400 may include a display interface that transmits graphics, text, and other data from a communications infrastructure (or framebuffer, not shown) for display on media component 405. Computing system 400 further includes input devices or peripherals. Peripheral devices may include one or more devices for interacting with a user's mobile communications device, such as a keyboard, microphone, wearable device, camera, one or more audio speakers, and other sensors. Peripherals may be external or internal to the user's mobile communications device. The touch screen can display typically graphics and text, and also provides a user interface (such as, but not limited to, a graphical user interface (GUI)) through which a subject can interact with the user's mobile communication device, such as accessing and interacting with with applications running on the device.

[0041] The elements of the proposed technical solution are in a functional relationship, and their joint use leads to the creation of a new and unique technical solution. Thus, all blocks are functionally connected.

[0042] All blocks used in the system can be implemented using electronic components used to create digital integrated circuits, which is obvious to a person skilled in the art. Not limited to, microcircuits can be used, the logic of which is determined during manufacture, or programmable logic integrated circuits (FPGAs), the logic of which is set by programming. For programming, programmers and debugging environments are used, which allow you to set the desired structure of a digital device in the form of a fundamental electrical diagrams or programs in special hardware description languages: Verilog, VHDL, AHDL, etc. An alternative to FPGAs can be programmable logic controllers (PLCs), basic matrix crystals (BMCs), which require a factory production process for programming; ASIC - specialized custom-made large integrated circuits (LSI), which are significantly more expensive for small-scale and single-piece production.

[0043] Typically, the FPGA chip itself consists of the following components:

• configurable logical blocks that implement the required logical function;

• programmable electronic links between configurable logic blocks;

• programmable input/output blocks that provide communication between the external output of the microcircuit and the internal logic.

[0044] Blocks can also be implemented using read-only memories.

[0045] Thus, the implementation of all used blocks is achieved by standard means based on the classical principles of implementing the fundamentals of computer technology.

[0046] As will be appreciated by one of skill in the art, aspects of the present technical solution may be implemented as a system, method, or computer program product. Accordingly, various aspects of the present technical solution may be implemented solely as hardware, as software (including application software, etc.), or as an embodiment combining software and hardware aspects, which may be generally referred to as a "module" , "system" or "architecture". In addition, aspects of the present technical solution may take the form of a computer program product implemented on one or more computer-readable media having a computer-readable program code that is implemented on them.

[0047] Any combination of one or more computer-readable media can also be used. The computer-readable storage medium can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or any suitable combination thereof. More specifically, examples (non-exhaustive list) of a computer-readable storage medium include: an electrical connection using one or more wires, a portable computer diskette; hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or Flash memory), fiber optic connection, compact disc read only memory (CD-ROM), optical storage device, magnetic storage device or any combination of the above. As used herein, a computer-readable storage medium can be any flexible storage medium that can contain or store a program for use by or in connection with a system, device, apparatus.

[0048] Program code embedded in a computer-readable medium may be transmitted using any medium, including, without limitation, wireless, wired, fiber optic, infrared, and any other suitable network, or a combination of the foregoing.

[0049] The computer program code for performing the operations for the steps of the present technical solution may be written in any programming language or combinations of programming languages, including an object-oriented programming language such as Python, R, Java, Smalltalk, C++, and so on, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may run on the user's computer in whole, in part, or as a separate software package, in part on the user's computer and in part on a remote computer, or in full on remote computer. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN), a wide area network (WAN), or a connection to an external computer (eg, via the Internet via ISPs).

[0050] Aspects of the present technical solution have been described in detail with reference to block diagrams, circuit diagrams and/or diagrams of methods, devices (systems), and computer program products in accordance with embodiments of the present technical solution. It should be appreciated that each block from the block diagram and/or diagrams, as well as combinations of blocks from the block diagram and/or diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, a special purpose computer, or other data processing device to create a procedure, such that the instructions executed by the computer processor or other programmable data processing device create the means to implement the functions/actions specified in block or blocks of a flowchart and/or diagram.

[0051] These computer program instructions may also be stored on a computer-readable medium that can control a computer other than a programmable data processing device or other than devices that operate in a particular manner such that the instructions stored on the computer-readable medium create a device including instructions that perform the functions/actions indicated in the block diagram and/or diagram.

SOURCES OF INFORMATION USED

1. URL: htps://pytorch.org/docs/stable/torchvision/models.html?highlight=resne fflorchvision. models. resnet34, accessed 12/25/2021.

2. Ronneberger O., Fischer R, Brox T. U-net: Convolutional networks for biomedical image segmentation // International Conference on Medical image computing and computer-assisted intervention. - Springer, Cham, 2015. - C. 234-241.

3. Wang X. et al. Skipnet: Learning dynamic routing in convolutional networks // Proceedings of the European Conference on Computer Vision (ECCV). - 2018. - C. 409-424.

4. Liu L. et al. On the variance of the adaptive learning rate and beyond //arXiv preprint arXiv: 1908.03265. - 2019.

5. URL: https://en.wikipedia.org/wiki/Accuracy_and_precision , accessed 12/25/2021. 6. URL: https://en.wikipedia.org/wiki/Cohen%27s_kappa, accessed:

12/25/2021.

Claims

FORMULA

1. A method for diagnosing pathologies based on artificial intelligence technology, which is performed by at least one computing device and includes the following steps:

• receive at least one image of the biopsy;

• search for areas containing tissue, as well as clustering areas into larger objects;

• read patches in high resolution;

• the patches obtained in the previous step are passed through a segmenting artificial neural network (ANN) to obtain a multi-channel mask with pathology classes;

• vectorization of the mask with pathology classes for representation in the form of polygons;

• group polygons according to the biopsy specimens from which they are extracted to obtain a preliminary diagnosis for each of the biopsy specimens;

• determine for each biopsy the total area of the lesion, the length of the lesion, the type of pathology and the Gleason sum.

2. The method of claim. 1, characterized in that the patches are read with overlap to reduce prediction artifacts due to edge effects.

3. The method according to claim 1, characterized in that the multi-channel mask has the number of channels equal to the number of detected pathologies.

4. The method according to claim 1, characterized in that the search for areas containing tissue is carried out on a lower resolution slide.

5. The method according to claim 1, characterized in that vectorization is carried out by representing in the form of coordinates for each class.

6. The method according to p. 1, characterized in that when performing vectorization for each class, the contours of pathologies are searched.

7. The method according to claim 6, characterized in that the search is carried out using the OpenCV library.

8. The method according to claim 1, characterized in that the polygons obtained from neighboring patches are combined to eliminate joints between them in the final visualization.

9. The method according to p. 1, characterized in that the ANN is modified into an encoder.

10. A system for diagnosing pathologies based on artificial intelligence technology, having a microservice architecture and containing at least a module for viewing an image with pathology, configured to edit the markup of the received image with pathology, connected to at least one processor and, by at least one storage device, and said storage device contains machine-readable instructions that, when executed by at least one processor, implement the steps of the method according to paragraphs. 1-9.