EA045328B1 - DEVICE AND METHOD FOR DETERMINING PATHOLOGY OF CHEST ORGANS BASED ON X-RAY IMAGES - Google Patents

Description

Field of technology to which the invention relates

The present invention relates to the field of image processing, and more particularly to a device and method for determining pathology of the chest organs based on X-ray images.

State of the art

In traditional methods of diagnosing pathologies of the chest organs, which have been used in medicine for decades, the operator of an X-ray machine or doctor visually examined the X-ray image obtained by the machine and made a conclusion about the presence or absence of a particular pathology in the patient being examined.

However, the high prevalence of cancer, heart and lung diseases, as well as the covid-19 outbreak, which has led to a significant increase in mortality from these diseases, entail enormous social costs. In medical institutions carrying out diagnostic and therapeutic activities, there is overload and a shortage of qualified medical specialists in the field of radiological studies; the assessment and interpretation of the results of radiological studies is influenced by the human factor; there are no necessary resources for conducting repeated readings of studies. The practice of carrying out diagnostic measures is associated with the risk of missing pathologies during the initial analysis of radiological studies.

These factors require fundamental research in this area in order to develop new and more effective therapeutic and diagnostic tools, including in order to improve the quality of medical decisions and, if possible, speed up the medical decision-making process.

All this contributes to the introduction of modern digital data processing techniques into medicine. The digitalization of healthcare is accompanied by rapidly growing volumes of digitized patient health information, clinical databases, and medical data sets that could be used to support clinical decision making. In recent years, artificial intelligence (AI) technologies have been increasingly used to work on such problems.

For example, in known methods of modern AI diagnostics, such as WO 2021/031279 A1 or WO 2021/091661 A1, a neural network, which was previously trained on a labeled set of images, receives a digitized X-ray image of the patient’s chest, analyzes it and makes a prediction whether a given patient has a pathology, thereby helping the doctor make the best informed decision.

Attempts are being made to use a variety of neural network architectures, different training methods and different methods of image pre-processing, but the accuracy of the known methods is such that a significant part of the medical community expresses the opinion that at the current level of development such systems do not help the doctor, but, on the contrary, interfere, since they may reduce the concentration of a specialist relying partly on AI, and errors in decision-making are possible, which can affect the quality of medical services. And due to the fact that the responsibility for the decision lies with the person, the human factor is still large and labor costs and qualification requirements are high.

The essence of the invention

In order to overcome at least some of the above-mentioned disadvantages of the prior art, the present invention is directed to improving the accuracy of devices and methods for determining pathology of the chest organs based on X-ray images.

According to a first aspect of the present invention, there is provided a device for determining the pathology of the chest organs based on the analysis of X-ray images, comprising a data receiving unit configured to receive data to be processed, containing an X-ray image of the chest organs in front projection, from a device requesting data processing, by communicating with him;

an image preparation unit configured to prepare, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the at least one processor, and in the process The image preparation unit is configured to automatically crop the image so that it covers only the lungs, by searching the image for the right lung and the left lung separately using at least two different mutually independent pre-trained image recognition methods other than neural networks, forming a plurality of rectangles, each of which covers one found lung and only it, based on the searches performed, forming a general rectangle that includes essentially only said plurality of rectangles, and cutting off areas from the image that extend beyond the general rectangle;

a pathology prediction block configured to analyze the prepared image using said neural network and determine the presence or absence of pathology in the image;

a report generation unit configured to generate, using at least one processor, a report on a study conducted in this device, wherein the study report contains at least one file containing an indication of the result of the processing performed in the pathology prediction block; and a data transmission unit configured to transmit a report to a device that has requested data processing by communicating with it.

In one embodiment, if it is determined that there is a pathology in the image, the report generating unit is further configured to identify possible areas in which signs of pathologies are found;

generating an image with visualization of pathology in the form of indicating the found areas with signs of pathology; and generating a study report containing the pathology imaging image.

In one embodiment, the identification of possible areas with signs of pathologies is performed using computer vision methods that analyze the input image of the neural network and the activation of the pathology prediction block within the neural network, which were obtained by running this image through it.

In one embodiment, a combination of Grad-CAM (weighted combination of gradient-based class activation maps) and Saliency maps methods is used as computer vision methods.

In one embodiment, the indication of the found areas with signs of pathologies is made in the form of a heat map or outline of the boundaries of the areas.

In one of the embodiments, for generating an image with pathology visualization, the report generation block is configured to superimpose an indication of the found areas with signs of pathologies on a copy of the input image of the neural network of the pathology prediction block.

In one embodiment, the indication of the found areas with signs of pathologies is linked to the size, shape and position of the image, which was directly analyzed by the neural network;

the image preparation unit is additionally configured to store the parameters of all transformations performed on the original image obtained in the data to be processed, and associated with changing its size, shifting, rotating, cropping and cropping; and to generate an image with pathology visualization, the report generation unit is configured to apply transformations inverse to the above stored transformations to indicate the found areas with signs of pathologies, so as to result in a pathology visualization with reference to the size, shape and position of the original X-ray image; and superimpose the resulting visualization of the pathology with reference to the size, shape and position of the original X-ray image on a copy of the original X-ray image.

In one embodiment, three pre-trained methods are used as pattern recognition methods: the Viola-Jones method using a cascade Haar classifier, the directional gradient histogram method, and the local binary pattern method.

According to the second aspect of the present invention, there is provided a method for determining the pathology of the chest organs based on the analysis of X-ray images, comprising the steps of receiving, using a communication device, data to be processed containing an X-ray image of the chest organs in front projection;

preparing, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, and when preparing the image, automatically cropping the image in this way , so that it covers only the lungs, by searching the image of the right lung and left lung separately by at least two different pre-trained image recognition methods independent of each other, other than neural networks, forming a plurality of rectangles, each of which covers one found lung and only it, based on the searches performed, forming a common rectangle, which includes essentially only the mentioned plurality of rectangles, and cutting off from the image areas that extend beyond the general rectangle;

using said neural network, the prepared image is analyzed and the presence or absence of pathology in the image is determined;

- 2 045328 generates, using at least one processor, a report that contains at least one file containing an indication of the presence or absence of a pathology; and transmitting a report using the communication device to the device that requested data processing.

Technical result

The present invention improves the efficiency of devices and methods for determining the pathology of the chest organs based on X-ray images.

This ensures increased accuracy in determining the pathology of the chest organs;

reducing requirements for the qualifications of medical personnel;

reducing the influence of the human factor (attention, fatigue, responsibility).

It should be understood that not each of the embodiments may simultaneously provide all of these advantages over all known solutions from the prior art. Accordingly, some embodiments may have only some of these advantages or other advantages relative to some known solutions.

These and other advantages of the present invention will become apparent upon reading the following detailed description taken in conjunction with the accompanying drawings.

Brief description of drawings

In fig. 1 is a schematic diagram of a medical decision support system according to the present invention.

In fig. 2 shows an example of an image to be processed in the pathology detection device.

In fig. 3 is a block diagram of a pathology detection apparatus according to the present invention.

In fig. Figures 4 and 5 show examples of images that should not be validated.

In fig. 6A, 6B shows an example of cropping an image.

In fig. Figure 7 shows an example of an image on which areas of interest are plotted as a heat map.

In fig. Figure 8 shows an example of an image on which the contours of areas of interest are plotted.

In fig. 9 is a flowchart of the pathology detection method according to the present invention.

It should be understood that the figures may be presented schematically and not to scale and are intended primarily to enhance understanding of the present invention.

Detailed description

General overview of the system.

Next, with reference to FIG. 1, a chest health decision support system 100 will be generally described.

The diagnostic medical organization (MO) 110 contains a device 120 for obtaining x-ray images (may also be interchangeably called x-ray diagnostic apparatus, x-ray machine, x-ray unit, x-ray, etc.). A medical specialist, using an X-ray machine 120, performs an X-ray examination of the chest organs of the patient being examined (including fluorography).

The medical organization 110 according to the present invention can be a clinic, a clinic, a doctor's office, a hospital, a hospital, a hospital, a sanatorium, a medical aid station, a pharmacy, a mobile unit, a mobile fluorography room, or any other organization, room or installation equipped with a device 120 for obtaining x-ray images.

The result of the study is generated in the form of an x-ray image, for example, in DICOM (Digital Imaging and Communications in Medicine), NIfTI (Neural Imaging Technology Initiative for Informatics), Analyze, Minc1, Minc2 or other suitable open or proprietary format. If necessary, before saving, a medical specialist can apply filters to the image that, in his opinion, can improve the perception of the image. In addition, in one embodiment, any preset filters specific to a particular device or manufacturer may be superimposed on the image.

The generated image is stored in a local data store 131, such as a store based on or as part of a PACS (DICOM Image Archiving and Communication System), RIS (Radiology Information System), MIS (Medical Information System) 130 or other system or device suitable for storage of medical data. If necessary, additional targeted information can be added to the saved image, such as information about the patient, about the study, about the medical organization, about the medical specialist, etc.

MIS 130 (including PACS or RIS) can be deployed on a general purpose computer or server, on specialized equipment or on other hardware for storing, processing and transmitting data suitable for implementing the present invention and known to a person skilled in the art. In an illustrative non-limiting example, MIS 130 may be a computer having a 4-core processor, 8 GB of RAM, 60 GB of storage space

- 3 045328 CentOS 7 operating system, 2 TB disk space for data, 1 TB disk space for a database and 2 network adapters.

Storage 131 is shown in FIG. 1 as part of the MIS 130 is only an example, and it should be understood that other implementations are possible in which the storage 131 is separate from the MIS 130, is part of the X-ray machine 120, or is part of another internal storage system of a medical organization. Implementations are also possible in which a medical organization does not use special systems for storing and transmitting medical data at all, but uses other available storage and transmission means for these purposes, such as a personal computer or laptop.

The data to be processed is then generated using a data communication device 132, such as a PACS, RIS, MIS 130, or other suitable open or proprietary system. As with storage 131, although communication device 132 is shown in FIG. 1 as part of MIS 130, it can also be separate from MIS 130 or be part of another internal storage system of a medical organization.

In one embodiment, when generating data to be processed in the data transmission device 132, depersonalization (anonymization, de-identification) of the chest X-ray image of the patient being examined can be performed. To do this, information about the patient, which one way or another can be considered personal, is replaced with anonymized data, from which a third party will not be able to restore the original data without proper access to it. For example, when using a PACS system, the fields PatientName (patient name), OtherPatientNames (other patient names), PatientID (patient identifier) and, if necessary, other fields containing or classified as personal data are anonymized.

The possible depersonalized data values may be predefined and known to all devices in the system, only trusted devices, or only the pathology device 160 so that they can determine whether the transmitted data is depersonalized.

The depersonalization process can occur either completely automatically or, if necessary, part of the depersonalization process or the entire depersonalization process can be performed with the participation of a person who can delete or edit data through the appropriate interface of the data communication device 132 or the medical information system 130.

Depersonalized data to be processed, an example of which is shown in FIG. 2 are transmitted from the medical organization 110 directly or through the central medical information system 150 to the pathology determination device 160. However, in one embodiment, depersonalization can be performed in the central medical information system 150, and in this case, non-anonymized data can first be transferred from the medical organization 110 to the central medical information system 150, and then depersonalized data can be transferred from it to the pathology device 160 data to be processed. The depersonalization process can occur either completely automatically or, if necessary, part of the depersonalization process or the entire depersonalization process can be performed with the participation of a person who can delete or edit data through the appropriate interface of the central medical information system 150.

The pathology detection device 160, having analyzed the X-ray image contained in the incoming data based on artificial intelligence (AI) methods, generates a prediction whether there is a pathology in this image. If present, pathology detection device 160 indicates areas in the image containing pathologies. If there is no pathology, then the image does not change.

Optionally, the pathology detection device 160 can generate a result report (or protocol) containing a description of the result in the form of text information.

The generated data (one or more images and/or a report) is sent back (directly or indirectly) from the pathology device 160 to the healthcare organization 110 that requested the data processing.

The results obtained from the device 160 for determining pathology are provided to the responsible person, for example, a radiologist, attending physician or other medical specialist, or a person who has access to such information and is responsible for receiving and processing it in the medical organization 110, using the viewing tool 140 , and with their use he makes a decision about the state of the chest organs, namely the presence or absence of a particular pathology. If necessary, the medical specialist, taking into account the obtained results of the device 160 for determining the pathology, can make a decision about the treatment of the patient. The viewer 140 in FIG. 1 is simplistically indicated as a display 140, but in the preferred embodiment it represents a doctor's workstation (automated workstation). In an illustrative non-limiting example, a doctor's workstation may be a computer based on an Intel Core i3 processor or its equivalent, having 8 GB of RAM, 40 GB of free disk space, a DVD-R/RW CD reader, a network connection speed of 5 Mbit/s, monitor with a screen resolution of 1920x1080, and viewing can

- 4 045328 performed using DICOM image viewing programs or via a web browser.

Thus, the system 100 helps to improve the accuracy of medical decision making.

An embodiment is also possible in which the data generated by the pathology determination device 160 is sent not directly to the medical organization 110 that requested the data processing, but first (directly or through the central medical information system 150) to a specialized expert organization 170 that produces medical opinions using the results operation of the pathology determination device 160, or to an external radiologist 180 serving as an expert or consultant. In this case, the medical organization 110, in response to the sent x-ray image, can receive from the expert organization 170 or from the external radiologist 180 (again, directly or through the central medical information system 150) an immediately ready conclusion or a preliminary conclusion that can be used for making a medical decision. This makes it possible to reduce the requirements for the qualifications and workload of the personnel of a medical organization 110 up to the possibility of not having a radiologist on staff at all, which provides a significant expansion of the geography of possible use of the system and allows you to receive highly qualified medical services in those locations where they were not previously available due to the lack of personnel with the appropriate qualifications, and on the other hand, allows radiologists to connect to the system from various remote locations without being tied to a specific medical organization.

It should be understood that in this document, the blocks 170 and 180 are simplistically referred to as the expert organization 170 and the external radiologist 180, however, technically from a system point of view, these blocks are equipment/devices for receiving, viewing, editing and transmitting data that are controlled by/ owned/used by expert organization 170 and external radiologist 180.

According to the present invention, the term external in relation to a radiologist means that this doctor is not on the staff of the medical organization 110 that conducted the X-ray examination and requested processing of the resulting image, and/or is not physically located in this organization and/or does not have access to the MIS 130 of this organization. In addition, it should be understood that the term radiologist in the context of the present invention implies that this is a medical specialist who has proven qualifications (knowledge, abilities, skills and experience) in the analysis (interpretation) of the results of x-ray examinations.

The expert organization 170 may be a medical or other organization charged with the functions of analyzing medical images and drawing up conclusions and having one or more specialists of the appropriate profile and proper qualifications to analyze the results of an X-ray examination.

Specialists of the expert organization 170 and external radiologists 180 can access data from a specialized workstation (doctor’s workstation) or using another suitable device, such as a computer, laptop, smartphone, tablet, VR helmet (virtual reality helmet), VR -glasses, etc.

The central medical information system 150 shown in FIG. 1, the expert organization 170 or the external radiologist 180 are not required elements of the proposed medical decision support system 100. Accordingly, in one embodiment, the healthcare organization 110 may communicate directly with the pathology detection device 160. This simplifies the implementation of the medical decision support system 100. Such an implementation may be convenient, for example, for cases where the number of medical organizations 110 served by the pathology determination device 160 is relatively small, and the expert organizations 170 or external radiologists 180 are not involved at all or their number is also relatively small.

In another embodiment, when the medical decision support system 100 includes multiple medical organizations 110 and/or multiple pathology devices 160, as well as expert organizations 170 or external radiologists 180, it is advisable to use a central medical information system 150. It should be noted , that the term central in this case indicates, first of all, not that this is a single central server that closes all possible connections, but that the central medical information system 150 occupies a place in the middle, in the center between the others participants in the medical decision support system 100, acting as an intermediate system for collecting, storing and redistributing data. Moreover, depending on the requirements of a specific application, the central medical information system 150 can be either concentrated (centralized) or distributed, including implemented in the cloud.

Images awaiting processing may be grouped into batches (series) for transmission to pathology determination device 160. Accordingly, the pathology determination device 160 may batch process the received images. Grouping into batches can be performed both in the medical organization 110 and in the central medical information system 150.

- 5 045328

The central medical information system 150 can change the size and content of packages received from medical organizations 110 - for example, sort images by their resolution in pixels, by priority or other parameters and create new packages, and if necessary, they can be added to the same package pictures from different medical organizations 110.

In one embodiment, the central medical information system 150 may perform initial validation or preprocessing of images. Also, initial validation or preprocessing is possible on the side of the medical organization 110. Specific validation or preprocessing operations are described later in this document in relation to the pathology detection device and therefore are not disclosed in detail here.

The structure of the device for determining the pathology of the chest organs.

Next, with reference to FIG. 3, a device 200 for detecting chest pathology based on X-ray images according to the present invention will be described in detail. It should be noted that the chest pathology detection device 200 is fully consistent with the pathology detection device 160 shown in FIG. 1, and has the same functions and capabilities, if applicable and consistent with the description of this section.

The pathology detection device 200 includes a data receiving unit 210, a data storage unit 220, a data validation unit 230, an image preparation unit 240, a pathology prediction unit 250, a report generation unit 260, a data transmission unit 270, and a training unit 280. Depending on the specific application, some of these blocks may be missing, as will be explained in more detail later in this document.

The data receiving unit 210 receives data to be processed containing an X-ray image of the chest of the patient being examined. The receiving unit 210 may be a separate chip, network card, or other suitable means capable of communicating with external devices in a wired and/or wireless manner, such as through a local area network (LAN), Internet, etc. protocol. using Ethernet, optical fiber, WiFi, 4G, etc. technologies.

The data storage unit 220 stores the data received by the data receiving unit 210 so that other units of the device 200 can use it at an appropriate time. Received data can only be stored for the duration of its processing and erased when its processing is completed. For these purposes, in one embodiment, a short-term storage device such as RAM or the like is used. In another embodiment, the received data may optionally be stored for a longer period of time than the immediate processing time, and then a non-volatile storage device such as a hard disk or the like may be used.

Additionally, the data storage unit 220 may store short-term or long-term data and/or files resulting from or during the operation of other units of the device.

The data validation unit 230 receives the data to be processed directly from the data receiving unit 210 or retrieves it from the data storage unit 220. Data validation unit 230 then checks whether the received data is suitable for processing.

As mentioned above in the description section of the medical decision support system 100, some of the validation operations can be performed on the side of the medical organization itself or on the side of the central medical information system 150. In this case, if the pathology detection device 200 knows which validation operations have already been performed for the data to be processed, the data validation unit 230 may not perform these operations, thereby simplifying and speeding up the processing and thereby increasing productivity. However, in another embodiment, pathology detection device 200 may not be aware of which validation steps have already been performed, or may re-run them for additional double-checking purposes. This can provide improved processing quality.

Moving specifically to the validation operations, an attempt is made to extract an image from the data to be processed. If the attempt fails, it is concluded that the image file is damaged or cannot be read. The reason for this may be factors such as the absence of an image in the data, the inability to read metadata, the presence of any anomalous and unaccounted for tag values, etc. In this case, the data is not transferred for processing, and an appropriate instruction is created based on it. Performing this operation in the data validation block 230 is preferable because even if it has already been performed previously by other devices, the data may be corrupted in the process of being sent to the pathology device 200 or may be in a format that is not currently available for one reason or another. device 200.

A check may also be made to see if the data to be processed is consistent with the type of processing being performed in the pathology determination device 200. For example, if the pathology detection device 200 is configured to process AP chest X-rays, a check may be made to determine whether the enclosed data contains an AP chest X-ray. Various options for implementing such a check are known to specialists in the art and may include, for example, a pre-trained neural network (neural network) that produces the appropriate classification, or other computer vision methods. If the check fails, then such data is not transferred for processing, and an appropriate instruction is created for it. Examples of such errors, when the study is declared as a direct chest x-ray, but in fact is not one and cannot be processed, are shown in Fig. 4 and 5.

In addition, a check can be made to determine whether the data is depersonalized. To do this, it is checked whether the fields that relate to personal data contain any values, and if they do, then whether these values are depersonalized. For example, if the PatientName field is empty or contains a predefined value of 0 or Anonymous, as indicated in FIG. 2 and fig. 5, then this field is considered to be depersonalized, and the field with the value Venus de Milo indicated in FIG. 4, is not depersonalized. If the data is not depersonalized, then the corresponding image is not sent for processing and an indication of the impossibility of processing is generated based on it.

It should be noted that the pathology device 200 may support several different valid depersonalized data values, in which case the data validator 230 may check whether the field values in the received data correspond to at least one of the corresponding supported values. If the value does not match the predefined acceptable value, then the assumption is made that the received data is not depersonalized. In this case, they are not submitted for processing, and a corresponding instruction is created for them.

It may also be checked whether the read image has a size (pixel resolution) equal to or greater than a predetermined minimum size supported by the device 200—for example, 1024x1024 pixels or 800x800 pixels, depending on the requirements of the particular application. If the original image is smaller than the minimum size, the device 200 may not provide sufficient accuracy, so the image is not sent for processing and an appropriate indication is generated from it.

In addition, a check may be performed whether the image is positive or negative. Various options for implementing such a check are known to specialists in the art and may include, for example, a pre-trained neural network (neural network) that produces the appropriate classification, or other computer vision methods. If the test determines that the image does not meet the input requirements of the pathology detection device 200, then the image preparation unit 240 can subsequently perform the appropriate conversion of the image to negative or positive.

Validation allows you to filter out data for which analysis cannot be carried out or the accuracy of processing will have a deliberately low accuracy. Accordingly, the load on the most resource-intensive part of the analysis is reduced and the accuracy of prediction increases. Additionally, filtering out images that are not depersonalized ensures that device 200 does not process personal data, reducing implementation and certification requirements.

The image preparation unit 240 receives from the data storage unit 220 and/or from the data validation unit 230 a validated chest x-ray image and performs preliminary transformations on it in order to prepare it for direct use in the pathology prediction unit 250.

In particular, the image preparation performed in block 240 may be as follows.

The X-ray image to be processed is read and, if necessary, converted to grayscale with a preset color depth. The brightness and color parameters of the original image depend on the parameters set by the radiologist when working with the X-ray machine and are contained in the image metadata. For example, the color depth of the original image may be 12-16 bits. In this case, for example, a matrix of integer pixel values from 0 to 255 can be obtained as a result of reading and converting to an image with a color depth of 8 bits. Also, as indicated above, the appropriate conversion of the image to negative or positive can be performed.

The original size of X-ray images is generally relatively large, averaging 2500x2500 pixels, that is, 2500 pixels in height (vertical) and 2500 pixels in length (horizontally). Images of this size can generally be processed using machine learning methods, but such models consume a lot of resources.

In addition, the range of sizes of the input X-ray images can be quite large, which can cause inconvenience if trying to process each individual image directly at the original size. In particular, it becomes difficult to select a set of processing methods that are equally effective for different sizes.

In this regard, before processing large radiographs, it is preferable to reduce their size. This allows, without significant loss of quality, to significantly reduce the resources spent: time, computing power, energy consumption. Various methods of reducing size are applicable - for example, compression, cropping, etc. Compression methods are known to specialists in

- 7 045328 of the art, therefore a detailed description of the compression process is not given here. As for cropping the image, it will be described below.

For example, in one embodiment, if a region of interest (VOI) is preset in the image parameters by the medical examiner, a corresponding image cropping (VOI LUT) may be performed to produce an image containing only the region of interest. For example, if a medical professional performing a chest x-ray specified an area of interest, then according to the parameters embedded in the image, cropping is applied, resulting in only the lung area in the image.

In another embodiment, in addition to or instead of the above-mentioned VOI LUT cropping, a search is made for the lung region in the image and crops the image by cutting off portions of the image not included in the found lung region.

To do this, first the size of the input image is reduced to a single first size. The first image size is predefined - for example, 800x800 pixels. The value chosen depends on the requirements of the specific application and the performance of the equipment used.

In particular, at the design stage, an estimate may be made of what is the smallest size of x-ray image that can be supplied to the device 200 from the medical organization 110 and/or from the central medical information system 150, and accordingly set the above-mentioned first size that is less than or equal to the smallest possible size of the input X-ray image.

However, in practice there may also be situations where the smallest possible size of the input X-ray image is too small or the less-less principle is, for one reason or another, not suitable for a particular application. Then the first size to which all input X-ray images are reduced can be set according to a different principle, and in such a scenario some of the input images will be compressed, some will be stretched, and some will remain unchanged.

The image is then cropped to include only the lungs. In particular, the image is analyzed separately by different pattern recognition methods - for example, the Viola-Jones method using a cascade classifier based on Haar features (Haar Cascades), the Histogram of Oriented Gradients (HOG) method and the method based on local binary patterns (Local Binary Patterns, LBP), resulting in the formation of several rectangles, each of which indicates an area that covers only the right and left lung. The aspect ratio of the rectangles depends on the size and shape of the lungs. The analysis may be performed in parallel, sequentially, or in any other suitable manner. The pattern recognition algorithms used must be pre-trained to recognize lungs in an image having the first dimension.

The Viola-Jones method, the directional gradient histogram method, and the local binary pattern method mentioned above are slightly faster than traditional convolutional neural networks and require a slightly smaller training set.

The Viola-Jones method has very high accuracy when the recognized object in the image has a rotation angle of no more than 30-35°. Accordingly, it is well applicable to the analysis of X-ray images, since the patient's position is typified to obtain the image, for example, in anterior projection or lateral projection.

The directional gradient histogram method is weakly sensitive to displacement, changes in image scale and brightness, and somewhat sensitive to changes in object orientation. However, for the reasons stated above, it is also well suited for X-ray analysis.

Methods based on local binary patterns generally have high recognition speed and accuracy, and are also weakly sensitive to image brightness and changes in object orientation.

Then one common rectangle is constructed (see example in Fig. 6A), which includes all previously obtained rectangles (1, 2 and 3 for the right and left lung). The input (downscaled) image is cropped to this overall rectangle, the remaining image regions are discarded, so the cropped image contains only the lung region (see example in Fig. 6B). In one embodiment, each side of the common rectangle contains at least one of the sides of the rectangles obtained as a result of image analysis, with none of the sides of these rectangles extending beyond the boundaries of the common rectangle.

The results of different image recognition methods in the general case are not identical (in the example in Fig. 6A, rectangles 1, 2 and 3 are different), therefore such a combination of results allows you to cover the lungs completely without erroneously cutting off their parts, but at the same time, and focus specifically on areas of the lungs, discarding areas not of interest for this study.

Additionally, there is a possibility that one or two of the pattern recognition methods used will fail to find the lung in the image. At the same time, the likelihood that all three methods will not find a lung is extremely small. Therefore, the use of several pattern recognition methods at once is justified, since it provides protection against data loss and increased accuracy

- 8 045328 predictions.

Other embodiments are also possible, when around the rectangles obtained as a result of the analysis, as shown in FIG. 6A, a small border zone (margin) of one or more pixels is additionally captured if the specific analysis methods chosen are not sufficiently accurate and may result in lost regions of interest. The sizes of indents up, down, right and left may vary. Optimal indentation sizes can be selected depending on the results demonstrated by the trained algorithms and the accuracy requirements. In a specific non-limiting example, if the image size being processed is 512x512 pixels and all three of the above pattern recognition techniques are applied, the padding may be 15 pixels (or less if the padding extends beyond the border of the image). This avoids data loss and improves prediction accuracy.

The aspect ratio of the lung image resulting from cropping (i.e., the overall rectangle) (Fig. 6B) depends on the size and shape of the lungs. For further processing it is necessary to bring it to a unified format. To do this, the image size is changed to the second size. The second image size is predefined - for example, a square of 224x224, 320x320 or 512x512 pixels. The selected value depends on the requirements of the specific neural network used further. As for the first image size indicated above (800x800 pixels), it allows you to significantly reduce the size of the original image to speed up processing, but at the same time, as a result of cropping, you still get a relatively large image that would not have to be stretched to bring it to the second size in most cases , which could potentially reduce processing accuracy.

The image reduced to the second size is subjected to normalization. In particular, pixel values from the original range (for example, [0...255]) are reduced to the range required by the neural network used further (for example, [0...1]). Various normalization methods are known to those skilled in the art and are not disclosed in detail here. In an illustrative non-limiting example, a simple normalization may be applied (X-Xmin)/(Xmax-Xmin), where X is the current pixel value and Xmin and Xmax are the minimum and maximum pixel values in the image being normalized.

It should be understood that there may be embodiments in which image normalization can be performed at an earlier stage - for example, even before the first compression of the image. However, it is preferable to perform it after reducing the cropped image to the second size, since this allows you to increase the accuracy at each of the previous stages of image preparation (pre-processing), as well as somewhat reduce the amount of calculations performed directly during normalization.

This creates an image that is ready for further processing.

The pathology prediction unit 250 receives from the data storage unit 220 and/or from the image preparation unit 240 a prepared chest X-ray image limited to the lung region and analyzes it. In particular, the pathology prediction unit 250, using a pre-trained neural network, determines the presence or absence of pathology and, if present, determines the most likely boundaries within which it is located.

There are a number of pathological conditions of the chest organs that can be identified by an x-ray image by a doctor. Creating a device that, based on AI, would analyze images and give an accurate indication for each of the many possible pathologies is difficult and seems impractical, since many pathologies have similar signs, and just a depersonalized image is often not enough for a more detailed prediction, even if neural networks are trained on large, well-labeled data sets.

Accordingly, this invention proposes not to analyze the differentiated probability of a specific pathology, but instead to determine the probability of the presence of at least one radiological sign from the following list:

pleural effusion;

pneumothorax;

atelectasis;

focus of darkening;

infiltration/consolidation;

dissemination;

cavity with decay;

cavity with liquid level;

calcification/calcified shadow in the lungs;

violation of the integrity of the cortical layer.

It should be noted that the term pneumonia unites a large group of diseases, each of which has its own etiology, pathogenesis, clinical picture, radiological signs, characteristic laboratory data and treatment features. Considering that many of the above radiological signs can be observed in pneumonia, the present invention makes it possible to detect, among other things, signs of pneumonia.

A doctor who has received the result of the above determination, has certain experience, and also, if necessary, has additional information about the patient, such as anamnesis, medical history, complaints, physical condition, test results, etc., can have a more complete picture and make a more accurate conclusion. Accordingly, it should be noted that the result of the proposed device is not a clinically significant medical opinion, but is used to support medical decision-making, and the clinical opinion should be made by a radiologist.

The image processing performed in block 250 may include the following.

The neural network interprets the resulting image in order to identify signs of pathologies, and the result of the work is the probability of the presence of at least one radiological sign from the above list.

The output of the neural network can output, for example, a binary number indicating whether there is pathology in a given image. 0 may indicate the absence of pathology (that is, that the patient is healthy), and 1 may indicate the presence of pathology. The opposite indication is also possible, where 1 is healthy, 0 is probably sick. In another embodiment, the output of the neural network may output a number in a certain range, for example, from 0 to 1 or from 0 to 100, indicating the probability of the presence of pathology. A text format for the indication is also possible.

Next, the preliminary training of the model will be described in more detail. In particular, it is produced using a supervised learning method based on a set of X-ray images that have been pre-processed and labeled by medical specialists with sufficient qualifications - for example, radiologists. In particular, as a result of visual examination of the images, the doctor indicates whether there is a pathology in the presented image. For example, 0 may indicate the absence of pathology (that is, that the patient is healthy), and 1 may indicate the presence of pathology, or vice versa. This is either done manually by the physician, or read automatically from patient data stored in the medical information system 130 of the medical organization 110 or in the central medical information system 150, or from image metadata if it contains this information, etc. Manually adding marks can be done by analyzing the text of the image description by a doctor, on the basis of which a conclusion was drawn up about the presence/absence of pathology in the image. It is also possible to automatically recognize existing description texts for/on images, however, it should be understood that the recognition accuracy in the general case is not 100%, which reduces the degree of confidence in such markup and introduces potential errors into the learning process.

The neural network training process is controlled by training block 280 by applying a training algorithm to the neural network being trained using the training data. The instructions (markings) from the doctor are used by the trained neural network as ground truth. For example, in one embodiment, the markup may be the number: 1 (sick). In another embodiment, the markup may represent another meaning - for example, healthy.

In an illustrative embodiment, 70% of the images taken for training can be used directly for training, and 30% for testing the model. In another embodiment, a different ratio may be used that is more suitable for the purposes of training a particular neural network. For example, to train a neural network, images can be divided into training, validation and test samples. The training set size can be 70%, the validation set size can be 15%, and the test set size can be 15% of the images submitted for training. Specific ratios depend on the number of training images and are intended to test the generalized model - the extent to which the research results are applicable to new data is checked before using the model in operational mode in a medical organization.

During the training process, images are prepared in the same way as described above in relation to block 240, and are fed to the input of the neural network, while images with both the presence and absence of pathologies are used. At each training step, the neural network makes predictions for one or more images. These predictions are compared with the indication of the true presence/absence of pathology, and the value of the loss function is calculated (how much the neural network was mistaken in identifying the presence of pathology). Next, using the gradient descent method and the backpropagation algorithm, all weights (weighting coefficients) of the neural network are changed in accordance with the selected learning rate parameter in the direction opposite to the calculated gradient in order to minimize the error in the current image(s). I). This step is repeated many times, and as a result of the training process, the weights of the neural network converge to the optimal ones. These weights are subsequently used by the network during device operation to identify pathologies in input images that have not been marked.

- 10 045328

In one embodiment, the output data of the pathology prediction block 250, that is, an indication of the probability of the presence of a pathology, is the final result of the operation of the device 200 and is further used by other devices - for example, it is stored in the data storage block 220, from where it can later be retrieved on request from other devices , and/or are sent to the data transmission unit 270, which transmits this result to another device - for example, to the central medical information system 150 or to the medical organization 110 that requested data processing.

In another embodiment, the output of the pathology prediction block 250 is also transmitted to the report generation block 260, which generates a report of the examination performed on the device 200. The examination report may contain at least one of the following: a) a pathology imaging image; b) text study protocol.

A pathology image may be generated for those x-ray images that have been analyzed to have a pathology probability greater than a predetermined pathology threshold, indicating that the radiologist should pay attention to the image. The threshold probability of the presence of pathology in the general case is 0.5 (or 50%), but depending on the accuracy indicators of a particular trained model and the degree of confidence in it, it can vary in practical application, either down or up. For example, a medical organization may specialize in the treatment of diseases, the early detection of which is critical in terms of prognosis of treatment, and therefore may request visualized images if the device has detected a probability of the presence of pathology of only 10% or more. In another example, a medical organization may be affiliated with an enterprise, and the bulk of its patients may have occupational chronic chest diseases, and some changes in the images of such patients may be considered acceptable, so imaging may be requested only in cases where the device has identified the possibility of the presence pathology 60% or more. It should be noted that many other situations are possible that are not listed herein but are within the scope of this invention. The ability to select the threshold probability of the presence of pathology allows you to adapt the results of the proposed device 200 to various specific applications, for example, to reduce the burden on medical personnel and at the same time ensure early detection of pathologies with the necessary accuracy.

An image with pathology visualization is a copy of the analyzed image, on which an indication of the found signs of pathologies is superimposed. For example, if the DICOM format is used, it can be generated as a DICOM Secondary Capture.

Various visualization options are possible. In one embodiment, the visualization can be performed in the form of a heat map, which provides a more intensive selection of those points and/or areas that are of greater interest from the point of view of the presence of pathologies, a less intensive selection of those points and/or areas that are of less interest , and those areas that presumably do not contain pathology are not highlighted. The principle of selection can be different - for example, selection can be done using a gradient of the same color (for example, shades of red or orange), using a gradient transition from one color to another color (for example, from yellow to red), using precise specifying the color (for example, blue - lower probability, blue - higher probability), using the size of the dot (for example, the lower the probability, the smaller the dot), etc. Various methods for constructing heat maps are applicable - for example, a combination of two methods shows improved performance: Grad-CAM (a weighted combination of gradient-based class activation maps) and Saliency maps (saliency maps). In particular, each of these methods separately analyzes the input image of the neural network and the activations (numbers) within the neural network that were obtained by running this image through it, after which the results of the methods are combined and a general heat map is constructed, an example of which is shown in fig. 7.

Heat maps allow you to adjust the amount of attention a healthcare professional should pay when examining a particular area in an image, eliminating the need for detailed visual inspection of the entire image. This ensures a reduction in the burden on medical personnel.

However, there may be situations where the heat map is not accurate enough, in which case the doctor, on the contrary, has to spend time studying an area that is not of interest. It also turned out that in some implementations, heat maps clutter the image, as a result of which the specialist has to spend time viewing two images simultaneously: both the original image and the image with the heat map applied to it.

Accordingly, in another embodiment of the present invention, the visualization may be in the form of delineations of boundaries (or contours) enclosing one or more areas in the image in which there is a likelihood of pathology, or in other words, in which the block

- 11 045328

250 pathology predictions, signs of pathology are detected. To implement this approach, various computer vision methods can be used to analyze the state of the neural network of the pathology prediction block 250. For example, as mentioned above, a combination of two methods shows improved performance: the Grad-CAM method (a weighted combination of gradient-based class activation maps) and Saliency maps (saliency maps). A predefined threshold heat value is applied to the heat map constructed on their basis, according to which the contour is drawn (for example, 0.25, 0.3, etc.). In this way, closed loops are formed that enclose regions with heat equal to or greater than the threshold, while regions with heat below the threshold are discarded. The heat maps themselves are not applied to the image; only the contours are visualized on it, as shown in the example in Fig. 8.

An image with the outlines of the boundaries of possible pathologies marked on it, as before, makes it possible to reduce the burden on medical personnel, while it is enough to study only it, without referring to the original image. In addition, this approach allows us to smooth out inaccuracies in defining areas of interest.

In one embodiment, an indication of the found signs of pathologies in the form of a heat map or boundaries of pathologies can be superimposed on a copy of the image that was directly analyzed by the neural network, that is, on an image that, compared to the original X-ray image obtained in the data to be processed, has been compressed, cropped and reduction to a single size. This simplifies the process of visualizing the pathology, but in this case the resulting image is different from the original one, so it may cause some discomfort for the radiologist comparing the two images.

In another embodiment, an indication of the detected pathology features in the form of a heat map or pathology boundaries may be superimposed on the original x-ray image obtained in the data to be processed. To implement this approach, the parameters of all transformations performed on the original image and associated with resizing, shifting, rotating, cropping and cropping are stored by the image preparation unit 240 - for example, in the storage unit 220. After developing methods for searching for signs of pathologies, a visualization of the pathology is obtained in the form of a heat map or the boundaries of pathologies with reference to the size, shape and position of the image, which was directly analyzed by the neural network. The inverse transformations of the above stored transformations are applied to the resulting visualization to produce a pathology visualization that is referenced to the size, shape, and position of the original x-ray image. This rendering is then superimposed on a copy of the original image. This somewhat complicates the processing, but allows you to reduce the time for image analysis by a radiologist and increase the accuracy of identifying pathologies. This is especially applicable to heat maps, where the comparison of the original image and the pathology image is in many ways an urgent need for the radiologist.

If necessary, the image with visualization of the pathology may contain in text form an indication of the probability of the presence of pathology, obtained as a result of the operation of the first neural network of the pathology prediction block 250. This reduces the need to study any other images or files, including the original image and/or file with a text study protocol.

Turning now to the textual study protocol, it can be noted that it can either complement the above-described pathology image or replace it. The text research protocol contains the results of the work and a description of the operation of the device 200 in the form of text information, for example, in CSV format. If the DICOM format is used, the protocol can be generated in the form of a DICOM Structured Report.

In particular, the textual research protocol may contain:

1) The name of the source image (a link to it in the form of a corresponding identifier - for example, StudyInstancelD).

2) Recommendation (or in other words, prediction). As stated above, this recommendation may have a binary value (eg, 0/1), may be a number in some range indicating the probability of having the pathology (eg, 0 to 1), or may be in text format (eg, probably sick/ probably healthy).

3) Time stamps - time of receiving the study, time of generating the report.

4) Information about device operation errors during the analysis, if any.

The study report obtained as a result of the operation of block 260, containing an image with pathology visualization and/or a text study protocol, is stored in data storage block 220, from where it can later be retrieved upon request from other devices, and/or sent to data transmission block 270, which transmits this report to another device - for example, to a central medical information system 150 or to a medical organization 110 that requested data processing.

In one embodiment, chest x-ray results obtained by the device 200 may be stored in the device for a period of time, if desired.

- 12 045328 preset time (for example, seven days) and then deleted. This may be done, for example, for backup purposes so that the analysis result can be queried and retransmitted without the need for a full re-analysis.

Thus, a chest pathology detection device is provided that is capable of automatically determining the probability of pathology with increased accuracy. This allows us to reduce the requirements for the qualifications of medical personnel and reduce the influence of the human factor (attention, fatigue, responsibility). It also provides the ability to use a large number of disparate data sets for training. In addition, the result of the study provides a comprehensive set of information necessary for making the correct medical decision with increased speed and accuracy.

In an illustrative non-limiting example for the purposes of the present invention, the pathology detection device 200 may be implemented as a cloud server having 24 computing cores, 64 GB RAM, 300 GB disk space for the CentOS 7 operating system, 12 TB disk space for data, 1 disk space, 5 TB for database and 2 network adapters.

Method for determining pathology of the chest organs.

Next, with reference to FIG. 9, a method 400 for determining chest pathology will be described.

It should be understood that the steps of method 400 correspond to the above-described functions performed by each of the blocks in the device 200, and if any information is not disclosed in connection with the method, but is disclosed in connection with the device, and vice versa, then it is not implied that in the device or method this function or step cannot be performed, but is done only to avoid cluttering the description with repetition of details. It should also be understood that depending on the particular application, some of the described method steps may not be used.

The pathology determination method 400 may comprise step S410 of receiving data to be processed comprising a chest X-ray image of the subject being examined.

The pathology determination method 400 may also include a step S430 in which validation is performed by checking whether the data to be processed is suitable for processing.

The pathology determination method 400 may also comprise a step S440 of preparing an image contained in the data to be processed for use by the neural network by performing one or more preliminary transformations on it.

The pathology detection method 400 comprises step S450, in which the prepared image is analyzed using a pre-trained neural network and the presence or absence of organ pathology in the image is determined.

The pathology determination method 400 may also comprise a step S451 where, if a pathology is present, the most likely boundaries within which it is located are determined using a pre-trained neural network.

The pathology determination method 400 may also comprise step S460, which generates an examination report comprising at least one of the following: a) a pathology imaging image; b) text study protocol.

The pathology determination method 400 may also comprise a step S470 of transmitting the processing result obtained in steps S450, S451 and S460 to the device requesting the data processing.

The pathology detection method 400 may also include a step S480 (not shown) in which the neural network is pre-trained using pre-labeled training images, wherein the label for each image includes an indication of the presence or absence of pathology of the organs in the area of the image for which it is desired to perform neural network training.

The pathology determination method 400 may also include a step S490 (not shown), in which, using the processing result obtained in steps S450, S451 and S460, a medical decision is made about the presence or absence of pathology of the organs in the image area and, if necessary, the nature of the pathology.

The pathology determination method 400 may also comprise a step S491 (not shown) in which a patient is prescribed and/or treated based on the processing result obtained in steps S450, S451 and S460 and/or the medical decision made in step S490.

The pathology determination method 400 may also include a step S420 (not shown) in which data received by the data receiving unit 210 and/or obtained as a result of or during execution of other method steps is stored.

The proposed method for determining the pathology of the chest organs provides increased accuracy in automatically determining the probability of pathology, reduces the requirements for the qualifications of medical personnel and reduces the influence of the human factor (attentiveness, fatigue, responsibility). In addition, the result of the study provides a comprehensive set of information necessary for making the correct medical decision with increased speed and accuracy.

- 13 045328

Application.

The devices and methods of the present invention can be used to process X-ray images of the chest to identify signs of pathology in them.

Additional implementation features.

Although this document may indicate that data is being transmitted/sent or received/received by a human (e.g., medical professional, doctor, expert), one skilled in the art should understand that such indication is used solely for the purpose of simplifying the description , when in fact it is implied that the data is transmitted/sent or received/received by the corresponding device used and/or controlled by that person.

One or more transmitter blocks or devices described herein and one or more receiving blocks or devices described herein may be physically implemented in the same transceiver block or device or in different blocks or devices.

A transmission device or unit in this document, for the sake of simplicity of description, may be referred to as a device or unit having the functions of not only transmitting, but also receiving data, information and/or signals. Likewise, the receiving device or unit may also include functions for transmitting data, information and/or signals.

The various illustrative blocks and modules described in connection with the disclosure herein may be implemented or executed by a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable logic device (PLD), discrete logic gate or transistor logic, discrete hardware components, or any combination of the foregoing, designed to perform the functions described herein. The general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (eg, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors plus a DSP core, or any other similar configuration).

Some of the units, individually or collectively, may constitute, for example, a computer, and include a processor that is configured to call and execute computer programs from memory to perform method steps or functions of the units in accordance with embodiments of the present invention. According to embodiments, the device may further include memory. The processor may call and execute computer programs from memory to perform the method. Memory may be a separate device, independent of the processor, or may be integrated into the processor. The memory may store code, instructions, commands and/or data for execution on a set of one or more processors of the described device. Codes, instructions, or commands may direct a processor to perform steps of a method or device function.

The functions described herein may be implemented in hardware, software executed by one or more processors, firmware, or any combination thereof. The hardware and software that implement the functions may also be physically located in different locations, including according to a distribution such that portions of the functions are implemented in different physical locations, that is, distributed processing or distributed computing may be performed.

If necessary (for example, if the volume of data and/or calculations that need to be performed on this data is large), multi-threaded data processing can be performed, which in a simple representation can be expressed in the fact that the entire set of data to be processed is divided into a set of subsets , and each processor core performs processing on its assigned subset of data.

The above memory may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. One skilled in the art will also understand that when talking about memory and the storage of data, programs, codes, instructions, commands, etc., it is meant to be a machine-readable (or computer-readable, processor-readable) storage medium. Computer readable media includes both non-transitory computer storage media and communication media, which includes any transmission medium that facilitates movement of a computer program or portion thereof from one location to another. A non-transitory computer-readable storage medium may be any available medium that can be used to carry or store desired program code media in the form of instructions or data structures, and that can be accessed by a computer, processor, or other general purpose or special purpose processing device. appointments.

By way of example, and not limitation, computer-readable media may comprise read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM). , flash memory, random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), ESDRAM, Synchronous Link DRAM (SLDRAM) and Direct Access Random Access Memory (DR RAM), Register, Cache, Semiconductor Memory, Magnetic Media , such as internal hard drives and removable drives, magneto-optical media and optical media such as CD-ROMs and digital versatile disks (DVDs), as well as any other storage media known in the art.

The information and signals described herein may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols and chips that may be exemplified in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combinations of the above, as applicable to the present invention.

At least one of the steps in the method or blocks in the device may use an artificial intelligence (AI) model to perform corresponding operations. The AI-related function may be performed through a processor and non-volatile and/or volatile memory.

The processor may include one or more processors. At the same time, one or more processors may be a general purpose processor such as a central processing unit (CPU), an application processor (AP), etc., a graphics processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU) ), and/or a specialized AI processor such as a neural processing unit (NPU).

One or more processors control the processing of input data in accordance with a predefined operating rule or artificial intelligence (AI) model stored in non-volatile memory and/or volatile memory. A predefined operating rule or artificial intelligence model can be obtained through training. In this case, the processor can perform a pre-processing operation to convert the data into a form suitable for use as input to an artificial intelligence model.

Learned means that by applying a learning algorithm to a trained AI model using a set of training data, a predefined operating rule or AI model with the desired characteristic is created. The training may be performed on the device itself on which the AI is running according to the embodiment, and/or may be implemented through a separate server/system.

An artificial intelligence model may include many layers of neural network. Each of the multiple layers of the neural network includes a plurality of weights (coefficients) and performs an operation for that layer by computing using the plurality of weights of that layer with respect to the input data or computation result of the previous layer.

Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN) ), generative adversarial networks (GANs), and deep Q-networks.

A learning algorithm is a method of training a predefined target device (such as a GPU or NPU based neural network) using a set of training data to invoke, enable, or control the target device to perform a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

It should be understood that although terms such as first, second, third, and the like may be used herein to describe various blocks, modules, networks, elements, components, areas, layers and/or sections, these blocks, modules , networks, elements, components, areas, layers and/or sections are not intended to be limited to these terms. These terms are used only to distinguish one block, module, network, element, component, area, layer or section from another block, module, network, element, component, area, layer or section. Thus, a first block, module, network, element, component, region, layer or section may be referred to as a second block, module, network, element, component, region, layer or section without departing from the scope of the present invention. As used herein, the term and/or includes any and all combinations of one or more of the respective items listed. Elements referred to in the singular do not exclude the plurality of elements unless specifically stated otherwise.

The functionality of the element specified in the description or claims as a single element,

-

Claims (9)

may be implemented in practice by means of several components of the device, and conversely, the functionality of elements specified in the description or claims as several separate elements may be implemented in practice by means of a single component. In one embodiment, some or all elements/blocks/modules of the proposed device are located in a common housing, can be placed on one frame/structure/printed circuit board/chip and are connected to each other structurally through installation (assembly) operations and functionally through communication lines . The mentioned lines or communication channels, unless otherwise indicated, are typical communication lines known to specialists, the material implementation of which does not require creative efforts. The communication line can be a wire, a set of wires, a bus, a track, a wireless communication line (inductive, radio frequency, infrared, ultrasonic, etc.). Communication protocols over communication lines are known to those skilled in the art and are not disclosed separately. The functional connection of elements should be understood as a connection that ensures the correct interaction of these elements with each other and the implementation of one or another functionality of the elements. Particular examples of functional communication may be communication with the ability to exchange information, communication with the ability to transmit electric current, communication with the ability to transmit mechanical motion, communication with the ability to transmit light, sound, electromagnetic or mechanical vibrations, etc. The specific type of functional connection is determined by the nature of the interaction of the mentioned elements, and, unless otherwise indicated, is provided by widely known means, using principles widely known in the art. The design of the elements of the proposed device is known to those skilled in the art and is not described separately in this document unless otherwise indicated. The elements of the device can be made of any suitable material. These component parts can be manufactured using known methods including, by way of example only, machining, investment casting, and crystal growth. The assembly, connection and other operations in accordance with the above description are also within the knowledge of a person skilled in the art and, therefore, will not be explained in more detail here. Although exemplary embodiments have been described in detail and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative and are not intended to limit the present invention, and that the invention should not be limited to the specific arrangements and structures shown and described, since Various other modifications and embodiments of the invention may be apparent to one skilled in the art based on the information set forth in the specification and knowledge of the prior art without departing from the spirit and scope of the present invention. CLAIM 1. A device for determining the pathology of the chest organs based on the analysis of X-ray images, containing a data receiving unit configured to receive data to be processed containing an X-ray image of the chest organs in a direct projection from the device requesting data processing by communicating with it ; an image preparation unit configured to prepare, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the at least one processor, and in the process The image preparation unit is configured to automatically crop the image so that it covers only the lungs, by searching the image for the right lung and the left lung separately using at least two different mutually independent pre-trained image recognition methods other than neural networks, forming a plurality of rectangles, each of which covers one found lung and only it, based on the searches performed, forming a general rectangle that includes essentially only said plurality of rectangles, and cutting off areas from the image that extend beyond the general rectangle; a pathology prediction block configured to analyze the prepared image using said neural network and determine the presence or absence of pathology in the image; a report generation unit configured to generate, using at least one processor, a report on a study conducted in this device, wherein the study report contains at least one file containing an indication of the result of the processing performed in the pathology prediction block; And - 16 045328 data transmission unit, configured to transmit a report to the device that requested data processing by communicating with it. 2. The device according to claim 1, in which, if it is determined that there is a pathology in the image, the report generation unit is additionally configured to identify possible areas in which signs of pathologies are found; generating an image with visualization of pathology in the form of indicating the found areas with signs of pathology; and generating a study report containing the pathology imaging image. 3. The device according to claim 2, in which the identification of possible areas with signs of pathologies is performed using computer vision methods that analyze the input image of the neural network and activation of the pathology prediction block within the neural network, which were obtained by running this image through it. 4. The device according to claim 3, in which a combination of Grad-CAM (weighted combination of gradient-based class activation maps) and Saliency maps (saliency maps) methods are used as computer vision methods. 5. The device according to claim 2, in which the indication of the found areas with signs of pathologies is made in the form of a heat map or outline of the boundaries of the areas. 6. The device according to claim 2, in which, to generate an image with visualization of pathology, the report generation unit is configured to superimpose an indication of the found areas with signs of pathologies on a copy of the input image of the neural network of the pathology prediction unit. 7. The device according to claim 2, in which the indication of the found areas with signs of pathologies is linked to the size, shape and position of the image, which was directly analyzed by the neural network; the image preparation unit is additionally configured to store the parameters of all transformations performed on the original image obtained in the data to be processed, and associated with changing its size, shifting, rotating, cropping and cropping; and to generate an image with pathology visualization, the report generation unit is configured to apply transformations inverse to the above stored transformations to indicate the found areas with signs of pathologies, so as to result in a pathology visualization with reference to the size, shape and position of the original X-ray image; and superimpose the resulting visualization of the pathology with reference to the size, shape and position of the original X-ray image on a copy of the original X-ray image. 8. The device according to claim 1, in which three pre-trained methods are used as pattern recognition methods: the Viola-Jones method using a cascade classifier based on Haar features, the directed gradient histogram method and the method based on local binary patterns. 9. A method for determining the pathology of the chest organs based on the analysis of X-ray images, comprising the steps of receiving data to be processed using a communication device, containing an X-ray image of the chest organs in a direct projection; preparing, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, and when preparing the image, automatically cropping the image in this way , so that it covers only the lungs, by searching the image of the right lung and left lung separately by at least two different pre-trained image recognition methods independent of each other, other than neural networks, forming a plurality of rectangles, each of which covers one found lung and only it, based on the searches performed, forming a common rectangle, which includes essentially only the mentioned plurality of rectangles, and cutting off from the image areas that extend beyond the general rectangle; using said neural network, the prepared image is analyzed and the presence or absence of pathology in the image is determined; generating, using at least one processor, a report that contains at least one file containing an indication of the presence or absence of pathology; and transmitting a report using the communication device to the device that requested data processing. -