WO2023121510A1 - Determining chest pathology on the basis of x-ray images - Google Patents

Abstract

The invention relates to the field of image processing, and more particularly to a device and method for determining a pathology from the analysis of X-ray images. Proposed is a device comprising: a unit for data to be processed, which contain an X-ray image of chest organs; a unit for preparing the image contained in the data to be processed for use by a neural network, wherein the image is prepared by performing one or more preliminary transformations thereon; a pathology prediction unit for determining the presence or absence of a pathology on the image using the neural network; a unit for generating a report on the examination carried out in the present device; and a unit for transmitting said report to the device that requested the processing of the data. The invention makes it possible to more accurately automatically determine the likelihood of a pathology, as well as allowing medical decisions to be taken with greater speed and accuracy.

Description

DETERMINATION OF THE PATHOLOGY OF THE CHEST ORGANS ON THE BASIS OF X-RAY IMAGES

The present invention relates to the field of image processing, and more specifically, to a device and method for determining the pathology of the chest organs based on x-ray images.

In the traditional methods of diagnosing pathologies of the chest organs, which have been used in medicine for decades, the operator of the X-ray unit or the doctor visually examined the X-ray image obtained by the unit and made a conclusion about the presence or absence of this or that pathology in the examined patient.

However, the high prevalence of cancer, heart and lung diseases, as well as the outbreak of covid-19, which has led to a significant increase in mortality from these diseases, entail huge social costs. In medical institutions that carry out diagnostic and therapeutic activities, there is an overload and shortage of qualified medical specialists in the field of radiological studies, the evaluation and interpretation of the results of radiological studies is influenced by the human factor, there are no necessary resources for re-reading studies. The practice of carrying out diagnostic measures is associated with the risk of missing pathologies in the initial analysis of radiological studies.

These factors require fundamental research in this area in order to develop new and more effective therapeutic and diagnostic tools, including in order to improve the quality of medical decisions and, if possible, speed up the process of making medical decisions.

All this contributes to the introduction of modern digital data processing techniques into medicine. The digitalization of healthcare is accompanied by a rapidly growing volume of digitized medical patient information, clinical databases and medical datasets that could be used to support medical decision making. In recent years, artificial intelligence (AI) technologies have been increasingly used to work on such tasks.

For example, in known modern AI diagnostic methods such as WO 2021/031279 A1 or WO 2021/091661 A1, a neural network that has been previously trained on a labeled set of images receives a digitized x-ray image of a patient's chest, analyzes it, and makes a prediction about whether whether a given patient has a pathology, thereby helping the doctor to make an optimal informed decision.

Attempts are being made to apply a variety of neural network architectures, different training methods and different methods of image preprocessing, however, the accuracy of known methods is such that a significant part of the medical community is of the opinion that at the current level of development, such systems do not help the doctor, but rather interfere, because they may reduce the concentration of a specialist who relies partly on AI, and there may be errors in decision-making, which can affect the quality of medical services. And due to the fact that the responsibility for the decision lies with the person, the human factor is still great and labor costs and qualification requirements are high.

The essence of the invention.

In order to overcome at least some of the aforementioned shortcomings of the prior art, the present invention is directed to improving the accuracy of devices and methods for determining chest pathology based on x-ray images.

According to the first aspect of the present invention, there is provided a device for determining the pathology of the chest organs based on the analysis of x-ray images, comprising:

a data receiving unit configured to receive data to be processed comprising a frontal chest x-ray image from the device requesting data processing by communicating therewith;

an image preparation unit configured to prepare, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, moreover, in the process The image preparation unit is configured to automatically crop the image so that it covers only the lungs by:

- searching the image of the right lung and the left lung separately using at least two different independent pre-trained pattern recognition methods other than neural networks,

- formation of a set of rectangles, each of which covers one found lung and only it, based on the performed searches,

- forming a general rectangle that includes essentially only the said plurality of rectangles, and

- clipping from the image areas that go beyond the general rectangle;

a pathology prediction unit configured to use said neural network to analyze the prepared image and determine the presence or absence of pathology in the image;

a report generation unit, configured to generate, using at least one processor, a report on an examination conducted in the device, the examination report containing at least one file containing an indication of a result of processing performed in the pathology prediction unit; And

a data transmission unit configured to transmit the report to the device requesting data processing by communicating with it.

In one embodiment, if the image is determined to have pathology, the reporting unit is further configured to:

identifying possible areas where signs of pathologies are found;

forming an image with the visualization of the pathology in the form of an indication of the found areas with signs of pathologies; And

generating a study report containing an image with visualization of the pathology.

In one embodiment, the identification of possible areas with signs of pathologies is performed using computer vision methods that analyze the input image of the neural network and the activation within the neural network of the pathology predictor that were obtained by running this image through it.

In one embodiment, a combination of Grad-CAM (weighted combination of gradient-based class activation maps) and Saliency maps (significance maps) methods are used as computer vision methods.

In one of the embodiments, the indication of the found areas with signs of pathologies is made in the form of a heat map or an outline of the boundaries of the areas.

In one of the embodiments for the formation of an image with visualization of the pathology, the report generation unit is configured to overlay an indication of the found areas with signs of pathologies on a copy of the input image of the neural network of the pathology prediction unit.

In one of the embodiments, the indication of the found areas with signs of pathologies is linked to the size, shape and position of the image, which was directly analyzed by the neural network;

the image preparation unit is additionally configured to save the parameters of all transformations performed on the original image obtained in the data to be processed and associated with its resizing, displacement, rotation, cropping and cropping; And

to form an image with visualization of the pathology, the report generation unit is configured to:

apply transformations inverse to the above saved transformations to indicate the found areas with signs of pathologies, in order to obtain a visualization of the pathology with reference to the size, shape and position of the original x-ray image; And

superimpose the resulting visualization of the pathology with reference to the size, shape and position of the original x-ray image on a copy of the original x-ray image.

In one embodiment, three pre-trained methods are used as pattern recognition methods: the Viola-Jones method using a cascade classifier based on Haar features, the directed gradient histogram method, and the local binary template based method.

According to the second aspect of the present invention, a method for determining the pathology of the chest organs based on the analysis of x-ray images is provided, comprising the steps of:

receive using the communication device to be processed data containing x-ray image of the chest in frontal projection;

using at least one processor, an x-ray image contained in the data to be processed is prepared by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, and when preparing the image, the image is automatically cropped in such a way so that it covers only the lungs, by:

- searching the image of the right lung and the left lung separately using at least two different independent pre-trained pattern recognition methods other than neural networks,

- formation of a set of rectangles, each of which covers one found lung and only it, based on the performed searches,

- forming a general rectangle that includes essentially only the said plurality of rectangles, and

- clipping from the image areas that go beyond the general rectangle;

using said neural network, the prepared image is analyzed and the presence or absence of pathology in the image is determined;

generate using at least one processor report, which contains at least one file containing an indication of the presence or absence of pathology; And

transmitting, using the communication device, a report to the device requesting data processing.

The present invention improves the efficiency of devices and methods for determining the pathology of the chest organs based on x-ray images. This provides:

- increasing the accuracy of determining the pathology of the chest organs;

- reducing the requirements for the qualification of medical personnel;

- reducing the influence of the human factor (mindfulness, fatigue, responsibility).

It should be understood that not each of the embodiments can simultaneously provide all of these advantages over all known prior art solutions. Accordingly, some embodiments may have only some of these advantages or other advantages over some known solutions.

These and other advantages of the present invention will become apparent upon reading the following detailed description with reference to the accompanying drawings.

It should be understood that the figures may be represented schematically and not to scale and are intended primarily to improve understanding of the present invention.

Figure.1

shows a schematic representation of a medical decision support system according to the present invention.

Figure.2

shows an example of an image to be processed in the pathology detection device.

Figure.3

shows a block diagram of a device for determining the pathology according to the present invention.

Figure.4

shows examples of images that should not pass validation.

Figure.5

shows examples of images that should not pass validation.

Figure.6A

An example of cropping an image is shown.

Figure.6B

An example of cropping an image is shown.

Figure.7

an example of an image with areas of interest plotted as a heat map is given.

Figure.8

an example of an image on which the contours of areas of interest are plotted is given.

Figure.9

shows a flowchart of the method for determining the pathology according to the present invention.

General overview of the system

Next, with reference to FIG. 1, the chest health decision support system 100 will be generally described.

The medical diagnostic facility (MO) 110 contains an x-ray imaging device 120 (also may be referred to interchangeably as a x-ray diagnostic machine, x-ray machine, x-ray unit, x-ray, etc.). A medical specialist using X-ray machine 120 performs an x-ray examination of the chest organs of the examined patient (including fluorography).

As a medical organization 110, according to the present invention, there can be a clinic, a polyclinic, a doctor's office, a hospital, a hospital, a hospital, a sanatorium, a medical care center, a pharmacy, a mobile unit, a mobile fluorography room, or any other organization, room or installation equipped with device 120 for obtaining x-ray images.

The result of the study is formed in the form of an x-ray image, for example, in the format of DICOM (Digital Imaging and Communications in Medicine), NIfTI (Neural Imaging Technology Initiative in Informatics), Analyze, Minc1, Minc2 or other suitable open or proprietary format. The medical specialist, if necessary, can apply filters to the image before saving, which, in his opinion, can improve the perception of the image. In addition, the image can be superimposed in one of the embodiments of any preset filters that are specific to a particular device or to a particular manufacturer.

The generated image is stored in a local data store 131, such as a store based on or as part of a PACS (DICOM Image Archiving and Communication System), RIS (Radiological Information System), MIS (Medical Information System) 130, or other system or device suitable for storage of medical data. If necessary, additional target information can be added to the saved image, such as information about the patient, about the study, about the medical organization, about the medical specialist, etc.

MIS 130 (including PACS or RIS) can be deployed on a general-purpose computer or server, on specialized equipment, or on other data storage, processing and transmission hardware suitable for implementing the present invention and known to a person skilled in the art. In an illustrative non-limiting example, MIS 130 may be a computer having a 4-core processor, 8 GB of RAM, 60 GB disk space for the CentOS 7 operating system, 2 TB disk space for data, 1 TB disk space for the database, and 2 network adapters.

Storage 131 is shown in FIG. 1 as part of the MIS 130 by way of example only, and it should be understood that other implementations are possible in which the storage 131 is separate from the MIS 130, is part of the X-ray machine 120, or is part of another internal storage system of the medical organization. Implementations are also possible in which a medical organization does not use special systems for storing and transmitting medical data at all, but uses other available storage and transmission means for these purposes, such as a personal computer or laptop.

The data to be processed is then generated using a data communication device 132 such as PACS, RIS, MIS 130, or other suitable open or proprietary system. As with the storage 131, although the communication device 132 is shown in FIG. 1 as part of MIS 130, it can also be separate from MIS 130 or be part of another internal storage system of a medical organization.

In one of the embodiments, when generating the data to be processed in the data transmission device 132, depersonalization (anonymization, de-identification) of the x-ray image of the chest of the examined patient can be performed. To do this, patient information, which in one way or another can be considered personal, is replaced with anonymized data, from which a third party will not be able to restore the original data without having proper access to it. For example, in the case of a PACS system, the fields “PatientName” (patient name), “OtherPatientNames” (other patient names), “PatientID” (patient identifier) and, if necessary, other fields containing personal data or related to such, are anonymized.

The possible values of the depersonalized data may be predefined and known to all devices in the system, only to trusted devices, or only to the pathology device 160 so that they can determine whether the transmitted data is depersonalized.

The depersonalization process can occur both completely automatically, and if necessary, part of the depersonalization process or the entire depersonalization process can be performed with the participation of a person who can delete or edit data through the appropriate interface of the data transfer device 132 or medical information system 130.

The depersonalized data to be processed, an example of which is shown in FIG. 2 are transmitted from the medical organization 110 directly or through the central medical information system 150 to the device 160 for determining the pathology. At the same time, in one of the embodiments, the implementation of depersonalization can be performed in the central medical information system 150, and in this case, non-anonymized data can first be transmitted from the medical organization 110 to the central medical information system 150, and then depersonalized data can be transmitted from it to the device 160 for determining the pathology. data to be processed. The depersonalization process can occur both completely automatically, and if necessary, part of the depersonalization process or the entire depersonalization process can be performed with the participation of a person who can delete or edit data through the appropriate interface of the central medical information system 150.

The pathology detecting device 160 analyzes an X-ray image contained in the received data based on AI techniques and makes a prediction as to whether the image contains a pathology. If so, the pathology detecting device 160 indicates areas in the image that contain pathologies. If there is no pathology, then the image does not change.

Optionally, the device 160 for determining the pathology can generate a report (or protocol) on the result of the work, containing a description of the result in the form of textual information.

The generated data (one or more images and/or report) is sent back (directly or indirectly) from the pathology device 160 to the medical organization 110 that requested data processing.

The results of the work received from the device 160 for determining the pathology are provided to the responsible person, for example, a radiologist, the attending physician or other medical specialist or a person who has access to such information and is responsible for receiving and processing it in the medical organization 110, using the viewer 140 , and using them, he makes a decision about the state of the chest organs, namely the presence or absence of a particular pathology. If necessary, the medical specialist, taking into account the results of the device 160 for determining the pathology, can make a decision on the treatment of the patient. Viewer 140 in FIG. 1 is simplistically referred to as display 140, however, in the preferred embodiment, it is an AWP (workstation) of a physician. In an illustrative non-limiting example, a physician's workstation may be a computer based on an Intel Core i3 processor or equivalent, having 8 GB of RAM, 40 GB of free disk space, a DVD-R/RW CD-ROM drive, a network connection speed of 5 Mbps, monitor with a screen resolution of 1920x1080 and can be viewed using DICOM image viewers or through a web browser.

Thus, the system 100 helps to improve the accuracy of medical decision making.

It is also possible that the data generated by the device 160 for determining the pathology is not sent directly to the medical organization 110 that requested data processing, but first (directly or through the central medical information system 150) to a specialized expert organization 170 that produces medical reports using the results. operation of the pathology device 160, or to an external radiologist 180 acting as an expert or consultant. In this case, the medical organization 110, in response to the sent x-ray image, can receive from the expert organization 170 or from the external radiologist 180 (again, directly or through the central medical information system 150) an immediately ready conclusion or a preliminary conclusion that can be used to make a medical decision. This allows to reduce the requirements for qualification and workload of the personnel of a medical organization 110 up to the possibility of not having a radiologist on the staff at all, which provides a significant expansion of the geography of the possible use of the system and allows you to receive highly qualified medical services in those locations where they were not available before. due to the lack of properly qualified personnel, and on the other hand, allows radiologists to connect to the system from various remote locations without being tied to a specific medical organization.

It should be understood that in this document blocks 170 and 180 are simply referred to as "expert organization 170" and "external radiologist 180", however, technically, from the point of view of the system, these blocks are equipment / devices for receiving, viewing, editing and transmitting data managed/owned/used by an expert organization 170 and an external radiologist 180.

According to the present invention, the term "external" in relation to a radiologist means that this doctor is not on the staff of the medical organization 110 that conducted the X-ray examination and requested the processing of the resulting image, and / or is not physically located in this organization and / or does not have access to MIS 130 of this organization. In addition, it should be understood that the term “radiologist” in the context of the present invention implies that this is a medical specialist who has a proven qualification (knowledge, skills, abilities and experience) in the analysis (interpretation) of the results of an x-ray examination.

The expert organization 170 may be a medical or other organization with the functions of analyzing medical images and drawing conclusions and having one or more specialists of the appropriate profile and appropriate qualifications for the analysis of the results of X-ray examination.

Specialists of the expert organization 170 and external radiologists 180 can access data from a specialized workstation (doctor's workstation) or using another suitable device, such as a computer, laptop, smartphone, tablet, VR helmet (virtual reality helmet), VR -glasses, etc.

The central medical information system 150 shown in FIG. 1, an expert organization 170 or an external radiologist 180 are not required elements of the proposed medical decision support system 100. Accordingly, in one embodiment, the implementation of the medical organization 110 may communicate directly with the device 160 to determine the pathology. This provides a simplified implementation of the medical decision support system 100 . Such an implementation can be convenient, for example, in cases where the number of medical organizations 110 served by the pathology device 160 is relatively small, and expert organizations 170 or external radiologists 180 are not involved at all or their number is also relatively small.

In another embodiment, when the medical decision support system 100 includes a plurality of medical organizations 110 and/or a plurality of pathology devices 160, as well as expert organizations 170 or external radiologists 180, it is advisable to use a central medical information system 150. It should be noted that the term “central” in this case indicates, first of all, not that this is a single central server that closes all possible connections, but that the central medical information system 150 occupies a place in the middle, in the center between the other participants in the medical decision support system 100, acting as an intermediate system for collecting, storing and redistributing data. In this case, depending on the requirements of a particular application, the central medical information system 150 can be both concentrated (centralized) and distributed, including those implemented in the cloud.

Images awaiting processing may be grouped into batches (series) for transfer to the device 160 for determining the pathology. Accordingly, the pathology detecting device 160 can perform batch processing of the received images. Grouping into packages can be performed both in the medical organization 110 and in the central medical information system 150. The central medical information system 150 can change the size and content of the packages received from the medical organizations 110 - for example, sort images by their resolution in pixels, by priority or other parameters and form new packages, and if necessary, images from different medical organizations 110 can be added to the same package.

In one embodiment, central health information system 150 may perform primary validation or pre-processing of images. Also, primary validation or pre-processing is possible on the side of the medical organization 110. Specific validation or pre-processing operations are described later in this document in relation to the device for determining the pathology, therefore, are not disclosed here in detail.

The structure of the device for determining the pathology of the chest organs

Next, with reference to FIG. 3, the device 200 for detecting chest pathology based on X-ray images according to the present invention will be described in detail. It should be noted that the chest pathology device 200 fully corresponds to the pathology device 160 shown in FIG. 1, and has the same functions and capabilities, if applicable and does not conflict with the descriptions in this section.

The pathology determination device 200 includes a data receiving unit 210, a data storage unit 220, a data validation unit 230, an image preparation unit 240, a pathology prediction unit 250, a report generation unit 260, a data transmission unit 270, and a learning unit 280. Depending on the particular application, some of these blocks may be omitted, as will be explained in more detail later in this document.

The data receiving unit 210 receives data to be processed containing an x-ray image of the chest of the patient being examined. The receiving unit 210 may be a separate chip, network card, or other suitable means capable of communicating with external devices in a wired and/or wireless manner, such as over a local area network (LAN) protocol, the Internet, etc. using Ethernet, fiber, WiFi, 4G, etc. technologies.

The data storage unit 220 stores the data received by the data receiving unit 210 so that other units of the apparatus 200 can use it at the appropriate time. Received data can only be retained while it is being processed and erased when its processing is completed. For these purposes, in one embodiment, a short-term storage device such as RAM or the like is used. In another embodiment, the received data may be stored for a longer period of time than the immediate processing time, if necessary, and then a long-term storage device such as a hard disk or the like may be used.

Additionally, the data storage unit 220 may store short-term or long-term data and/or files resulting from or during the operation of other units of the device.

The data validator 230 receives the data to be processed directly from the data receiving unit 210 or retrieves it from the data storage unit 220 . The data validator 230 then checks whether the received data is suitable for processing.

As mentioned above in the description section of the medical decision support system 100, part of the validation operations can be performed on the side of the medical organization itself or on the side of the central medical information system 150. In this case, if the device 200 for determining the pathology knows which validation operations have already been performed with regard to the data to be processed, the data validator 230 may not perform these operations, which can simplify and speed up processing, and thereby improve productivity. However, in another embodiment, the pathology device 200 may not know which validations have already been performed, or it may re-perform them for further revalidation. This can improve the processing quality.

Moving specifically to the validation operations, an attempt is made to extract an image from the data to be processed. If the attempt fails, then it is concluded that the image file is corrupted or cannot be read. The reason for this may be factors such as the absence of an image in the data, the impossibility of reading metadata, the presence of any anomalous and unaccounted for tag values, etc. In this case, the data is not transferred for processing, and a corresponding indication is created for it. It is preferable to perform this operation in the data validation block 230, since even if it has already been performed by other devices, the data may be corrupted in the process of being sent to the pathology device 200 or may be in a format that is not currently available for one reason or another. device 200.

It can also be checked whether the data to be processed matches the type of processing that is performed in this device 200 to determine the pathology. For example, if the pathology detecting apparatus 200 is to process AP chest X-rays, a check may be made whether the attached data contains a AP chest X-ray. Various implementations of such a check are known to those skilled in the art and may include, for example, a pre-trained neural network (neural network) that produces the appropriate classification, or other computer vision methods. If the check fails, then such data is not transferred for processing, and a corresponding indication is created for them. Examples of such errors, when the study is claimed to be an X-ray of the chest in a direct projection, but in fact it is not and cannot be processed, are shown in Fig. 4 and 5.

In addition, a check can be made whether the data is depersonalized. To do this, it is checked whether the fields that relate to personal data contain any values, and if so, whether these values are depersonalized. For example, if the "PatientName" field is empty or contains the predefined value "0" or "Anonymous" as indicated in FIG. 2 and FIG. 5, then this field is considered to be depersonalized, and the field with the value "Venus de Milo", indicated in FIG. 4, is not depersonalized. If the data is not depersonalized, then the corresponding image is not transferred for processing, and an indication of the impossibility of processing is generated on it.

It should be noted that the pathology device 200 may support several different valid values for the depersonalized data, in which case the data validator 230 may check if the field values in the received data match at least one of the respective supported values. If the value does not match the predefined valid value, then it is assumed that the received data is not depersonalized. In this case, they are not transferred for processing, and a corresponding indication is created for them.

It can also be checked if the captured image has a size (pixel resolution) equal to or greater than a predetermined minimum size supported by the device 200, such as 1024x1024 pixels or 800x800 pixels, depending on the requirements of the particular application. If the source image is smaller than the minimum size, then the device 200 may not be accurate enough, so the image is not submitted for processing and an indication is generated for it.

In addition, a check can be made whether the image is a positive or a negative. Various implementations of such a check are known to those skilled in the art and may include, for example, a pre-trained neural network (neural network) that produces the appropriate classification, or other computer vision methods. If the inspection reveals that the image does not meet the input requirements of the pathology detection device 200, then the image preparation unit 240 can subsequently perform an appropriate conversion of the image to a negative or a positive.

Validation allows you to filter out data that cannot be analyzed or the accuracy of processing will have a deliberately low accuracy. Accordingly, the load on the most resource-intensive part of the analysis is reduced and the prediction accuracy is increased. In addition, screening out images that are not depersonalized ensures that no personal data is processed on the device 200 side, which reduces requirements for its implementation and certification.

The image preparation unit 240 receives from the data storage unit 220 and/or from the data validation unit 230 the validated chest x-ray image and performs preliminary transformations on it in order to prepare it for direct use in the pathology prediction unit 250 .

In particular, the image preparation performed in block 240 may be as follows.

The X-ray image to be processed is read and, if necessary, converted to grayscale with a predetermined color depth. The brightness and color parameters of the original image depend on the parameters set by the radiologist when working with the X-ray unit and are contained in the image metadata. For example, the color depth of the original image may be 12-16 bits. In this case, for example, as a result of reading and converting to an image with a color depth of 8 bits, a matrix of integer pixel values \u200b\u200bfrom 0 to 255 can be obtained. Also, as mentioned above, an appropriate conversion of the image to a negative or positive can be performed.

The initial size of X-ray images is generally relatively large and averages 2500x2500 pixels, that is, 2500 pixels in height (vertically) and 2500 pixels in length (horizontally). Images at this size are generally amenable to processing using machine learning methods, but such models are very resource intensive.

In addition, the range of sizes of input x-ray images can be quite large, which can cause inconvenience if you try to process each individual image directly in the original size. In particular, it becomes difficult to choose a set of processing methods that are equally effective for different sizes.

In this regard, before processing large radiographs, it is preferable to reduce their size. This allows, without significant loss of quality, to significantly reduce the resources expended: time, computing power, energy consumption. Various methods of size reduction are applicable - for example, compression, cropping (cropping), etc. Compression techniques are known to those skilled in the art, so a detailed description of the compression process is not provided here. As for image cropping, it will be described later.

For example, in one embodiment, if a region of interest (VOI) is predefined in the image parameters by the medical professional performing the study, appropriate image cropping (VOI LUT) can be performed to obtain an image containing only the region of interest. For example, if the medical specialist who performed the chest x-ray has specified an area of interest, then cropping is applied according to the parameters embedded in the image, as a result of which only the lung region remains on the image.

In another embodiment, in addition to or instead of the aforementioned VOI LUT cropping, a lung region is searched for in the image and the image is cropped by cutting off parts of the image that are not included in the found lung region.

To do this, first the size of the input image is reduced to a single first size. The first image size is predefined - for example, 800x800 pixels. The value chosen depends on the requirements of the particular application and the performance of the equipment being used.

In particular, at the design stage, an assessment can be made of what is the smallest size of the x-ray image that can enter the device 200 from the medical organization 110 and / or from the central medical information system 150, and in accordance with this, the aforementioned first size is set, which is less than or equal to the smallest possible size of the input x-ray image.

However, in practice there are also situations where the smallest possible size of the input x-ray image is too small or the principle of "less than less" for one reason or another is not suitable for a particular application. Then the first size, to which all input X-ray images are reduced, can be set according to a different principle, and in such a scenario, some of the input images will be compressed, some will be stretched, and some will remain unchanged.

Then cropping (crop, crop) of the image is performed so that it covers only the lungs. In particular, the image is analyzed separately by different pattern recognition methods - for example, the Viola-Jones method using a cascade classifier based on Haar features (Haar Cascades), the Histogram of Oriented Gradients (HOG) method, and the method based on local binary patterns (Local Binary Patterns, LBP), resulting in the formation of several rectangles, each of which indicates an area that covers only the right and left lung. The aspect ratio of the rectangles depends on the size and shape of the lungs. The analysis may be performed in parallel, sequentially, or in any other suitable manner. The pattern recognition algorithms used must be pre-trained to recognize lungs in an image having a first dimension.

The Viola-Jones method, the directed gradient histogram method, and the local binary template method mentioned above are somewhat faster than traditional convolutional neural networks and require a slightly smaller training set.

The Viola-Jones method has a very high accuracy when the recognizable object in the image has a rotation angle of no more than 30-35 degrees. Accordingly, it is well applicable to the analysis of X-ray images, since the position of the patient is typed to obtain an image, for example, in frontal projection or in lateral projection.

The directional gradient histogram method is weakly sensitive to displacement, scale and brightness of the image and somewhat sensitive to changes in object orientation. However, for the reasons mentioned above, it is also well applicable to the analysis of X-ray images.

Methods based on local binary templates generally have high recognition speed and accuracy, and are also weakly sensitive to image brightness and object orientation changes.

Then one general rectangle is built (see example in Fig. 6A), which includes all previously obtained rectangles (1, 2 and 3 for the right and left lung). The input (reduced to the first size) image is cropped to this general rectangle, the remaining areas of the image are discarded, so the cropped image contains only the lung region (see example in Fig. 6B). In one embodiment, each side of the general rectangle contains at least one of the sides of the rectangles obtained as a result of image analysis, with none of the sides of these rectangles protruding beyond the general rectangle.

The results of different pattern recognition methods are generally not identical (in the example in Fig. 6A, rectangles 1, 2, and 3 are different), so this combination of results allows you to cover the lungs completely without erroneously cutting off their parts, but at the same time focus on areas of the lungs, excluding areas that are not of interest for this study.

In addition, there is a chance that one or two of the pattern recognition methods used will not be able to find the lung in the image. However, the probability that all three methods will not find the lung is extremely small. Therefore, the use of several methods of pattern recognition at once is justified, since it provides protection against data loss and increases the accuracy of prediction.

Other embodiments are also possible, when around the rectangles obtained as a result of the analysis, as shown in FIG. 6A, a small marginal zone (padding) of one or more pixels is additionally captured if the particular analysis methods chosen are not precise enough and may result in the loss of regions of interest. The size of the indents up, down, right and left may vary. The optimal indent sizes can be selected depending on the results demonstrated by the trained algorithms and the requirements for accuracy. In a specific non-limiting example, if the image being processed is 512x512 pixels and all three of the above pattern recognition methods are applied, the padding may be 15 pixels (or less if the padding extends beyond the image boundary). This avoids data loss and improves prediction accuracy.

The aspect ratio of the cropped lung image (ie, the overall rectangle) (FIG. 6B) depends on the size and shape of the lungs. For further processing, it is required to bring it to a single format. To do this, the image is resized to the second size. The second image size is predefined - for example, in the form of a square 224x224, 320x320 or 512x512 pixels. The selected value depends on the requirements of the specific neural network used further. As for the first image size (800x800 pixels) mentioned above, it allows you to significantly reduce the size of the original image to speed up processing, but at the same time get a relatively large image as a result of cropping, which in most cases would not have to be stretched to bring it to the second size , which could potentially reduce processing accuracy.

The image brought to the second size is subjected to normalization. In particular, pixel values from the original range (for example, [0…255]) are reduced to the range required by the neural network applied further (for example, [0…1]). Various normalization methods are known to those skilled in the art and are not detailed here. In an illustrative non-limiting example, simple normalization (X-Xmin)/(Xmax-Xmin) can be applied, where X is the value of the current pixel, and Xmin and Xmax are the minimum and maximum pixel values in the image being normalized.

It should be understood that embodiments are possible in which image normalization can be performed at an earlier stage - for example, even before the first image compression. Nevertheless, it is preferable to perform it after bringing the cropped image to the second size, since this allows you to increase the accuracy at each of the previous stages of image preparation (pre-processing), as well as somewhat reduce the amount of calculations performed directly during normalization.

Thus, an image is created, prepared for further processing.

The pathology prediction unit 250 receives from the data storage unit 220 and/or from the image preparation unit 240 the prepared chest x-ray image limited to the lung region and analyzes it. In particular, the pathology predictor 250, using a pretrained neural network, determines the presence or absence of a pathology and, if present, determines the most likely boundaries within which it is located.

There are a number of pathological conditions of the chest organs that can be identified by an x-ray image by a doctor. Creating an AI-based device that would analyze images and give an accurate indication for each of the many possible pathologies is difficult and impractical, since many pathologies have similar signs, and a depersonalized image alone is often not enough for a more detailed prediction, even if neural networks are trained on large qualitatively labeled data sets.

Accordingly, in this invention it is proposed not to analyze the differentiated probability of a particular pathology, but instead to determine the probability of the presence of at least one radiological sign from the following list:

- Pleural effusion;

- Pneumothorax;

- Atelectasis;

- Center of blackout;

- Infiltration/consolidation;

- dissemination;

- Cavity with decay;

- Cavity with liquid level;

- Calcification / calcified shadow in the lungs;

- Violation of the integrity of the cortical layer.

It should be noted that the term "pneumonia" unites a large group of diseases, each of which has its own etiology, pathogenesis, clinical picture, radiological signs, characteristic laboratory data and therapy features. Considering that many of the radiological signs listed above can be observed in pneumonia, the present invention allows, among other things, to detect signs of pneumonia.

A doctor who has received the result of the above determination, has some experience, and also, if necessary, has additional information about the patient, such as anamnesis, medical history, complaints, physical condition, test results, etc., can have a more complete picture and make more accurate conclusion. Accordingly, it should be noted that the result of the operation of the proposed device is not a clinically significant medical opinion, but is used to support medical decision-making, while the clinical opinion should be taken by a radiologist.

The image processing performed in block 250 may be as follows.

The neural network interprets the resulting image in order to identify signs of pathologies, while the result of the work is the probability of having at least one radiological sign from the above list.

At the output of the neural network, for example, a binary number can be displayed indicating whether there is a pathology in this image. "0" may indicate the absence of pathology (that is, that the patient is healthy), and "1" may indicate the presence of pathology. The reverse indication is also possible, where “1” is healthy, “0” is probably sick. In another embodiment, the output of the neural network may be a number in a range, such as 0 to 1 or 0 to 100, indicating the likelihood of pathology. A text format of the indication is also possible.

Next, the pre-training of the model will be described in more detail. In particular, it is produced by a supervised learning method based on a plurality of x-ray images that have been pre-processed and labeled by medical professionals with sufficient qualifications, such as radiologists. In particular, as a result of a visual examination of the images, the doctor indicates whether there is a pathology in the presented image. For example, "0" may indicate the absence of pathology (that is, that the patient is healthy), and "1" may indicate the presence of pathology, or vice versa. This is done either manually by the doctor, or is read automatically from the patient data stored in the medical information system 130 of the medical organization 110 or in the central medical information system 150, or from the image metadata if it contains this information, etc. Adding labels manually can be done by analyzing the image description text by the doctor, on the basis of which a conclusion was made about the presence / absence of pathology in the image. It is also possible to automatically recognize existing description texts for images/images, however, it should be understood that the recognition accuracy in the general case is not equal to 100%, which reduces the degree of confidence in such markup and introduces potential errors in the learning process.

The learning process of the neural network is controlled by the learning block 280 by applying a learning algorithm to the trained neural network using the training data. Instructions (markup) from the doctor are used by the trained neural network as ground truth. For example, in one embodiment, the markup may be a number: "1" (sick). In another embodiment, the markup may be a different value, such as "healthy".

In an exemplary embodiment, 70% of the training images can be used directly for training and 30% for model testing. In another embodiment, a different ratio may be used, more suitable for the purposes of training a particular neural network. For example, to train a neural network, images can be divided into training, validation, and test sets. The training set size can be 70%, the validation set size can be 15%, and the test set size can be 15% of the images submitted for training. Specific ratios depend on the number of training images and are intended to test the generalized model - the extent to which the results of the study are applicable to new data is checked before using the model in production mode in a medical organization.

During the learning process, images are prepared in the same way as described above with respect to block 240, and fed to the input of the neural network, using images with both the presence and absence of pathologies. At each training step, the neural network calculates predictions for one or more images. These predictions are compared with the indication of the true presence/absence of pathology, and the value of the loss function is calculated (how badly the neural network was wrong in detecting the presence of pathology). Further, using the gradient descent method and the backpropagation algorithm, all weights (weight coefficients) of the neural network are changed in accordance with the selected learning rate parameter in the direction opposite to the calculated gradient in order to minimize the error on the current image(s) ( I). This step is repeated many times, and as a result of the learning process, the weights of the neural network converge to the optimal ones. In the future, these weights are used by the network during the operation of the device to detect pathologies in the input images that have not been labeled.

In one of the embodiments, the output data of the pathology prediction block 250, that is, an indication of the probability of the presence of a pathology, is the end result of the operation of the device 200 and is further used by other devices - for example, stored in the data storage block 220, from which it can later be retrieved upon request from other devices , and/or sent to the data transfer unit 270, which transmits this result to another device - for example, to the central medical information system 150 or to the medical organization 110 that requested data processing.

In another embodiment, the output of the pathology predictor 250 is also passed to a report generator 260, which generates a report on the examination performed on the device 200. The examination report may comprise at least one of the following: a) a pathology image; b) text protocol of the study.

A pathology image can be generated for those x-ray images for which the analysis revealed a probability of pathology that exceeds a predetermined threshold probability of pathology, which indicates that the radiologist should pay attention to such an image. The threshold probability of the presence of a pathology in the general case is 0.5 (or 50%), but depending on the accuracy of a particular trained model and the degree of confidence in it, it can vary in practical application both up and down. For example, a medical organization may specialize in the treatment of diseases, the early detection of which is critical in terms of treatment prognosis, so it may request imaging images if the device detects a probability of pathology of only 10% or more. In another example, a medical organization may be attached to an enterprise, and the main contingent of its patients may have occupational chronic diseases of the chest, and some changes in the images in such patients may be considered acceptable, so imaging may be requested only for cases where the device has detected the likelihood of having pathology 60% or more. It should be noted that many other situations are possible, which are not listed in this document, but are included in the scope of this invention. The ability to select a threshold probability of the presence of pathology allows the results of the proposed device 200 to be adapted to various specific applications, for example, to reduce the burden on medical personnel and at the same time provide early detection of pathologies with the required accuracy.

An image with visualization of pathology is a copy of the analyzed image, on which an indication of the found signs of pathologies is superimposed. For example, in the case of using the DICOM format, it can be generated as a DICOM Secondary Capture (secondary DICOM snapshot).

Various visualization options are possible. In one of the embodiments, the visualization can be performed in the form of a heat map, on which more intensive selection of those points and/or areas that are of greater interest in terms of the presence of pathologies is performed, less intense selection of those points and/or areas that are of less interest , and those areas that, presumably, do not contain pathology, are not highlighted. The principle of selection can be different - for example, selection can be performed using a gradient of the same color (for example, shades of red or orange), using a gradient transition from one color to another (for example, from yellow to red), using an exact specifying a color (for example, blue - less likely, blue - more likely), using the size of the dot (for example, the lower the probability, the smaller the dot), etc. Various methods for constructing heat maps are applicable - for example, a combination of two methods shows increased performance: Grad-CAM (a weighted combination of class activation maps based on gradients) and Saliency maps (significance maps). In particular, each of these methods separately analyzes the input image of the neural network and the activations (numbers) inside the neural network that were obtained when running this image through it, after which the results of the methods are combined, and a general heat map is built, an example of which is shown in Fig. 7.

Heat maps allow you to adjust the amount of attention that a medical specialist should pay when studying a particular area in the image, eliminating the need for a detailed visual inspection of the entire image. Thus, the burden on medical personnel is reduced.

However, there are situations when the heat map is not accurate enough, in which case the doctor, on the contrary, has to spend time studying an area that is not of interest. It also turned out that in some implementations, heat maps clutter up the image, as a result of which the specialist has to spend time viewing two images at the same time: both the original image and the image with the heat map applied to it.

Accordingly, in another embodiment of the present invention, the visualization may be performed as a delineation of boundaries (or contours) covering one or more areas in the image in which there is a possibility of pathology, or in other words, in which signs of pathology are detected by the pathology predictor 250 . To implement this approach, various computer vision methods can be used that analyze the state of the neural network of the pathology predictor 250 . For example, as mentioned above, increased performance is shown by a combination of two methods: the Grad-CAM method (a weighted combination of class activation maps based on gradients) and Saliency maps (significance maps). A predetermined threshold value of heat is applied to the heat map built on their basis, according to which the contour is drawn (for example, 0.25, 0.3, etc.). In this way, closed loops are formed that enclose regions with heat equal to or above the threshold, while regions with heat below the threshold are discarded. The heat maps themselves are not applied to the image, only the contours are rendered on it, as shown in the example in Fig. 8.

The image with the outlines of the boundaries of possible pathologies printed on it, as before, makes it possible to reduce the burden on medical personnel, while it is enough to study only it without referring to the original image. In addition, this approach makes it possible to smooth out inaccuracies in the determination of areas of interest.

In one embodiment, an indication of the found signs of pathologies in the form of a heat map or pathological boundaries can be superimposed on a copy of the image that was directly analyzed by the neural network, that is, on the image that, compared with the original X-ray image obtained in the data to be processed, was compressed, cropped and bringing them to the same size. This simplifies the process of visualizing the pathology, but in this case the resulting image differs from the original, so this may cause some inconvenience for the radiologist comparing these two images.

In another embodiment, an indication of the found signs of pathologies in the form of a heat map or the boundaries of pathologies can be superimposed on the original x-ray image obtained in the data to be processed. To implement this approach, the parameters of all transformations performed on the original image and associated with changing its size, shifting, rotating, cropping and cropping are stored by the image preparation block 240 - for example, in the storage block 220 . After working out the methods for searching for signs of pathologies, a visualization of the pathology is obtained in the form of a heat map or the boundaries of pathologies with reference to the size, shape and position of the image, which was directly analyzed by the neural network. The reverse transformations of the above stored transformations are applied to the resulting imaging in order to obtain a pathology imaging with reference to the size, shape and position of the original x-ray image. Then such visualization is superimposed on a copy of the original image. This somewhat complicates the processing, but reduces the time for image analysis by a radiologist and improves the accuracy of pathology identification. This is especially applicable to heat maps, when the comparison of the original image and the image with the visualization of the pathology is in many ways an urgent need for the radiologist.

If necessary, the pathology visualization image may contain in text form an indication of the probability of pathology, obtained as a result of the operation of the first neural network of the pathology prediction block 250 . This reduces the need to study any other images or files, including the original image and/or a file with a text study protocol.

Turning now to the text protocol of the study, it can be noted that it can both supplement the above image with visualization of the pathology, and replace it. The text protocol of the study contains the results of the work and a description of the operation of the device 200 in the form of textual information, for example, in CSV format. In the case of using the DICOM format, the protocol can be generated in the form of a DICOM Structured Report (structured DICOM report).

In particular, the text protocol of the study may contain:

1) The name of the original image (a link to it in the form of an appropriate identifier - for example, StudyInstanceID).

2) Recommendation (or in other words, prediction). As noted above, this recommendation may be a binary value (for example, 0 / 1), may be a number in some range indicating the likelihood of a pathology (for example, from 0 to 1), or may be in text format (for example, “probably sick / "probably healthy").

3) Timestamps - the time the study was received, the time the report was generated.

4) Information about errors in the operation of the device during the analysis, if any.

The study report obtained as a result of the operation of block 260, containing an image with visualization of the pathology and / or a text protocol of the study, is stored in the data storage block 220, from where it can later be retrieved upon request from other devices, and / or sent to the data transmission block 270, which transmits this report to another device - for example, to the central medical information system 150 or to the medical organization 110 that requested data processing.

In one embodiment, chest x-ray results acquired by device 200 may be stored in the device for a predetermined time (eg, seven days) as needed and then deleted. This can be done, for example, for the purpose of backup, so that the result of the analysis can be requested and retransmitted without the need for a full reanalysis.

Thus, there is provided a chest pathology detecting device capable of automatically detecting pathology probability with improved accuracy. This reduces the requirements for the qualification of medical personnel and reduces the influence of the human factor (attention, fatigue, responsibility). It also provides the possibility of using a large number of disparate data sets for training. In addition, the result of the study provides a comprehensive set of information necessary for making the correct medical decision with increased speed and accuracy.

In an illustrative non-limiting example for the purposes of the present invention, the pathology detection device 200 can be implemented as a cloud server having 24 computing cores, 64 GB RAM, 300 GB disk space for the CentOS 7 operating system, 12 TB disk space for data, disk space 1, 5 TB for the database and 2 network adapters.

Method for determining the pathology of the chest organs

Next, with reference to FIG. 9, a method 400 for determining chest pathology will be described.

It should be understood that the steps of the method 400 correspond to the above-described functions performed by each of the blocks in the device 200, and if any information is not disclosed in relation to the method, but is disclosed in relation to the device, and vice versa, then this does not imply that in the device or method this function or step cannot be performed, but is done only in order not to clutter up the description with repetition of details. It should also be understood that, depending on the particular application, some of the method steps described may not be used.

The pathology determination method 400 may comprise step S410, in which data to be processed is received containing an X-ray image of the chest of the patient being examined.

The pathology determination method 400 may also comprise a step S430 in which validation is performed by checking whether the data to be processed is suitable for processing.

The pathology determination method 400 may also include step S440, which prepares an image contained in the data to be processed for use by a neural network by performing one or more preliminary transformations on it.

The pathology determination method 400 comprises step S450, in which the prepared image is analyzed using a pre-trained neural network and the presence or absence of organ pathology in the image is determined.

The pathology determination method 400 may also comprise step S451, in which, if there is a pathology, the most likely boundaries within which it is located are determined using a pretrained neural network.

The pathology determination method 400 may also comprise step S460 generating an examination report containing at least one of the following: a) a pathology imaging image; b) text protocol of the study.

The pathology determination method 400 may also comprise step S470, which transmits the processing result obtained in steps S450, S451, and S460 to the device requesting data processing.

The pathology determination method 400 may also include step S480 (not shown) in which the neural network is pre-trained using the pre-labeled training images, where the label for each image contains an indication of the presence or absence of pathology of the organs in the image area, in relation to which it is required to perform neural network training.

The pathology determination method 400 may also comprise step S490 (not shown) in which, using the processing result obtained in steps S450, S451 and S460, a medical decision is made on the presence or absence of pathology of the organs in the image area and, if necessary, on the nature of the pathology.

The pathology determination method 400 may also comprise step S491 (not shown) in which the patient is prescribed and/or treated based on the processing result obtained in steps S450, S451 and S460 and/or the medical decision made in step S490.

The pathology detection method 400 may also include step S420 (not shown) storing data received by the data receiving unit 210 and/or resulting from or during other method steps.

The proposed method for determining the pathology of the chest organs provides increased accuracy of automatic determination of the probability of pathology, reduces the requirements for the qualification of medical personnel and reduces the influence of the human factor (mindfulness, fatigue, responsibility). In addition, the result of the study provides a comprehensive set of information necessary for making the correct medical decision with increased speed and accuracy.

Devices and methods according to the present invention can be used to process x-ray images of chest organs in order to detect signs of pathologies in them.

Additional Implementation Features

Although this document may indicate that data is being transmitted/sent or received/received by a person (e.g., medical professional, physician, expert), one skilled in the art should understand that such indication is used solely for the purpose of simplifying the description. , while in fact it is understood that the data is transmitted / sent or received / received by the corresponding device used and / or controlled by this person.

One or more transmission units or devices (transmitters) described herein and one or more receiving units or devices (receivers) may be physically implemented in the same transceiver unit or device or in different blocks or devices.

A device or a transmission unit in this document, for the sake of simplicity of description, may be referred to as a device or unit having the functions of not only transmitting, but also receiving data, information and/or signals. Similarly, the receiving device or unit may also include data, information and/or signaling functions.

Various illustrative blocks and modules described in connection with the disclosure herein may be implemented or executed by a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable logic device (PLD), discrete logic element or transistor logic, discrete hardware components, or any combination of the foregoing, designed to perform the functions described in this document. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (eg, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors together with a DSP core, or any other similar configuration).

Some blocks individually or collectively may be, for example, a computer, and include a processor that is configured to call and execute computer programs from memory to perform method steps or block functions in accordance with embodiments of the present invention. According to embodiments, the device may further include a memory. The processor may call and execute computer programs from memory to execute the method. The memory may be a separate device independent of the processor or may be integrated with the processor. The memory may store code, instructions, commands, and/or data for execution on a set of one or more processors of the device described. Codes, instructions, commands may direct the processor to perform the steps of a method or device function.

The functions described herein may be implemented in hardware, software running on one or more processors, firmware, or any combination of the foregoing. The hardware and software implementing the functions may also be physically located in different locations, including such a distribution that parts of the functions are implemented in different physical locations, that is, distributed processing or distributed computing may be performed.

If necessary (for example, if the amount of data and / or calculations that need to be performed on this data is large), multi-threaded data processing can be performed, which in a simple representation can be expressed in the fact that the entire set of data to be processed is divided into a set of subsets , and each processor core performs processing on its assigned subset of data.

The above memory may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. One of ordinary skill in the art will also understand that when referring to memory and storage of data, programs, codes, instructions, commands, and the like, it is understood that there is a machine-readable (or computer-readable, processor-readable) storage medium. Computer-readable storage media includes both non-transitory computer storage media and communication media, including any transmission medium that facilitates movement of a computer program, or portion thereof, from one place to another. A non-transitory computer-readable storage medium can be any available medium that can be used to carry or store a desired piece of program code in the form of instructions or data structures, and that can be accessed by a computer, processor, or other general purpose or special processing device. destination.

By way of example, and not limitation, computer-readable media may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), flash memory. , Random Access Memory (RAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) , accelerated speed synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct access bus random access memory (DR RAM), register, cache, semiconductor memories, magnetic media such as internal hard disks and removable disks, magneto-optical media and optical media such as CD-ROMs and digital versatile disks (DVDs), as well as any other storage media known in the art.

The information and signals described herein may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols, and elementary signals that may be exemplified in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combinations of the above, if applicable to the present invention.

At least one of the steps in the method or blocks in the device may use an artificial intelligence (AI) model to perform the respective operations. The AI-related function may be executed via a processor and non-volatile and/or volatile memory.

The processor may include one or more processors. At the same time, one or more processors may be a general purpose processor such as a central processing unit (CPU), an application processor (AP) and the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU). ), and/or a dedicated AI processor such as a Neural Processing Unit (NPU).

One or more processors direct the processing of input data in accordance with a predetermined operating rule or artificial intelligence (AI) model stored in non-volatile memory and/or non-volatile memory. A predetermined operating rule or artificial intelligence model can be obtained through training. In doing so, the processor may perform a pre-processing operation on the data to convert it into a form suitable for use as input to the artificial intelligence model.

"Obtained by training" means that by applying a learning algorithm to a trainable artificial intelligence model using a set of training data, a predefined operation rule or AI model with a desired characteristic is created. The training may be performed on the device itself running the AI according to the embodiment and/or may be implemented via a separate server/system.

An artificial intelligence model may include multiple neural network layers. Each of the multiple layers of the neural network includes a plurality of weights (coefficients) and performs a work operation for a given layer by computing using the plurality of weights of a given layer with respect to the input or calculation result of a previous layer.

Examples of neural networks include, but are not limited to, Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN). ), generative adversarial networks (GANs), and deep Q-nets.

A learning algorithm is a method of training a predetermined target device (eg, a GPU or NPU based neural network) using a set of training data to call, enable, or control the target device to perform a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, partially supervised learning, or reinforcement learning.

It should be understood that although terms such as "first", "second", "third" and the like may be used in this document to describe various blocks, modules, networks, elements, components, regions, layers and/or sections ., these blocks, modules, networks, elements, components, areas, layers and/or sections should not be limited by these terms. These terms are only used to distinguish one block, module, network, element, component, region, layer, or section from another block, module, network, element, component, region, layer, or section. Thus, a first block, module, network, element, component, region, layer, or section may be referred to as a second block, module, network, element, component, region, layer, or section, without departing from the scope of the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the respective listed positions. Elements mentioned in the singular do not exclude the plurality of elements, unless otherwise specified.

The functionality of an element specified in the description or claims as a single element may be practiced by means of several components of the device, and conversely, the functionality of elements indicated in the description or claims as several separate elements may be practiced by means of a single component.

In one of the embodiments, some or all elements/blocks/modules of the proposed device are located in a common housing, can be placed on the same frame/structure/printed circuit board/chip and are structurally connected to each other through assembly (assembly) operations and functionally through communication lines . The mentioned communication lines or channels, unless otherwise indicated, are typical communication lines known to specialists, the material implementation of which does not require creative efforts. The communication link may be a wire, a set of wires, a bus, a track, a wireless link (inductive, RF, infrared, ultrasonic, etc.). Communication protocols over communication lines are known to those skilled in the art and are not disclosed separately.

The functional connection of elements should be understood as a connection that ensures the correct interaction of these elements with each other and the implementation of one or another functionality of the elements. Particular examples of functional communication may be communication with the ability to exchange information, communication with the ability to transmit electric current, communication with the ability to transmit mechanical motion, communication with the ability to transmit light, sound, electromagnetic or mechanical vibrations, etc. The specific type of functional connection is determined by the nature of the interaction of the mentioned elements, and, unless otherwise indicated, is provided by well-known means, using principles well-known in the art.

The design of the elements of the proposed device is known to specialists in this field of technology and is not described separately in this document, unless otherwise indicated. The elements of the device can be made from any suitable material. These components can be manufactured using known methods including, by way of example only, machining, investment casting, crystal growth. Assembly, connection, and other operations as described herein are also within the knowledge of a person skilled in the art, and thus will not be explained in more detail here.

Although exemplary embodiments have been described in detail and shown in the accompanying drawings, it should be understood that such embodiments are illustrative only and are not intended to limit the present invention, and that the present invention should not be limited to the particular arrangements and structures shown and described, since a person skilled in the art on the basis of the information set forth in the description and the knowledge of the prior art may be obvious various other modifications and embodiments of the invention without going beyond the essence and scope of this invention.

Claims (9)

1. A device for determining the pathology of the chest organs based on the analysis of x-ray images, comprising:
   a data receiving unit configured to receive data to be processed comprising a frontal chest x-ray image from the device requesting data processing by communicating therewith;
   an image preparation unit configured to prepare, using at least one processor, an x-ray image contained in the data to be processed by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, moreover, in the process The image preparation unit is configured to automatically crop the image so that it covers only the lungs by:
   - searching the image of the right lung and the left lung separately using at least two different independent pre-trained pattern recognition methods other than neural networks,
   - formation of a set of rectangles, each of which covers one found lung and only it, based on the performed searches,
   - forming a general rectangle that includes essentially only the said plurality of rectangles, and
   - clipping from the image areas that go beyond the general rectangle;
   a pathology prediction unit configured to use said neural network to analyze the prepared image and determine the presence or absence of pathology in the image;
   a report generation unit, configured to generate, using at least one processor, a report on an examination conducted in the device, the examination report containing at least one file containing an indication of a result of processing performed in the pathology prediction unit; And
   a data transmission unit configured to transmit the report to the device requesting data processing by communicating with it.
2. The apparatus of claim. 1, wherein if it is determined that the image has a pathology, the reporting unit is further configured to:
   identifying possible areas where signs of pathologies are found;
   forming an image with the visualization of the pathology in the form of an indication of the found areas with signs of pathologies; And
   generating a study report containing an image with visualization of the pathology.
3. The device according to claim 2, in which the identification of possible areas with signs of pathologies is performed using computer vision methods that analyze the input image of the neural network and the activation of the pathology prediction block within the neural network, which were obtained by running this image through it.
4. The device according to claim 3, in which a combination of Grad-CAM methods (a weighted combination of class activation maps based on gradients) and Saliency maps (significance maps) are used as computer vision methods.
5. The device according to claim. 2, in which the indication of the found areas with signs of pathologies is made in the form of a heat map or an outline of the boundaries of the areas.
6. The device according to claim 2, in which, in order to generate an image with visualization of the pathology, the report generation unit is configured to superimpose an indication of the found areas with signs of pathologies on a copy of the input image of the neural network of the pathology prediction unit.
7. The device according to claim 2, in which the indication of the found areas with signs of pathologies is linked to the size, shape and position of the image, which was directly analyzed by the neural network;
   the image preparation unit is additionally configured to save the parameters of all transformations performed on the original image obtained in the data to be processed and associated with its resizing, displacement, rotation, cropping and cropping; And
   to form an image with visualization of the pathology, the report generation unit is configured to:
   apply transformations inverse to the above saved transformations to indicate the found areas with signs of pathologies, in order to obtain a visualization of the pathology with reference to the size, shape and position of the original x-ray image; And
   superimpose the resulting visualization of the pathology with reference to the size, shape and position of the original x-ray image on a copy of the original x-ray image.
8. The device according to claim 1, in which three pre-trained methods are used as pattern recognition methods: the Viola-Jones method using a cascade classifier based on Haar features, the directional gradient histogram method, and the method based on local binary patterns.
9. A method for determining the pathology of the chest organs based on the analysis of x-ray images, comprising the steps, in which:
   receive using the communication device to be processed data containing x-ray image of the chest in frontal projection;
   using at least one processor, an x-ray image contained in the data to be processed is prepared by performing one or more preliminary transformations on it for use by a neural network operating on the basis of at least one processor, and when preparing the image, the image is automatically cropped in such a way so that it covers only the lungs, by:
   - searching the image of the right lung and the left lung separately using at least two different independent pre-trained pattern recognition methods other than neural networks,
   - formation of a set of rectangles, each of which covers one found lung and only it, based on the performed searches,
   - forming a general rectangle that includes essentially only the said plurality of rectangles, and
   - clipping from the image areas that go beyond the general rectangle;
   using said neural network, the prepared image is analyzed and the presence or absence of pathology in the image is determined;
   generate using at least one processor report, which contains at least one file containing an indication of the presence or absence of pathology; And
   transmitting, using the communication device, a report to the device requesting data processing.

Images