JP7115114B2 - X-ray image object recognition system - Google Patents

Description

The present invention relates to an X-ray image object recognition system for recognizing an object to be imaged from an X-ray image.

In the medical field that deals with X-ray images, part recognition is performed on the original image (raw data) acquired at the time of radiography as preprocessing to obtain the final output X-ray image after radiography. The image quality of the output X-ray image is improved by correcting the data using the optimum image processing parameters for each part. For example, for parts such as humerus, hip joints, shoulder joints, ribs, and foreign bodies (metals/pacemakers), the image quality of the output X-ray image is improved by applying different gamma values, correcting gray levels, and suppressing noise. Improvements are made. Therefore, in order to improve the image quality of such X-ray images, it is necessary to recognize as accurately as possible the region whose data is to be corrected. is required.

Here, Patent Document 1 discloses an example of a method for extracting a specific part from an image. In Patent Document 1, when processing a medical image (X-ray image) containing a plurality of structures (eg, bones), the outline of a specific structure (eg, ribs) based on the anatomical position is obtained as prior information. After storing a pre-shaped model consisting of a set of in a storage unit and arranging the pre-shaped model in a medical image, at a position where a plurality of contour lines of the arranged pre-shaped model overlap with respect to the medical image, An image feature amount is calculated by performing first-order differentiation on the pixel value. Then, candidate points of a specific structure are detected based on the contour lines of the preliminary shape model and the image feature amount. As a result, it is possible to accurately extract a specific structure ((contour of ribs)) from an X-ray image in which a plurality of structures overlap.

On the other hand, in recent years, due to the development of GPUs (Graphics Processing Units) capable of arithmetic processing of large amounts of data, deep learning called deep learning has attracted attention. Deep learning is learning using a deep neural network (DNN). A DNN is a neural network (learning network) designed to perform pattern recognition with a multi-layered algorithm modeled after the neural circuits of humans and animals. By using a DNN that has been pre-trained using a large amount of data, it is possible to automatically extract features from input data and perform object recognition without human effort.

Such deep learning is applied to various techniques or fields as shown below.
(A) Speech Recognition Technology that recognizes human voices and outputs them as text data, or captures the characteristics of voices to identify the person who is speaking.
(B) Natural language processing Technology that allows computers to process and understand natural language (written and spoken) that people use on a daily basis, such as document summarization and machine translation.
(C) Anomaly detection A technology that detects signs of anomalies from time-series detection data from sensors attached to industrial equipment, such as in factory monitoring (detection of failures and abnormal operations).
(D) Image recognition Technology that recognizes and detects features such as characters, faces, and general objects from input images and videos in the fields of face recognition, autonomous driving, and emotion analysis.

In recent years, application of deep learning is progressing not only in the above-described technologies or fields, but also in the medical field. Adaptation of deep learning makes it possible, for example, to extract regions of target parts such as bones from an input X-ray image. This makes it possible to perform preprocessing such as data correction on the extracted region and improve the image quality of the finally output X-ray image.

Here, conventionally, part recognition for image quality improvement was performed by having a developer develop a detection (extraction) algorithm for a target part using its know-how and executing the algorithm by a machine (computer). However, extraction accuracy varies due to differences in X-ray imaging conditions (radiation dose, imaging position, etc.) and individual differences (shape differences in structures such as bones in the body), so complex extraction algorithms have been developed. was sought. As described above, DNN automatically extracts features from input data without human effort, so applying deep learning to target part detection eliminates the need for humans to develop complex extraction algorithms. is very effective in

However, in order to apply deep learning to target part detection, a sufficient amount of learning data is required. If it is difficult to obtain a sufficient amount of training data, DNN training and object recognition (object inference and prediction) by deep learning are possible even with a small amount of training data, but over-learning occurs and recognition performance deteriorates. more likely. In other words, if the amount of data during learning is sufficient, even if data other than learning data is input to the DNN during inference, the DNN predicts a value close to the original correct answer, as shown by the solid line graph in FIG. can do. On the other hand, if the amount of data at the time of learning is small, at the time of inference, the DNN can only predict the correct answer when the learned data is input (state of over-learning). When data is input, as indicated by the solid-line graph in FIG. 19, a value far from the original correct value (see the dashed-line graph) is predicted. As a result, object recognition performance is degraded.

Therefore, when the amount of learning data is small, it is necessary to increase the amount of learning data in order to suppress over-learning. As such processing, it is known to perform data augmentation to artificially increase the number of images by adding artificial operations such as movement, rotation, enlargement/reduction, and inversion to the original images. there is As shown in FIG. 20, the original image data (see black circles) is appropriately extended to create new image data (see white circles), and the learning data is artificially increased for learning. Excessive learning of learning data by the DNN is suppressed. As a result, at the time of inference, as shown by the solid line graph in FIG. 20, the DNN can predict a value that is close to the original correct answer for the input data, thereby suppressing deterioration in recognition performance due to over-learning. It becomes possible.

Japanese Patent Application Laid-Open No. 2018-15022 (Claim 1, paragraphs [0008], [0018], [0030] to [0056], see FIG. 1, etc.)

By the way, the X-ray images described above are data that are difficult to obtain in large quantities due to the peculiarities of the imaging apparatus, the problem of exposure to radiation, the problem of personal information, and the like. Therefore, in order to improve the recognition performance of the DNN for input X-ray images, the above-described data extension for pseudo-increasing the number of X-ray images is essential.

However, since X-ray images have many variations in acquired images due to differences in imaging target parts, imaging directions, etc., it is necessary to simply move, rotate, enlarge/reduce, or invert the original image. If data augmentation is performed by adding an operation, an image of a scene that is actually impossible is created, and based on the image, there is a possibility that the DNN will learn a shooting scene that is actually impossible. For example, when the original image is a frontal chest X-ray image and this image is rotated within the plane to create a new image for data augmentation, if the original image is rotated too much, normal radiography cannot be performed. A frontal chest X-ray image (for example, a lateral X-ray image (rotated 90° from the upright state) or an upside down X-ray image (image rotated 180° from the upright state) is acquired. If the original image is a frontal chest X-ray image of a child, data augmentation is performed by reducing the image to create a new image, and a very small frontal chest X-ray image that cannot be obtained in normal radiography is obtained. be.

When unintended learning is performed based on images created by such unintended data augmentation (see white circles in FIG. 21), DNN is input at the time of inference as shown by the solid line graph in FIG. A value deviated from a value close to the original correct answer (see the dashed line graph) is predicted for the X-ray image, resulting in deterioration of the object recognition performance. Therefore, when performing data extension of an X-ray image, it is necessary to appropriately set parameters for data extension so that unintended data extension does not occur. not considered at all.

In addition, since X-ray images have many variations due to differences in imaging conditions such as the part to be imaged and the imaging direction, for example, if a plurality of DNNs corresponding to each imaging condition are prepared, the corresponding DNN can be obtained for each imaging condition. It is also considered possible to perform inference (part recognition) using However, in this case, in order to select a DNN according to the imaging conditions from among a plurality of DNNs, it becomes necessary for the operator (radiological technologist) who performs the X-ray imaging to input the imaging conditions. Annoy me. In the field of X-ray images, inference based on input images is performed for the purpose of preprocessing for improving the image quality of output X-ray images as described above, so it is desirable that this processing be performed efficiently. Also, it is desirable that the processing load is small. Considering the above, a single DNN is prepared instead of preparing a plurality of DNNs for each imaging condition. It is desirable to be able to recognize the region to be imaged with

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and its object is to appropriately set parameters for performing data augmentation on X-ray images during learning, thereby preventing unintended data augmentation. This allows the learning network to perform appropriate machine learning using the data-augmented images so that the learning network can recognize (infer) objects with high accuracy. To provide an X-ray image object recognition system capable of recognizing an object with a single learning network even if an X-ray image photographed under such photographing conditions is input.

An X-ray image object recognition system according to one aspect of the present invention includes a learning network that performs machine learning using a learning set including an X-ray image of an object and a correct label corresponding to the object, and from the learning set, an imaging information calculation unit for calculating imaging information for deriving imaging conditions at the time of X-ray imaging of the object; and data expansion for creating a new X-ray image from the X-ray image based on the imaging information a data augmentation parameter determination unit that determines a data augmentation parameter to be used in the data augmentation parameter, performs the data augmentation based on the data augmentation parameter, and machine-learns the learning network using the acquired new X-ray image and the correct label The learning network recognizes an X-rayed object from an X-ray image input after performing machine learning using the new X-ray image, and outputs the recognition result. Output.

In the above-mentioned X-ray image object recognition system, the imaging information calculation unit combines the X-ray image included in the learning set with the correct label having a shape corresponding to the region of the object included in the X-ray image. Based on this, the photographing information may be calculated.

In the X-ray image object recognition system described above, the imaging information may be the number of pixels in the region of the object corresponding to the correct label in the X-ray image.

In the X-ray image object recognition system described above, the imaging information calculation unit may calculate the imaging information based on a rectangular area surrounding the object in the X-ray image included in the learning set.

In the X-ray image object recognition system described above, the imaging information may be the area of the rectangular region.

In the X-ray image object recognition system described above, the imaging information calculation unit further calculates, from the learning set, out-of-object region information indicating a region other than the object in the X-ray image included in the learning set. good too.

In the above-mentioned X-ray image object recognition system, the outside-object region information is information of an X-ray irradiation field, and the imaging information calculation unit includes the X-ray irradiation field region included in the learning set. The information of the irradiation field may be calculated based on the correct label of the corresponding shape.

In the X-ray image object recognition system described above, the out-of-object region information is information of an X-ray irradiation field, and the imaging information calculation unit calculates the following based on histogram information of the X-ray images included in the learning set , the information of the irradiation field may be calculated.

In the X-ray image object recognition system described above, the out-of-object region information is information of an X-ray irradiation field, and the imaging information calculation unit divides each pixel value of the X-ray image included in the learning set into two. The information of the irradiation field may be calculated based on the valued binarized image.

In the X-ray image object recognition system described above, the data augmentation parameter determination unit may determine the data augmentation parameter based on the imaging information, the extra-object region information, and a preset threshold value. .

In the above X-ray image object recognition system, the data augmentation parameter may be at least one of reduction/enlargement ratio, shift amount, and rotation angle of the X-ray image.

In the X-ray image object recognition system described above, the data augmentation parameter determination unit determines, when the total number of pixels of the object region in the X-ray image included in the learning set is equal to or greater than a first threshold, the The reduction/enlargement ratio as a data expansion parameter may be set to a value that magnifies or reduces the X-ray image.

In the X-ray image object recognition system described above, the data augmentation parameter determination unit determines a second total number of pixels in the region of the object in the X-ray image included in the learning set that is smaller than the first threshold. The reduction/enlargement ratio may be set to a value that magnifies the X-ray image if it is equal to or less than the threshold.

In the X-ray image object recognition system described above, the data augmentation parameter determination unit may determine at least one of the shift amount and the rotation angle along with the reduction/enlargement ratio.

In the X-ray image object recognition system described above, the data augmentation parameter determination unit determines, when the total number of pixels of the extra-object region in the X-ray image included in the learning set is less than a third threshold, The settable range of data extension parameters may be restricted.

In the X-ray image object recognition system described above, the object is at least an X-ray low-transmittance region in which the amount of X-rays transmitted in the person is relatively low and a high X-rays-transmittance region in which the amount of X-rays transmitted is relatively high. One may be included.

In the X-ray image object recognition system described above, the low X-ray transmission area may include a bone area of the person, and the high X-ray transmission area may include a lung area of the person.

In the above X-ray image object recognition system, the learning network may be composed of a neural network.

According to the above configuration, since the data augmentation parameter is appropriately determined based on the imaging information, the data augmentation can be appropriately performed based on the data augmentation parameter, and the intention of the X-ray image at the time of learning can be obtained. It is possible to avoid a situation in which data expansion that does not occur is performed. As a result, the learning network can be appropriately machine-learned using the data-augmented images, so that during inference, the learning network accurately recognizes the object (the part to be imaged) from the input X-ray image. (Inference) becomes possible. In addition, data augmentation is performed using data augmentation parameters determined based on imaging information, and the learning network is subjected to machine learning. Even if an image is input, it becomes possible to recognize the object. That is, at the time of inference, it is possible to recognize an object with a single learning network regardless of the imaging conditions of the input X-ray image.

FIG. 4 is an explanatory diagram schematically showing a state in which a person is photographed from the front using X-rays; FIG. 4 is an explanatory diagram showing an example of an X-ray image of the chest obtained by frontal radiography; FIG. 4 is an explanatory diagram schematically showing a state of oblique imaging of a person using X-rays; FIG. 3 is an explanatory diagram showing an example of a chest X-ray image obtained by oblique imaging; FIG. 4 is an explanatory diagram showing an example of an X-ray image of a human humerus; FIG. 4 is an explanatory diagram showing an example of an X-ray image of a person's hip joint; FIG. 10 is an explanatory diagram showing the flow of processing for generating a learning model for frontal chest images; FIG. 10 is an explanatory diagram showing the flow of processing for generating a learning model for chest oblique images; FIG. 10 is an explanatory diagram showing the flow of processing for generating a learning model for humerus images; FIG. 10 is an explanatory diagram showing a flow of processing when a learning model corresponding to an imaging condition is read from among a plurality of learning models and inference is performed; 1 is a block diagram showing a schematic configuration of an X-ray image object recognition system according to an embodiment of the present invention; FIG. 4 is a flow chart showing the flow of processing in the learning method of the learning network provided in the X-ray image object recognition system; FIG. 3 is an explanatory diagram showing an example of X-ray images included in a learning set; FIG. 11B is an explanatory diagram showing an example of a correct label created based on the X-ray image of FIG. 11A; FIG. 11C is an explanatory diagram showing both the correct label in FIG. 11B and the correct label corresponding to the irradiation field; FIG. 10 is an explanatory diagram showing an example of X-ray images included in another learning set; FIG. 13B is an explanatory diagram showing an example of a correct label created based on the X-ray image of FIG. 13A; FIG. 10 is an explanatory diagram showing an example of using a rectangular area for object recognition in an X-ray image as a correct label; FIG. 4 is an explanatory diagram schematically showing an example of a histogram of an X-ray image; FIG. 4 is an explanatory diagram showing an example of a binarized image; 4 is a flow chart showing the flow of processing during object recognition in the X-ray image object recognition system; FIG. 4 is an explanatory diagram showing the relationship between learning data and correct answers and the relationship between input data to be inferred and predicted values when there is sufficient learning data; FIG. 4 is an explanatory diagram showing the relationship between learning data and correct answers and the relationship between input data to be inferred and predicted values when there is little learning data; FIG. 10 is an explanatory diagram showing the relationship between learning data and correct answers and the relationship between input data to be inferred and predicted values when data extension is performed appropriately; FIG. 10 is an explanatory diagram showing the relationship between learning data and correct answers and the relationship between input data to be inferred and predicted values when unintended data extension is performed;

One embodiment of the present invention will be described below with reference to the drawings. First, before describing the X-ray image object recognition system of this embodiment, a supplementary description of the above-described problems will be provided.

(Supplementary information about the assignment)
Algorithms that enable object recognition by deep learning have been introduced in various papers, among which R-CNN (Regions with Convolutional Neural Networks), Fast R-CNN, Faster R-CNN, YOLO (You Only Look Once ) are well-known algorithms. These algorithms are intended for images used in international contests such as the "ImageNet Large Scale Visual Recognition Challenge (ILSVRC)," which competes for object recognition accuracy. For example, using the ILSVRC2012 data set, it is possible to randomly train 50,000 images of data in 1,000 specified object categories, and train a neural network on a total of 50 million image data. can. Therefore, a neural network that employs each of the above algorithms does not require complicated data augmentation processing. In other words, it is possible to expand the data by randomly performing processes such as movement, rotation, enlargement/reduction, and inversion on the original image.

In addition, the input image described in the paper on SegNet, a well-known method for semantic segmentation that grasps images at the pixel level, is a method that assumes images limited to in-vehicle scenes. is unnecessary. Furthermore, since the in-vehicle image is a scene that can be seen in front of the vehicle, the positions of the road surface, the sky, buildings, vehicles and people in front are limited. The same is true for U-Net, which assumes cell detection, and although the position and size of captured images of cells differ, the shape does not change significantly, so complex data expansion is not required. is unnecessary.

On the other hand, as mentioned above, X-ray images in the medical field are data that are difficult to obtain in large amounts, and data expansion is essential in order to improve the recognition performance of DNN. There are many variations in acquired images due to differences in shooting direction, etc.

For example, when X-ray imaging is performed for the purpose of diagnosing the lung field and ribs, frontal imaging shown in FIG. 1A and oblique imaging shown in FIG. 2A are performed. Here, FIG. 1B shows an example of a chest X-ray image obtained by frontal imaging, and FIG. 2B shows an example of a chest X-ray image obtained by oblique imaging.

Further, for example, when diagnosing a humerus, a femur, a hip joint, or the like, X-ray imaging is performed centering on a specific trunk. At this time, in order to reduce exposure to radiation, X-ray imaging is performed in a state in which areas other than the specific trunk, which is an imaging target, are covered with a radiation protection sheet or the like (radiation exposure suppression). For example, FIG. 3 shows an example of an X-ray image of a humerus, and FIG. 4 shows an example of an X-ray image of a hip joint. In addition, since the position of the humerus changes in various ways by rotating the arm, the humerus in particular has a high degree of freedom in X-ray imaging.

Thus, there are many variations in X-ray images. Therefore, in order to simply and accurately recognize parts of an input X-ray image, for example, learning is performed for each class (imaging target part) to generate a DNN (learning model) for each class, Based on the information (e.g., imaging target region, imaging direction) input by an operator (e.g. radiological technologist) during X-ray imaging, the learning model corresponding to the class is read (selected) and region recognition (inference) is performed. method is conceivable.

For example, when generating a learning model (learning model for frontal chest image) for recognizing a region to be imaged from an X-ray image of the frontal chest, as shown in FIG. and a correct label, data augmentation is performed on the X-ray image, and a DNN is learned using the newly generated X-ray image and its correct data, for a chest front image Generate a learning model for Similarly, when generating a learning model (learning model for chest oblique image) for recognizing a region to be imaged from a chest oblique X-ray image, as shown in FIG. Using a learning set containing the X-ray image and the correct label of, performing data extension on the X-ray image, and learning the DNN using the newly generated X-ray image and its correct data, Generate a training model for chest oblique images. When generating a learning model (humerus image learning model) for recognizing a region to be imaged from an X-ray image of the humerus, as shown in FIG. and the correct label, data augmentation is performed on the X-ray image, and the DNN is learned using the newly generated X-ray image and its correct data, for the humerus image Generate a learning model. At the time of inference, as shown in FIG. 8, a learning model corresponding to an imaging condition (class) input by the operator at the time of X-ray imaging is read from among a plurality of learning models, and the X-ray image is stored in the read learning model. data is input, the target part is inferred, and the result is output.

However, when a learning model is prepared for each shooting condition as described above, in order to select a predetermined learning model from among multiple learning models, it is necessary for the photographer to input the shooting conditions as described above. In addition to troubling the operator, the processing becomes complicated. As described above, in the field of X-ray images, the above inference (part recognition) is performed for the purpose of preprocessing for improving the image quality of output X-ray images, so it is desirable that this processing be performed efficiently. Also, it is desirable that the processing load is small. For that purpose, instead of generating a learning network (learning model) for each imaging condition, a (single) learning network is generated collectively, and for any X-ray image captured under various imaging conditions, It is desirable to be able to recognize the region to be imaged by inputting it into a single learning network.

In addition, since X-ray images captured under different imaging conditions are of various types and have many variations, the following situations may occur.
(a) The data is irregular between each X-ray image (the aspect ratio of each X-ray image is inconsistent).
(b) The angles at which the objects to be photographed are shown in each X-ray image are not uniform (due to frontal photography, oblique photography, etc.).
(c) A foreign substance (for example, a pacemaker, a bolt, a necklace, etc. implanted in the body) other than the object to be imaged appears in the X-ray image.
(d) The amount of X-ray images with correct data is uneven among a plurality of classes (for example, many frontal chest X-ray images are collected as original images, but hip joint X-ray images are difficult to collect).

Therefore, it is necessary to implement data expansion to ensure these situations, and it is necessary to add artificial operations assuming many variations. However, if data expansion is simply performed by adding artificial operations such as movement, rotation, enlargement/reduction, and inversion to the original image, since the number of X-ray images is originally small, rotation In some cases, an image that cannot be obtained by normal radiography is created, such as an overexposed chest front X-ray image or an X-ray image that is smaller than a child's front chest X-ray image. When such unintended data expansion is performed and unintended learning is performed, as shown in FIG. Unpredictable and poor object recognition performance.

Therefore, in this embodiment, for an image input during learning, shooting conditions are estimated based on the input image and the correct label, parameters for data expansion are determined based on the estimation results, and based on the determined parameters, By performing data augmentation using the 3D method, it is possible to avoid unintended data augmentation and enable highly accurate object recognition. Object recognition is possible for any X-ray image. The X-ray image object recognition system of this embodiment will be described below.

(Configuration of X-ray image object recognition system)
FIG. 9 is a block diagram showing a schematic configuration of the X-ray image object recognition system 1 of this embodiment. The X-ray image object recognition system 1 includes a storage unit 2, a communication unit 3, an overall control unit 4, a learning network 5, an imaging information calculation unit 6, a data augmentation parameter determination unit 7, and a learning processing unit 8. It has Among them, the learning network 5, the photographing information calculation unit 6, the data expansion parameter determination unit 7, and the learning processing unit 8 are configured by a GPU capable of arithmetic processing of a large amount of data. Such an X-ray image object recognition system 1 can be composed of, for example, a PC (personal computer). Note that FIG. 9 shows only the configuration directly related to the present embodiment, and omits illustration of other configurations such as an input unit (for example, a mouse or keyboard) and a display unit (for example, a liquid crystal display device). there is

Here, in the present embodiment, the term “object” refers to an object to be recognized (deduced or predicted) based on an X-ray image, and is an X-ray low-power object through which a person has a relatively small amount of X-ray transmission. It includes at least one of a transparent region and a high X-ray transparent region through which a relatively large amount of X-rays are transmitted. The low X-ray transmission region includes, for example, human bone regions (cranium, cervical vertebrae, vertebral bodies, shoulder blades, ribs, pelvis, extremities, etc.), and the high X-ray transmission region includes, for example, the lung region of the human. .

The storage unit 2 is a memory for storing various kinds of information, and is composed of, for example, a hard disk. Recording media such as a RAM (Random Access Memory), a ROM (Read Only Memory), an optical disk, a magneto-optical disk, and a non-volatile memory can also be used. It may be configured by appropriately selecting from The various types of information include a learning set of the X-ray image of the object and the correct label (details will be described later) corresponding to the object, information of the X-ray image after data augmentation and the learning set of the correct label, etc. included. The communication unit 3 is an interface for communicating with the outside, and includes an input/output port, an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like. Therefore, for example, it is possible to acquire the information of the correct label of the X-ray image and the object, which are the basis of data expansion, from the outside via the communication unit 3 and store them in the storage unit 2 . The overall control unit 4 is composed of, for example, a CPU (Central Processing Unit), and controls the operation of each unit of the X-ray image object recognition system 1 .

The learning network 5 is a learning model that performs machine learning (supervised learning) using a learning set (for example, a learning set including an X-ray image of an object and a correct label corresponding to the object) stored in the storage unit 2. be. In this embodiment, the learning network 5 is composed of a neural network. As the neural network, known networks such as R-CNN, Fast R-CNN, Faster R-CNN, FCN (Fully Convolutional Networks), SegNet, U-Net can be used. Neural networks are not limited to these.

The imaging information calculation unit 6 calculates imaging information for deriving imaging conditions for X-ray imaging of an object from the learning set. Based on the imaging information calculated by the imaging information calculator 6, the data augmentation parameter determining unit 7 determines data augmentation parameters used when performing data augmentation to create a new X-ray image from the X-ray image for learning. do. The learning processing unit 8 performs data extension based on the data extension parameter, and machine-learns the learning network 5 using the acquired new X-ray image and the correct label. Details of the photographing information calculation unit 6, the data expansion parameter determination unit 7, and the learning processing unit 8 will be described together in the following description of operations.

(Operation of X-ray image object recognition system (during learning))
Next, the operation of the X-ray image object recognition system 1 of this embodiment will be described. In this embodiment, the learning network 5 is trained using a learning set including X-ray images for learning and correct labels before object recognition (inference) for input images. FIG. 10 is a flow chart showing the flow of processing in the learning method of the learning network 5. As shown in FIG. This learning method includes a learning set preparation step (S1), an imaging information calculation step (S2), a data expansion parameter determination step (S3), a data expansion step (S4), and a machine learning step (S5). . Details of each step will be described below.

<S1; learning set preparation step>
In S1, a learning set including X-ray images for learning and correct labels is prepared. Here, the learning set is prepared by preparing the learning set in an external PC or database (server) (not shown) and transmitting the data of the learning set from the PC or the like to the X-ray image object recognition system 1. do. Note that the learning set described above may be created and prepared inside the X-ray image object recognition system 1 .

FIG. 11A shows an example of X-ray images included in the learning set, and FIG. 11B shows an example of correct labels created based on the X-ray images. For example, from the X-ray image of FIG. 11A, a third person can grasp each region of the humerus, the ribs overlapping the lung fields, and the ribs not overlapping the lung fields. Therefore, a third party can artificially manipulate the shape of each area using a predetermined figure creation software on an external PC to create a shape corresponding to the shape of each region (if the shape is the same, it is not the same). correct labels L1 to L3 for (including both very close cases) are created. The learning set information including the X-ray image and the correct labels L1 to L3 is transmitted from the PC to the X-ray image object recognition system 1 and stored in the storage unit 2. FIG. At this time, as shown in FIG. 12, the third party further creates a correct label L4 having a shape corresponding to the irradiation field, which indicates an area in the X-ray image that is not irradiated with X-rays, and adds it to the learning set. may be included. In FIGS. 11B and 12, reference character B indicates a background area (the same applies to other drawings).

FIG. 13A shows an example of X-ray images included in another learning set, and FIG. 13B shows an example of correct labels created based on the X-ray images. In this example, from the X-ray image of FIG. 13A, the correct labels L11 and L12 of the shapes corresponding to the shapes of the ribs overlapping the lung field and the ribs not overlapping the lung field contained in the X-ray image are displayed as third labels. The figure shows a case where a person created the document on a PC. The learning set information including the X-ray image and the correct labels L11 and L12 is transmitted from the PC to the X-ray image object recognition system 1 and stored in the storage unit 2 in the same manner as described above.

The above-mentioned correct labels are created (given) in advance in shapes corresponding to regions of the bones of the human body (for example, skull, cervical vertebrae, vertebral bodies, shoulder blades, ribs, pelvis, limbs, etc.). is labeled. Correct labels having shapes corresponding to characteristic structures (for example, organs such as the heart and lung fields) may be created in advance even for regions other than the above bones.

It should be noted that the correct label described above can be said to be a label created (assigned) for the purpose of segmentation of the object, since the region and shape of the object correspond to each other in the X-ray image. However, as shown in FIG. 14, a rectangular area for object recognition in the X-ray image may be used as the correct label. The figure shows an example in which a rectangular region R1 surrounding the lung field, a rectangular region R2 surrounding the heart, a rectangular region R3 surrounding the head of the humerus, and a rectangular region R4 surrounding the ribs in the X-ray image are used as correct labels. .

<S2; Photographing Information Calculation Process>
The imaging information calculation unit 6 calculates the imaging conditions for X-ray imaging based on the X-ray images included in the learning set prepared in S1 and the correct label having a shape corresponding to the region of the object included in the X-ray image. , to calculate shooting information for deriving . For example, when calculating the imaging information for the X-ray image shown in FIG. 11A, the imaging information calculator 6 calculates the number of pixels in the regions of the object corresponding to the correct labels L1 to L3 in the X-ray image. The number of pixels is a value unique to the region to be imaged and the imaging direction during X-ray imaging, and based on the number of pixels, it is possible to derive, for example, that "the humerus and ribs were imaged from the front". Therefore, the number of pixels constitutes imaging information for deriving imaging conditions during X-ray imaging. Note that the calculation of the imaging information (the number of pixels in the object region) at this time is performed for each learning set including the X-ray image for learning and the correct label.

The imaging information calculation unit 6 may also calculate imaging information based on a rectangular area surrounding an object in the X-ray images included in the learning set. For example, when calculating imaging information for the X-ray image shown in FIG. A rectangular region R2, a rectangular region R3 surrounding the head of the humerus, and a rectangular region R4 surrounding the ribs may be set, and the area (or the number of pixels) of each of the rectangular regions R1 to R4 may be calculated. The area is a value unique to the region to be imaged and the imaging direction at the time of X-ray imaging, and based on the area, it is possible to derive, for example, that "the chest was imaged from the front". Therefore, the area also constitutes imaging information for deriving imaging conditions during X-ray imaging. Note that the calculation of the imaging information (the area of the rectangular region) at this time is performed for each X-ray image for learning.

In addition, the imaging information calculation unit 6 may further calculate, from the learning set prepared in S1, out-of-object region information indicating a region other than the object in the X-ray image included in the learning set. As the above-described non-object area information, here, information on an area not irradiated with X-rays in an X-ray image, that is, information on an X-ray irradiation field can be considered. At least one of the following three methods can be adopted as a method of calculating the information of the irradiation field.

(1) As shown in FIG. 12, when a correct label L4 having a shape corresponding to the irradiation field in the X-ray image is created in advance and the correct label L4 is included in the learning set, the imaging information calculation unit 6 calculates the information of the irradiation field based on the correct label L4 included in the learning set. For example, the imaging information calculation unit 6 determines that the region corresponding to the correct label L4 in the X-ray image of the learning set is the irradiation field region, and calculates (outputs) the number of pixels in that region as the irradiation field information. )do.

(2) The imaging information calculator 6 calculates the information of the irradiation field based on the histogram information of the X-ray images included in the learning set. FIG. 15 schematically shows an example of a histogram of an X-ray image. Generally, in an X-ray image, a region through which X-rays are difficult to pass, such as a bone region, appears white, and a region through which X-rays easily pass appears black. The irradiated field appears whitest in the X-ray image because it is generated by shielding the part other than the target part so that the X-rays do not pass through in order to prevent exposure during X-ray imaging. Therefore, as shown in FIG. 15, the imaging information calculation unit 6 creates a histogram showing the relationship between the pixel value and the frequency in the X-ray image, and determines if the pixel value is equal to or greater than the threshold value Th with respect to the number of pixels in the entire X-ray image. By calculating the ratio of the total of certain frequencies, the ratio of the irradiation field area to the entire X-ray image can be calculated as the irradiation field information.

(3) The imaging information calculation unit 6 calculates information of the irradiation field based on a binarized image obtained by binarizing each pixel value of the X-ray image included in the learning set. As described in (2) above, the irradiated field appears whitest in the X-ray image. For example, if the possible range of each pixel value of an X-ray image is 0 (black) to 4095 (white), by considering 4000 as a threshold value, the pixel value from 0 to 4000 is set to "0", and the pixel value can be binarized so that 4001 to 4095 are set to "1". For example, if the above binarization processing is performed on the X-ray image of FIG. 11A, a binarized image as shown in FIG. 16 is obtained. In FIG. 16, a region T1 having a binarized pixel value of “1” corresponds to an area outside the irradiation field, and a region T2 having a binarized pixel value of “0” corresponds to the irradiation field. corresponds to the region of By binarizing each pixel value of the X-ray image in this manner, the imaging information calculation unit 6 can recognize the area T1 outside the irradiation field from the binarized image, and thereby the pixels of the area T1 can be recognized. The number can be calculated (output) as information of the irradiation field.

<S3; Data extension parameter determination step>
The data augmentation parameter determination unit 7 determines data augmentation parameters based on the imaging information acquired in S2, the non-object area information (irradiation field information), and a preset threshold value. More specifically, it is as follows. Here, as an example, the size of the X-ray image is described as 480 pixels×360 pixels, but the threshold value shown below can be adjusted as appropriate according to the image size.

First, the data augmentation parameter determination unit 7 determines that “background size (sum of pixels in unlabeled region)/image size (number of pixels in entire image)≧0.90” in the X-ray images included in the learning set. or whether "the area outside the irradiation field is 30000 pixels (third threshold) or more" is satisfied. When the above conditions are satisfied, it can be determined that the X-ray image has an area outside the irradiation field and the like, and the area occupied by the object is limited to the entire X-ray image.

Next, the data augmentation parameter determination unit 7 satisfies “the total number of pixels in the region corresponding to the correct label L1 (see FIG. 11B) indicating the humerus≧the first threshold value (for example, 15000 pixels)” in the X-ray image. decide whether to If the condition is satisfied, the humerus area occupies a fairly large proportion in the X-ray image, so it can be determined that the X-ray image is an image of an adult, that is, an image in which the scale of the object is large relative to the entire image. In this case, the data expansion parameter determination unit 7 determines that if the X-ray image is enlarged any further, it is highly likely that the image will be an image of an impossible imaging scene (not suitable for other imaging scenes). Randomly set the X-ray image reduction/enlargement ratio between 0.6 and 1 times. That is, when the total number of pixels in the object region in the X-ray image is equal to or greater than the first threshold, the data expansion parameter determination unit 7 sets the X-ray image reduction/enlargement ratio to the same size as the X-ray image. Or set it to a value to shrink.

Next, the data augmentation parameter determination unit 7 determines whether or not the X-ray image satisfies “the total number of pixels in the region corresponding to the correct label L1 indicating the humerus≦second threshold value (for example, 3000 pixels)”. to decide. If the above conditions are satisfied, the humerus is fairly small, so the X-ray image can be determined to be an image of a child, that is, an image with a small scale of the object relative to the entire image. In this case, the data expansion parameter determination unit 7 determines that if the X-ray image is made smaller than this, it is highly likely that the image will be an image of an impossible imaging scene. is set randomly between 1 and 1.4 times. That is, when the total number of pixels in the region of the object in the X-ray image is equal to or less than the second threshold, which is smaller than the first threshold, the data expansion parameter determination unit 7 sets the reduction/enlargement ratio of the X-ray image to , to a value that magnifies or magnifies the X-ray image.

On the other hand, if none of the above conditions are satisfied, that is, if "the second threshold < the total number of pixels in the region corresponding to the correct label L1 indicating the humerus < the first threshold" in the X-ray image, the data The extension parameter determination unit 7 determines that the image is unlikely to be an image of an impossible imaging scene (suitable for other imaging scenes) regardless of whether the original X-ray image is enlarged or reduced, and data extension is performed. A reduction/enlargement ratio of the X-ray image as a parameter is randomly set between 0.8 and 1.2 times.

Next, the data expansion parameter determining unit 7 determines other data expansion parameters according to the scale setting (reduction/enlargement ratio setting). For example, when the reduction/enlargement ratio of the X-ray image is set to 0.6 to 1, the data expansion parameter determination unit 7 sets the shift amount of the X-ray image within a range of ±40 pixels in the vertical, horizontal, and diagonal directions. It is set randomly, and the rotation angle of the X-ray image is set randomly within a range of ±6°. In addition, when the X-ray image reduction/enlargement ratio is set to 1 to 1.4 times, the data expansion parameter determination unit 7 sets the shift amount of the X-ray image within a range of ±20 pixels in the vertical, horizontal, and diagonal directions. It is set randomly, and the rotation angle of the X-ray image is set randomly within a range of ±2°. Furthermore, when the X-ray image reduction/enlargement ratio is set to 0.8 to 1.2 times, the data expansion parameter determination unit 7 sets the shift amount of the X-ray image to ±30 pixels in the vertical, horizontal, and diagonal directions. The range is set randomly, and the rotation angle of the X-ray image is set randomly within the range of ±4°.

The setting of the shift amount and the like may be performed within a fixed range according to the scale setting (in conjunction with the scale information as it is), but it may be variable according to the actual scale setting information. may For example, when the X-ray image reduction/enlargement ratio is set to A times between 1 and 1.4 times, the data expansion parameter determination unit 7 sets the shift amount of the X-ray image to ±40 in the vertical, horizontal, and diagonal directions. The ratio of the scale may be reflected in the shift amount, etc. by randomly setting the number of pixels within the range of ×A and setting the rotation angle of the X-ray image randomly within the range of ±6° ×A. .

In addition, in the present embodiment, the data augmentation parameter determination unit 7 determines that, when the total number of pixels in the non-object region (region outside the irradiation field) in the X-ray image is less than 30000 pixels (third threshold), the X-ray Determine that the image is a frontal image of the chest. Since many frontal images of the chest are likely to be collected for both adults and children, there is little need to expand the range of data expansion. Therefore, the data extension parameter determination unit 7 limits the settable range of the data extension parameters. That is, the data expansion parameter determining unit 7 sets the reduction/enlargement ratio, shift amount, and rotation angle to values that do not expand data, or sets values that cause only a small amount of data expansion. For example, the data expansion parameter determination unit 7 randomly sets the reduction/enlargement ratio in the range of 0.9 to 1.1 times, and the shift amount of the X-ray image is set in the range of ±10 pixels in the vertical, horizontal, and diagonal directions. , and the rotation angle of the X-ray image is set randomly within a range of ±1°.

<S4; data extension step>
The learning processing unit 8 performs data extension to create new X-ray images from the X-ray images included in the learning set based on the data extension parameters determined in S3. Types of data expansion include Horizontal Flip (horizontal flip), Vertical Flip (vertical flip), Crop (cutting out randomly from one image), Scale (Crop while changing the scale), Rotation (image Rotate), Cutout (increase generalization ability by masking part of the image), Shift (change image position), etc. Here, data expansion of Scale, Rotation, and Shift is performed based on the data expansion parameters determined in S3. That is, data extension (Scale, Rotation, Shift) is performed with the determined reduction/enlargement ratio, rotation angle, and shift amount. Data extensions other than Scale, Rotation, and Shift may be performed as required.

<S5; machine learning step>
The learning processing unit 8 machine-learns the learning network 5 using the new X-ray image and the correct label obtained by the data extension in S4. As a learning algorithm for the learning network 5, a general error backpropagation method (backpropagation) can be used. The error backpropagation method uses a value (likelihood (score)) output from the final layer of the learning network 5 for an input of an image (pixel value) to the learning network 5 and a value (likelihood (score) )), the weight (connection weight) of each node (unit) constituting the learning network 5 is sequentially applied from the final layer side to the input layer side using the steepest descent method so that the squared error between )) is minimized. It is a method of change. Through such machine learning, a learned learning network 5 (learning model) is obtained.

When a weighted learning network such as SegNet is used as the learning network 5, the weight (contribution rate) of each class (region) is changed for each input image (new data-extended X-ray image). can be For example, when an X-ray image includes a humerus region and a background region, if the area of the humerus region is much smaller than the area of the background region, the network is learned by being pulled by the background region. As a result, there is a risk that the accuracy of recognizing the humerus region in the network after training may decrease. By setting the weight of each class for each input image during learning as described above, it is possible to suppress a decrease in the recognition accuracy of each region after learning.

By the way, in the SegNet paper (A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, https://arxiv.org/pdf/1511.00561.pdf), the weight of each class (class balancing) is calculated from all images used during training However, this embodiment differs from the method of the above paper in that the weight of each class is calculated for each input image rather than for all images.

(Operation of X-ray image object recognition system (during inference))
When the learning network 5 is machine-learned as described above, the learning network 5 can be used to recognize an object included in an input image (object inference and prediction). FIG. 17 is a flow chart showing the flow of processing during object recognition in the X-ray image object recognition system 1 .

When an X-ray image of an object to be recognized is input to the input layer of the learning network 5 (S11; X-ray image input step), the learning network 5 extracts X-rays from the input X-ray image. The photographed object is recognized (S12; inference step), and the recognition result is output (S13; output step).

When the data augmentation parameter is determined based on the shooting information as in the present embodiment, data augmentation is performed based on the determined data augmentation parameter, and the learning network 5 undergoes machine learning, the IoU (Intersection over Union) value is 70. % or more. Note that the IoU is an index representing the degree of overlap between the correct region and the predicted region, and the larger the IoU value, the closer the prediction is to the correct answer, and the higher the identification performance. On the other hand, the IoU value was 65% when the data augmentation parameter was not determined based on the imaging information and the data augmentation was performed randomly and the learning network was machine-learned. Therefore, according to the method of this embodiment, it can be said that the accuracy of recognizing an object included in an X-ray image is improved.

In S11, the information of the data augmentation parameter acquired during learning may be used to change the size of the input X-ray image according to the data augmentation parameter. In this case, it is possible to make the input X-ray image closer to the size at the time of data expansion, and to perform object recognition with higher accuracy.

(effect)
As described above, according to the X-ray image object recognition system 1 of the present embodiment, the data augmentation parameter determination unit 7 determines data augmentation parameters based on the imaging information calculated by the imaging information calculation unit 6 (S2 , S3). Since the data augmentation parameters are appropriately determined in consideration of the imaging conditions at the time of X-ray imaging of the object, the learning processing unit 8 performs data augmentation based on the data augmentation parameters (S4). A simulated image (an upside-down frontal chest X-ray image) that would not be possible with normal radiography (for example, an X-ray image of the frontal chest taken in an upright position) is created. It is possible to avoid situations where unintended data extension is performed. Therefore, the learning processing unit 8 machine-learns the learning network 5 using the new X-ray image acquired by appropriate data extension and the correct label (S5). Even if an X-ray image other than learning data is input during recognition (recognition), it is possible to appropriately predict the X-rayed object from the input X-ray image, and the object can be accurately recognized (deduced). ) (S11 to S13).

In addition, the data augmentation parameter is determined based on the shooting information of the object, the data augmentation is performed based on the determined data augmentation parameter, and the learning network is machine-learned using the image after the data augmentation. , the single learning network 5 can handle X-ray images captured under various imaging conditions. That is, the object can be recognized by the same learning network 5 regardless of the X-ray image captured under any imaging conditions. Therefore, as in the case of preparing a learning network for each shooting condition and performing inference, it is possible to input the shooting conditions by the photographer in order to select the learning network according to the shooting conditions from among multiple learning networks. can be made unnecessary. This leads to improvement in processing efficiency when object recognition is performed as preprocessing for improving the image quality of an X-ray image to be output, enabling rapid preprocessing.

Further, the imaging information calculation unit 6 calculates imaging information based on the X-ray images included in the learning set and the correct label of the shape corresponding to the region of the object included in the X-ray image (S2). In this manner, the above-described effects of the present embodiment can be obtained in the configuration in which the imaging information calculation unit 6 calculates imaging information using the X-ray image and the correct label.

In particular, the imaging information is the number of pixels in the region of the object corresponding to the correct label in the X-ray image. Since the number of pixels reflects the imaging region and imaging direction at the time of X-ray imaging, it can be effectively used as imaging information for deriving the imaging conditions at the time of X-ray imaging.

At this time, the imaging information calculation unit 6 may calculate the imaging information based on the rectangular area surrounding the object in the X-ray images included in the learning set (S2). In this manner, even with the configuration in which the imaging information calculation unit 6 calculates the imaging information based on the rectangular area, the above-described effects of the present embodiment can be obtained.

In particular, the imaging information is the area of the rectangular area. Since the area of the rectangular region reflects the imaging site and imaging direction at the time of X-ray imaging, it can be effectively used as imaging information for deriving the imaging conditions at the time of X-ray imaging.

Further, the imaging information calculation unit 6 may further calculate, from the learning set, out-of-object region information indicating a region other than the object in the X-ray image included in the learning set (S2). In this case, the data augmentation parameter determining unit 7 can determine the data augmentation parameters further taking into account the extra-object area information, so that the data augmentation parameters are accurately determined so that unintended data augmentation is not performed. becomes possible.

Here, the out-of-object region information may be information on the X-ray irradiation field. Then, the imaging information calculation unit 6 calculates the information of the irradiation field (for example, the number of pixels ) may be calculated. When the learning data includes the correct label, the information of the irradiation field can be reliably obtained based on the correct label.

Further, the imaging information calculation unit 6 may calculate the information of the irradiation field based on the histogram information of the X-ray images included in the learning set (see FIG. 15). Since the irradiation field appears white in the X-ray image, the information of the irradiation field (for example, the ratio of the irradiation field area to the entire image area) can be reliably obtained based on the histogram information.

Further, the imaging information calculation unit 6 may calculate the information of the irradiation field based on a binarized image obtained by binarizing each pixel value of the X-ray image included in the learning set (see FIG. 16). Since the irradiated field appears white in the X-ray image, information (for example, the number of pixels) of the irradiated field can be reliably obtained based on the binarized image.

The data augmentation parameter determination unit 7 also determines data augmentation parameters based on the imaging information, the non-object area information, and a preset threshold value (S3). In this case, the data extension parameter can be appropriately determined from the three types of information, ie, the imaging information, the non-object area information, and the threshold.

Here, the data expansion parameter may be at least one of the X-ray image reduction/enlargement ratio, shift amount, and rotation angle. If these parameters are not properly set, there is a high possibility that unintended data expansion will be performed and an unintended image will be created. In this embodiment, the data expansion parameter determination unit 7 can appropriately set the data expansion parameters based on the shooting information, so the data expansion parameters to be set include at least one of the reduction/enlargement ratio, shift amount, and rotation angle. This makes it possible to reliably perform appropriate data extension. In other words, it is possible to reliably avoid unintended data expansion that creates an unintended image.

Further, the data augmentation parameter determining unit 7 sets the reduction/enlargement rate as the data augmentation parameter to X The line image is set to the same size or reduced value (S3). If the total number of pixels of the object is greater than or equal to the first threshold, further enlargement of the X-ray image will likely result in an image of an impossible imaging scene. Therefore, by setting the reduction/magnification ratio of the X-ray image to the same size or a value that reduces the X-ray image, it is possible to reliably avoid the situation where an image of an impossible imaging scene is created by data expansion. can.

Further, the data augmentation parameter determination unit 7 sets the reduction/enlargement rate when the total number of pixels in the region of the object in the X-ray image included in the learning set is equal to or less than a second threshold that is smaller than the first threshold. , is set to a value that magnifies or magnifies the X-ray image (S3). If the total number of pixels of the object is less than or equal to the second threshold, further reduction of the X-ray image will likely result in an image of an impossible imaging scene. Therefore, by setting the reduction/enlargement ratio of the X-ray image to a value that magnifies or equalizes the X-ray image, it is possible to reliably avoid the situation where an image of an impossible imaging scene is created by data expansion. can.

In addition, the data expansion parameter determination unit 7 determines at least one of the shift amount and the rotation angle along with the reduction/enlargement ratio (S3). By determining the shift amount and/or rotation angle together with the reduction/enlargement ratio, various data extensions can be performed on the X-ray images for learning to increase the number of images.

Further, the data augmentation parameter determination unit 7 limits the settable range of the data augmentation parameter when the total number of pixels in the extra-object region in the X-ray image included in the learning set is less than the third threshold (S3 ). If the total number of pixels in the extra-object region is less than the third threshold, the X-ray image is considered to be a frontal image of the chest that tends to gather in large numbers. In addition, even if data expansion is performed, it is desirable to perform data expansion by a very small amount in order to avoid data expansion that would result in an improbable shooting scene. By limiting the settable range of the data expansion parameter as described above, the data expansion parameter can be set to a value for which no data expansion is performed or a value for which only a small amount of data expansion is performed. As a result, it is possible to avoid unnecessary data expansion, or to avoid data expansion that would result in an impossible shooting scene.

Further, the above-described object includes at least one of a low X-ray transmission region in which the amount of X-ray transmission in a person is relatively low and a high X-ray transmission region in which the amount of X-ray transmission is relatively high. By subjecting the learning network 5 to machine learning by performing the data augmentation of the present embodiment on an X-ray image obtained by X-raying such an object, the object can be accurately recognized at the time of inference.

The low X-ray transmission region includes a bone region of a person, and the high X-ray transmission region includes a lung field region of a person. Therefore, even if the object includes a human bone region and a lung region, the data augmentation of the present embodiment is performed on the X-ray image obtained by X-raying such an object so that the learning network 5 can be used by the machine. By learning, the object can be accurately recognized at the time of inference.

Also, the learning network 5 is composed of a neural network. As a result, the learning network 5 can perform machine learning to improve object recognition accuracy.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention can be used in a system for recognizing an object to be imaged from an X-ray image.

REFERENCE SIGNS LIST 1 X-ray image object recognition system 5 learning network 6 imaging information calculation unit 7 data augmentation parameter determination unit 8 learning processing unit

Claims (15)

a learning network that performs machine learning using a learning set that includes an X-ray image of an object and a correct label corresponding to the object;
an imaging information calculation unit that calculates imaging information for deriving imaging conditions for X-ray imaging of the object from the learning set;
a data augmentation parameter determination unit that determines a data augmentation parameter to be used when performing data augmentation for creating a new X-ray image from the X-ray image, based on the imaging information;
a learning processing unit that performs the data extension based on the data extension parameter and machine-learns the learning network using the acquired new X-ray image and the correct label,
The learning network recognizes an X-rayed object from an input X-ray image after performing machine learning using the new X-ray image, and outputs the recognition result,
The imaging information calculation unit calculates the imaging information based on the X-ray image included in the learning set and the correct label having a shape corresponding to the region of the object included in the X-ray image,
An X-ray image object recognition system , wherein the photographing information is the number of pixels in a region of the object corresponding to the correct label in the X-ray image. a learning network that performs machine learning using a learning set that includes an X-ray image of an object and a correct label corresponding to the object;
an imaging information calculation unit that calculates imaging information for deriving imaging conditions for X-ray imaging of the object from the learning set;
a data augmentation parameter determination unit that determines a data augmentation parameter to be used when performing data augmentation for creating a new X-ray image from the X-ray image, based on the imaging information;
a learning processing unit that performs the data extension based on the data extension parameter and machine-learns the learning network using the acquired new X-ray image and the correct label,
The learning network recognizes an X-rayed object from an input X-ray image after performing machine learning using the new X-ray image, and outputs the recognition result,
The imaging information calculation unit calculates the imaging information based on a rectangular area surrounding the object in the X-ray image included in the learning set;
An X-ray image object recognition system, wherein the imaging information is the area of the rectangular region. 3. The imaging information calculation unit further calculates, from the learning set, out-of-object region information indicating a region other than the object in the X-ray image included in the learning set. An X-ray image object recognition system as described. The outside-object area information is information on an X-ray irradiation field,
4. The method according to claim 3, wherein the imaging information calculation unit calculates the information of the irradiation field based on a correct label having a shape corresponding to the region of the irradiation field of the X-ray, which is included in the learning set. An X-ray image object recognition system as described. The outside-object area information is information on an X-ray irradiation field,
4. The X-ray image object recognition system according to claim 3, wherein said imaging information calculation unit calculates information of said irradiation field based on histogram information of said X-ray image included in said learning set. The outside-object area information is information on an X-ray irradiation field,
4. The imaging information calculation unit calculates the information of the irradiation field based on a binarized image obtained by binarizing each pixel value of the X-ray image included in the learning set. X-ray image object recognition system according to . 7. The data augmentation parameter determination unit according to any one of claims 3 to 6, wherein the data augmentation parameter determination unit determines the data augmentation parameter based on the imaging information, the non-object area information, and a preset threshold value. X-ray image object recognition system according to . 8. The X-ray image object recognition system according to claim 7, wherein said data augmentation parameter is at least one of reduction/enlargement ratio, shift amount, and rotation angle of said X-ray image. The data augmentation parameter determination unit determines the reduction/enlargement ratio as the data augmentation parameter when the total number of pixels in the region of the object in the X-ray image included in the learning set is equal to or greater than a first threshold. is set to a value that magnifies or reduces the X-ray image. The data augmentation parameter determination unit performs the reduction/ 10. The X-ray image object recognition system according to claim 9, wherein the magnification is set to a value that magnifies or magnifies the X-ray image. 11. The X-ray image object recognition system according to claim 9, wherein the data augmentation parameter determination unit determines at least one of the shift amount and the rotation angle along with the reduction/enlargement ratio. The data augmentation parameter determination unit limits the settable range of the data augmentation parameter when the total number of pixels in the extra-object region in the X-ray image included in the learning set is less than a third threshold. The X-ray image object recognition system according to any one of claims 8 to 11, characterized in that: 3. The object includes at least one of a low X-ray transmissive region in a person through which the amount of X-rays penetrating is relatively low and a high X-ray transmissive region in which the amount of X-rays penetrating is relatively large. 13. The X-ray image object recognition system according to any one of 1 to 12. 14. The X-ray image object recognition according to claim 13, wherein the low X-ray transmission area includes a bone area of the person, and the high X-ray transmission area includes a lung area of the person. system. 15. The X-ray image object recognition system according to any one of claims 1 to 14, wherein said learning network comprises a neural network.

Images