JP2023154994A - Image processing device, method for operating image processing device, and program - Google Patents

Abstract

To acquire a bone density analysis area of a lumbar spine accurately.SOLUTION: An image processing device includes: first acquisition means for acquiring a spinal region and an intervertebral region in an input image; and second acquisition means for acquiring a bone density analysis area, which is an object region to measure bone density of a lumbar spine, using the acquired spinal region and the acquired intervertebral region.SELECTED DRAWING: Figure 3

Description

The disclosure herein relates to an image processing device, an operating method for the image processing device, and a program.

Bone density is measured for diagnosing osteoporosis and determining the effectiveness of therapeutic drugs. Bone density is the amount of bone per 1 ^cm2 . As a technique for measuring bone density, a method using images obtained by DXA (Dual energy X-ray Absorptiometric) method is known. In the DXA method, X-ray photography is performed using two types of tube voltages, and materials such as bone and soft tissue are discriminated based on the difference in absorption rate. The image acquired using the DXA method in this manner is called an energy subtraction image (energy subtraction image, material discrimination image).

In order to measure bone density, it is necessary to further acquire a target area for bone density measurement (hereinafter referred to as a bone density analysis area) from among the bone areas of the ENE sub-image. When measuring the bone density of the lumbar vertebrae, generally the bone density analysis area is from the first lumbar vertebrae to the fourth lumbar vertebrae, or from the second lumbar vertebrae to the fourth lumbar vertebrae.

Patent Document 1 discloses a technique for automatically acquiring a bone density analysis region using a machine learning model.

JP2021-37164A

In Patent Document 1, a bone density analysis region within the femur is obtained by machine learning. However, since the spine is a series of vertebral bodies and similar structures between vertebrae, when segmenting the lumbar vertebrae using a machine learning model, the area to be acquired may be omitted, and the bone density analysis area There is a risk that the images may be obtained with a vertical shift.

Therefore, one of the purposes of the disclosure of this specification is to obtain a bone density analysis region of the lumbar vertebrae with high accuracy. Another objective of the disclosure of this specification is to obtain with high precision an analysis region in which similar structures are consecutively connected.

An image processing device according to an embodiment disclosed herein includes a first acquisition unit that acquires a spinal region and an intervertebral region in an input image, and the acquired spinal region and the acquired intervertebral region. and second acquisition means for acquiring a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae.

According to at least one embodiment of the disclosure of this specification, it is possible to obtain a bone density analysis region of the lumbar vertebrae with high accuracy. Further, according to at least other embodiments disclosed in this specification, an analysis region in which similar structures are consecutively connected can be acquired with high accuracy.

1 shows an example of a schematic configuration of an X-ray imaging system according to a first embodiment. An example of an input image according to Example 1 is shown. An example of a flowchart of image processing according to the first embodiment is shown. An example of an acquisition result by the first acquisition unit according to Example 1 is shown. 4 shows an example of an intervertebral region correction processing target according to the first embodiment. Another example of the intervertebral region correction processing target according to the first embodiment is shown. An example of a reference area according to Example 1 is shown. An example of a rotation correction method according to the first embodiment is shown. An example of a straight line generation method according to the first embodiment is shown. An example of a background area according to Example 1 is shown. An example of a flowchart of image processing according to the second embodiment is shown. An example of a spine image according to Example 2 is shown. An example of a flowchart of image processing according to the third embodiment is shown. An example of an acquisition result by the first acquisition unit according to Example 3 is shown. An example of a flowchart of image processing according to Example 4 is shown. An example of an acquisition result by the first acquisition unit according to Example 4 is shown. An example of a schematic configuration of a radiation imaging system according to Example 5 is shown. 12 is a diagram for explaining an example of radiographic processing according to Example 5. FIG. 12 is a diagram for explaining an example of radiographic processing according to Example 5. FIG. FIG. 7 is a diagram for explaining acquisition targets and correction targets according to Example 5;

Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numbers are used between the drawings to indicate identical or functionally similar elements. Further, in each drawing, some components, members, and processes that are not important for the explanation may be omitted from illustration.

Note that in the following embodiments, an imaging system using X-rays as an example of radiation will be described, but the imaging system according to the present disclosure may use other radiation. Here, the term radiation can include, in addition to X-rays, for example, alpha rays, beta rays, gamma rays, particle rays, cosmic rays, and the like.

Note that in the following, a machine learning model refers to a learning model using a machine learning algorithm. Specific algorithms for machine learning include the nearest neighbor method, the Naive Bayes method, decision trees, and support vector machines. Another example is deep learning, which uses neural networks to generate feature quantities and connection weighting coefficients for learning by itself. Further, as an algorithm using a decision tree, there are also methods using gradient boosting such as LightGBM and XGBoost. As appropriate, any of the above algorithms that can be used can be applied to the following embodiments and modifications. Further, the teaching data refers to learning data, and is composed of a pair of input data and output data.

Note that a trained model is a machine learning model that follows any machine learning algorithm such as deep learning, and has been trained (learning) in advance using appropriate teaching data (learning data). . However, although a trained model is obtained in advance using appropriate training data, it does not mean that no further training is performed, and additional training can be performed. Additional learning can also occur after the device is installed at the site of use.

(Example 1)
EMBODIMENT OF THE INVENTION Hereinafter, with reference to FIG. 1 thru|or FIG. 10, the image processing apparatus and the operating method of an image processing apparatus based on Example 1 of this invention are demonstrated. FIG. 1 is a diagram showing a configuration example of an X-ray imaging system 100, which is an example of a radiographic system according to the present embodiment. The X-ray imaging system 100 includes an X-ray generator control unit 102, an X-ray tube 103, an FPD (Flat Panel Detector) 106, and an image processing device 120. Note that the configuration of the X-ray imaging system 100 is also simply referred to as an X-ray imaging device. The image processing device 120 processes information based on captured X-ray images.

When the exposure switch is pressed, the X-ray generator control unit 102 applies a high voltage pulse to the X-ray tube 103 to generate X-rays, and the X-ray tube 103 emits X-rays. When X-rays are emitted from the X-ray tube 103, the X-rays pass through the subject (not shown) on the pedestal 104 and the grid 105, and reach the FPD 106. The arriving X-rays are converted into visible light by the FPD 106 and output as an X-ray image. The output X-ray image is transferred to the image processing device 120 via the control unit 101.

The FPD 106 includes an X-ray detection section (not shown) including a pixel array for generating a signal according to X-rays. The X-ray detection section detects the X-rays that have passed through the grid as an image signal. In the X-ray detection section, pixels that output signals according to incident light are arranged in an array (two-dimensional area). The photoelectric conversion element of each pixel converts the X-rays converted into visible light by the phosphor into electrical signals and outputs them as image signals. In this way, the X-ray detection section is configured to detect X-rays that have passed through the grid and obtain an image signal (X-ray image). The FPD 106 outputs the read image signal (X-ray image) to the control unit 101 according to instructions from the control unit 101.

The control unit 101 transfers the X-ray image acquired from the FPD 106 to the image processing device 120 that processes the X-ray image and the storage unit 109 that stores the results of image processing and various programs. The storage unit 109 is configured of, for example, ROM (Read Only Memory) or RAM (Random Access Memory). The storage unit 109 can store images output from the control unit 101, images processed by the image processing device 120, and calculation results calculated by the image processing device 120.

The image processing device 120 processes the input image and outputs calculation results, images, and the like. The image processing device 120 is provided with a first acquisition section 121 and a second acquisition section 122. The first acquisition unit 121 extracts and acquires a spinal region and an intervertebral region from the input image using a learned model described later. The second acquisition unit 122 uses the vertebral region and intervertebral region acquired by the first acquisition unit 121 to acquire a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae. Details of the processing by the first acquisition unit 121 and the second acquisition unit 122 will be described later. Furthermore, the image processing device 120 can calculate the bone density by performing bone density analysis on the acquired bone density analysis region.

Note that the control unit 101 and the image processing device 120 can be configured by a computer equipped with a processor and a memory. Note that the control unit 101 and the image processing device 120 may be configured by a general computer, or may be configured by a computer dedicated to the X-ray imaging system.

Each component of the control unit 101 and the first acquisition unit 121 included in the image processing device 120 includes, for example, a processor such as one or more CPUs (Central Processing Units), and a storage unit 109. Functions are configured using the loaded program. Note that the processor may be, for example, an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or an FPGA (Field-Programmable Gate Array). Furthermore, the components of the control unit 101 and each component included in the image processing device 120 may be configured with an integrated circuit such as an ASIC that performs a specific function. Furthermore, the internal configuration of the control unit 101 and the image processing device 120 may include a graphic control unit such as a GPU, a communication unit such as a network card, and an input/output control unit such as a keyboard, display, or touch panel.

The display unit 107 (monitor) displays an X-ray image (digital image) received by the control unit 101 from the FPD 106, an image processed by the image processing device 120, patient information, user notification, etc. The operation unit 108 can input instructions to the image processing device 120 and the FPD 106 via the user interface. The operation unit 108 may be configured using an input device such as a mouse or a keyboard, for example. The control unit 101 functions as an example of a display control unit that can control the display unit 107 in response to instructions from the operation unit 108 and the like.

In this embodiment, the image processing device 120 and the control unit 101 are described as separate devices, but they can also be configured by the same device, for example, one computer. Note that the display section 107 may be configured with a touch panel display, and in this case, the display section 107 can also be used as the operation section 108.

FIG. 2 shows an example of an image near the lumbar vertebrae acquired by the X-ray imaging system 100. Furthermore, for the sake of explanation, FIG. 2 shows a first lumbar vertebra L1, a second lumbar vertebra L2, a third lumbar vertebra L3, and a fourth lumbar vertebra L4, which are examples of bone density measurement targets. The image processing device 120 in this embodiment extracts a bone density analysis region using an image near the lumbar vertebrae as shown in FIG. 2 as an input image.

The input image may be an energy subtraction image (energy sub-image) acquired by the DXA method. This energy sub-image may be generated from images taken by X-rays at two types of tube voltages using the kV switching method. In addition, this energy sub image is generated from images of each layer obtained by one exposure using a multilayer sensor made of two types of materials (phosphors) with different X-ray absorption rates as the FPD 106. It may be. Furthermore, a simple X-ray image obtained by one exposure (simple X-ray photography) may be used as the input image. Further, when the FPD 106 is a laminated sensor, an image on the upper layer side (on the subject side) may be used as the input image. Note that a plurality of images related to different energies obtained by the DXA method may be used as the input image.

Hereinafter, with reference to FIG. 3, image processing by the image processing device 120 for extracting a bone density analysis region in this embodiment will be described. FIG. 3 shows an image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When the image processing flow according to this embodiment is started, the process moves to step S11.

In step S11, the first acquisition unit 121 of the image processing device 120 uses the trained model to extract (segmentation) the spinal region and the intervertebral region from the input image. The spine region referred to here refers to a region including the outline and interior of the spine. Alternatively, only the outline (outer frame) of the spine may be treated as the spine region. Furthermore, the intervertebral region referred to herein refers to a region including the outline and interior of the intervertebral region. Alternatively, only the contour (outer frame) between the vertebrae may be treated as the intervertebral region. Additionally, the intervertebral region may be a line. In this way, the first acquisition unit 121 is an example of a first acquisition unit that acquires the spinal region and the intervertebral region in the input image using the trained model. In the following, it is assumed that the acquisition process of various regions such as the spinal region, the intervertebral region, and the bone density analysis region includes the acquisition of images showing these regions and the identification of these regions.

An example of the extracted (obtained) results is shown in FIGS. 4(a) and 4(b). The white region in FIG. 4(a) is the extraction result (obtained result) of the spinal region, and the white region in FIG. 4(b) is the extraction result of the intervertebral region. Although FIG. 4A shows an example in which the spinous process is not included as a spinal region, the spinous process may be included.

The trained model may be stored in the storage unit 109, and the first acquisition unit 121 may call and use the trained model stored in the storage unit 109. A trained model for extracting the spinal region and a trained model for extracting the intervertebral region may be provided separately. These trained models may have the same structure and differ only in weight. Further, the trained model for extracting the spinal region and the trained model for extracting the intervertebral region may be a common model, and the extraction results of the spinal region and the intervertebral region may be output from each channel.

The trained model here can be generated by, for example, learning a combination of an image near the lumbar vertebrae and an image in which the spinal region and intervertebral region within the image are labeled as training data. Note that if a trained model for extracting the spinal region and a trained model for extracting the intervertebral region are separately provided, the labeled images may be trained separately. Specifically, a combination of an image near the lumbar vertebrae and an image labeled with the spinal region within that image, and a combination of an image near the lumbar vertebrae and an image labeled with the intervertebral region within that image, respectively. Each trained model can be generated by learning as teacher data.

Furthermore, the image processing device 120 may perform preprocessing on the input image before the segmentation process. An example of the processing may be processing to improve contrast. As a process for improving contrast, for example, a process of linearly extending the maximum brightness value in an image to 1 and the minimum brightness value in the image to 0 can be cited. Alternatively, processing such as histogram flattening may be used. Another example of pre-processing may be processing to emphasize edges. An example of processing for emphasizing edges is filter processing. Another example of pre-processing may be standardization processing in which the average value of the brightness values of the image is set to 0 and the variance is set to 1. Further, the image processing device 120 may perform a combination of the above pre-processing. Note that these pre-processings may also be performed on teacher data to be trained by a trained model.

The input image may be an image obtained by trimming an image acquired by the X-ray imaging system 100 using the operation unit 108 while viewing the image displayed on the display unit 107 by the operator. Further, the trimmed image may be resized to a specified image size. The designated image size here may be, for example, the input image size of the trained model. Further, the input image may be an image obtained by trimming a wider area than the trimming area selected by the operator. For example, if the operator selects a trimming region to include the second lumbar vertebra L2 to the fourth lumbar vertebra L4, the input image is further expanded in the vertical direction so that the fifth lumbar vertebra L5 is included in the input image. It may also be an image. Therefore, as an example, the image processing device 120 may generate the input image so as to include a region obtained by vertically extending the region trimmed by the operator by 1/4 times in the direction of the fifth lumbar vertebra L5. Further, the input image may be an image that is automatically trimmed by the image processing device 120 based on the imaging range so as to leave the vicinity of the lumbar vertebrae. Note that in these cases, similarly cropped images can be used as input data for the teacher data.

Furthermore, if there is a region extracted as an intervertebral region outside the extracted spinal region, the first acquisition unit 121 may consider this region as a segmentation error and remove it from the intervertebral region extraction result. .

Next, in step S12, the second acquisition unit 122 of the image processing device 120 extracts a bone density analysis region from the extraction results of step S11 (FIGS. 4(a) and 4(b)). Here, a case will be described in which the region of the second to fourth lumbar vertebrae L2 to L4 is extracted as the bone density analysis region. Note that the combination of bone density analysis regions is not limited to this, and for example, the region of the first lumbar vertebra L1 may also be extracted.

Here, as mentioned above, since the spine is a series of vertebral bodies and similar structures between the vertebrae, if the lumbar vertebrae are segmented using a trained model that performs processing according to the learning tendency. In some cases, an area to be acquired may be omitted or an extra area may be acquired. FIG. 5A shows an example of an extraction result of an intervertebral region when an intervertebral region to be acquired is missing and an extra region is acquired. When a bone density analysis region is set based on the intervertebral region shown in Fig. 5(a), some areas may be extracted between the intervertebral regions due to missing intervertebral regions or extra acquired regions. The lumbar vertebrae area (bone density analysis area) shifts from its original position.

On the other hand, the second acquisition unit 122 according to the present embodiment performs extraction using a trained model using the relationship between the spinal region and the intervertebral regions included in the spinal region and the relationship between the intervertebral regions. Performs correction processing for the intervertebral region. The second acquisition unit 122 can extract the bone density analysis region with high accuracy by extracting the bone density analysis region based on the corrected intervertebral region.

First, the second acquisition unit 122 calculates the center of gravity position of each intervertebral region extracted in step S11. Next, the second acquisition unit 122 calculates the vertical position (center of gravity Y coordinate) of these center of gravity positions. Note that in the following, the vertical coordinate in the image will be the Y coordinate, and the horizontal position will be the X coordinate. Furthermore, any known method may be used to calculate the center of gravity. In FIG. 5A, the Y coordinate of the center of gravity is shown by a dotted line for an example of the extraction result of the intervertebral region described above. Based on the interval between these Y coordinates of the center of gravity, the second acquisition unit 122 performs the correction process described below as necessary.

As exemplified by the coordinates H1 and H2 in FIG. One of the intervertebral regions may be regarded as an outlier (or an inference error) and excluded from the intervertebral regions. As an example, when the median value of the interval between the Y coordinates of the center of gravity of each intervertebral region is ΔYm, the threshold value set here may be, for example, ΔYm/4. However, the method for setting the threshold value is not limited to this, and may be set by any method. As an example of the exclusion method here, of the two corresponding intervertebral regions, the intervertebral region with the larger area (H1 side in Fig. 5(a)) is left, and the intervertebral region with the smaller area (the H1 side in Fig. 5(a)) is left. 5(a) on the coordinate H2 side) may be excluded. Alternatively, the upper intervertebral region may be excluded while leaving the lower intervertebral region in the image. Conversely, the upper intervertebral region may be left in place and the lower intervertebral region may be excluded.

In addition, the second acquisition unit 122 detects an error when the interval between the Y-coordinates of the centroids of the intervertebral regions is larger than a set threshold, as exemplified by the coordinate H3 and the coordinate H4 in FIG. 5(a). A new intervertebral region may be added at a position that is the midpoint of their centers of gravity, considering the values (or inference errors). As an example, when the median value of the interval between the Y coordinates of the center of gravity of each intervertebral region is ΔYm, the threshold value set here may be, for example, ΔYm×2. However, the method for setting the threshold value is not limited to this, and may be set by any method.

For the intervertebral region added at this time, as shown in the Y coordinate position of coordinate H3.5 in FIG. It may be a line segment with a width of 10 mm and a height of 1 mm. However, the size and shape of the intervertebral region are not limited to this, and can be set to any size.

Another example of correction processing will be explained using FIGS. 6(a) and 6(b). The second acquisition unit 122 acquires the second acquisition unit 122 from the upper end of the extracted spinal region (coordinate H1 in FIG. 6(a)), in other words, from the upper end of the spinal region to the inside of the image. The distance in the vertical direction to the Y coordinate of the center of gravity of the intervertebral region (coordinate H2 in FIG. 6(b)) is calculated. If this distance is larger than the threshold, the second acquisition unit 122 acquires the height of the midpoint between the upper end of the spinal region and the center of gravity of the first intervertebral region counting from the upper end of the spinal region to the inside of the image. An intervertebral region may be added at the position (coordinate H1.5 in FIG. 6(b)).

As an example, the threshold value set here may be ΔYm×2, where ΔYm is the median value of the interval between the Y coordinates of the center of gravity of each intervertebral region. In addition, the intervertebral region to be added here is, for example, as shown at the Y coordinate position of coordinate H1.5 in FIG. It may be a line segment with a width of 10 mm and a height of 1 mm, with the center of gravity positioned at the midpoint between the center of gravity and the center of gravity. Note that rows refer to the horizontal arrangement of images, and columns refer to the vertical arrangement of images. However, the method for setting the threshold value and the intervertebral region to be added is not limited to this, and may be set using any method. Furthermore, the second acquisition unit 122 may simply set only the center of gravity position of the intervertebral region, without setting the intervertebral region to be added as a line segment. Note that if the spinal region or intervertebral region extracted in step S12 is tilted, the second acquisition unit 122 tilts and sets the intervertebral region to be added in accordance with the tilt of the spinal region or intervertebral region. It's fine.

Similarly, from the lower end of the extracted spinal region (coordinate H4 in FIG. 6(a)), the center of gravity Y coordinate of the first intervertebral region counting inward from the lower end of the spinal region (FIG. 6(b)) The vertical distance to the coordinate H3) is calculated. If this distance is larger than the threshold, the center point of the lower end of the spinal region and the center of gravity of the first intervertebral region counting inward from the lower end of the spinal region is set at the coordinate H3.5 in FIG. 6(b). An intervertebral region may be added as shown at the Y coordinate location of .

Regarding the intervertebral regions processed in this way and their centers of gravity, the second acquisition unit 122 obtains the center of gravity G1 of the fourth intervertebral region counting from the bottom (from the pelvis side) of the image, as shown in FIG. Set. Similarly, set the center of gravity G2 of the third intervertebral region counting from the bottom, the center of gravity G3 of the second intervertebral region counting from the bottom, and the center of gravity G4 of the first intervertebral region counting from the bottom, respectively. .

Next, the second acquisition unit 122 sets horizontal lines LR1, LR2, LR3, and LR4 passing through the centers of gravity G1, G2, G3, and G4 (four dotted lines in FIG. 7), respectively.

If the number of extracted intervertebral regions is less than four (predetermined number), the second acquisition unit 122 draws four lines that vertically divide the extracted spinal region into five equal parts. I can do it. The second acquisition unit 122 may extend these four lines to both the left and right ends of the image and set them as horizontal lines LR1, LR2, LR3, and LR4 in order from the top of the image. Through this process, the second acquisition unit 122 can divide the spinal region into four (predetermined number). In addition, when this process is performed, after the bone density analysis area is extracted in the following process, the control unit 101 displays a message indicating that the bone density analysis area may be out of alignment and prompting the operator to manually correct it. It may also be displayed on the display unit 107.

As shown in FIG. 7, the second acquisition unit 122 sets reference regions SR1, SR2, and SR3 in the horizontal direction that are sandwiched between these horizontal lines and have a vertical width.

Before performing this process, if the extraction result of the spinal region in step S12 is tilted as shown in FIG. 8(a), the second acquisition unit 122 may perform rotation correction on the entire image. As an example of a method for setting an angle for correcting the rotation of an image, the second acquisition unit 122 determines the horizontal center of gravity position (Fig. 8 Find the black dot in (b). Thereafter, the second acquisition unit 122 rotates the image so that the approximate straight line (dotted line in FIG. 8(b)) of the obtained center of gravity position becomes a line in the vertical direction of the image as shown in FIG. 8(c). It may be corrected. Note that the rotational correction may be performed before the intervertebral region correction processing, or may be performed before the setting processing of the centers of gravity G1 to G4, etc.

Furthermore, instead of performing image rotation correction, the second acquisition unit 122 sets lines LR1, LR2, LR3, LR4 and reference regions SR1, SR2, SR3 obliquely based on the detected inclination of the spinal region. It's okay.

Furthermore, if the intervertebral region extracted in step S12 is inclined, the second acquisition unit 122 may draw the lines LR1, LR2, LR3, and LR4 diagonally. An example of how to draw a line at this time will be explained using FIG. 9. FIG. 9 shows an example of an extracted intervertebral region 901, which is tilted. An example of how to draw a line for this intervertebral region 901 is to take the vertical midpoint in each row or thinned out rows of the intervertebral region 901 (black dots in FIG. This is the method of drawing a dotted line).

Another example of how to draw a straight line at this time is to draw an approximate straight line of the spinal region in the vicinity of the intervertebral region as shown in FIG. It may also be a method of subtracting. Furthermore, the second acquisition unit 122 may set reference regions SR1, SR2, and SR3 that are sandwiched between the lines LR1, LR2, LR3, and LR4, have a vertical width, and are parallel to these lines. Further, the second acquisition unit 122 may perform rotation correction on the image based on the slope of the approximate straight line here.

The second acquisition unit 122 extracts and acquires a spinal region (corresponding to a vertebral body) extracted in the reference region SR1 as a second lumbar vertebra L2, which is a bone density analysis region. Similarly, the spinal region within the reference region SR2 is extracted as the third lumbar vertebra L3, which is the bone density analysis region, and the vertebrae region within the reference region SR3 is extracted as the fourth lumbar vertebra L4, which is the bone density analysis region.

In this way, the second acquisition unit 122 can extract and acquire the bone density analysis region of the lumbar vertebrae with high precision by combining the extraction results of the spinal region and the intervertebral region. That is, the second acquisition unit 122 is an example of a second acquisition unit that acquires a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae based on the acquisition result of the first acquisition unit 121.

Next, in step S13, the image processing device 120 sets a background area for calculating the bone transmittance. An example of a method for setting the background area will be described below.

The background region includes the second lumbar vertebrae L2, the third lumbar vertebrae, and the second lumbar vertebra L2, the third lumbar vertebrae within the reference regions SR1, SR2, and SR3 set in step S12, as shown as background regions BG11, BG12, BG21, BG22, BG31, and BG32 in FIG. 10, for example. It is set on the left and right sides of the lumbar vertebra L3 and the fourth lumbar vertebra L4. An example of a method for searching these background areas will be described below.

The vertical widths of the background regions BG11, BG12, BG21, BG22, BG31, and BG32 can be set to the same value as the vertical widths of the respective reference regions SR1, SR2, and SR3. The widths of the background areas BG11, BG12, BG21, BG22, BG31, and BG32 can be specified values set in advance. As an example, this width may be specified as 20 mm. The designated value of the width may be stored in advance in the storage unit 109. Alternatively, it may be specified by the operator using the operation unit 108.

The position of the background region BG11 may be automatically selected in the region to the left of the second lumbar vertebra L2 in the reference region SR1 such that the variance of the luminance values within the background region is minimized. Similarly, the position of the background region BG12 may be automatically selected in the region to the right of the second lumbar vertebra L2 within the reference region SR1 such that the variance of the luminance values within the background region is minimized. Further, similarly, the background regions BG21 and BG22 may be selected in the left and right regions of the third lumbar vertebra L3, and the background regions BG31 and BG32 may be selected in the left and right regions of the fourth lumbar vertebra L4.

At this time, the region of the spinous process may be excluded from being selected as the background region. For example, when searching for the background region BG11, the image processing device 120 may exclude from the search range the range designated as the spinous process region to the left from the region extracted as the second lumbar vertebra L2. For example, a range of 15 mm may be designated as the spinous process region. When searching the background regions BG12, BG21, BG22, BG31, and BG32, the left and right spinous process regions of the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4 may be similarly excluded.

The method for setting the background area is not limited to this, and may be set using any method. For example, the operator may specify the background area using the operation unit 108.

The control unit 101 may cause the display unit 107 to display each of the bone density analysis region and the background region extracted as described above. Further, based on the bone density analysis area and the background area displayed on the display unit 107, the operator may use the operation unit 108 to modify and determine the bone density analysis area and the background area. Similarly, the center of gravity and straight lines that appear in the above calculation process may be corrected and determined by the operator using the operation unit 108. Furthermore, the background area does not need to be displayed on the display unit 107.

With reference to the bone density analysis region and the background region, the image processing device 120 calculates the bone density in each bone density analysis region. Bone density can be calculated using a known method, for example, using information on the thickness of bone and soft tissue in an image obtained by the DXA method. Alternatively, bone density may be calculated using information on the atomic number and areal density obtained by the DXA method. The control unit 101 may display the calculated bone density on the display unit 107.

Further, the control unit 101 may cause the display unit 107 to display any value useful for diagnosis based on the calculated bone density. For example, the control unit 101 may display on the display unit 107 a value compared with the average value of bone density of the same age group or other age groups. Further, the control unit 101 may display, for example, time-series data compared with past measurement results on the display unit 107. Further, the control unit 101 may cause the display unit 107 to display, for example, the average value of the bone density at the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4 as the final bone density measurement value.

Further, the control unit 101 may control the display unit 107 to display the acquisition result or acquisition process of the bone density analysis region to the operator. Here, the acquisition process includes each process by the second acquisition unit 122 described above, and in the display of the acquisition process, the area, center of gravity, line, etc. acquired and set in each process may be displayed. For example, a reference region, a background region, etc. when extracting the acquired spinal region, intervertebral region, bone density analysis region, etc. may be displayed. Further, the operation unit 108 can function as an example of an operation unit for the operator to operate while viewing the acquisition result or acquisition process by the second acquisition unit 122.

(Example 2)
Next, Example 2 of the present invention will be described. The configuration example of the X-ray imaging system according to the present embodiment is the same as the configuration example of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. Note that the configuration example of the X-ray imaging system according to the present embodiment will be described using the same reference numerals as the configuration example of the X-ray imaging system 100 according to the first embodiment, and a description thereof will be omitted. Similar to the first embodiment, the image processing device 120 in this embodiment uses an image as shown in FIG. 2 as an input image and extracts a bone density analysis region. This example will be described below, focusing on the differences from Example 1.

FIG. 11 shows an image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When the image processing according to this embodiment is started, the process moves to step S21.

In step S21, the first acquisition unit 121 uses the trained model to extract (segmentation) a spinal region from the input image. An example of the extracted results is shown in FIG. 4(a) similarly to Example 1. Note that the trained model here can be generated by, for example, learning a combination of an image near the lumbar vertebrae and an image in which the spinal region within the image is labeled as training data.

In step S22, the image processing device 120 generates a spine image in which the region resulting from the extraction in step S21 is extracted from the input image. An example of the spine image here is shown in FIG. This process can be generated, for example, by multiplying the input image with a mask in which the area extracted in step S21 is set to 1 and the other areas are set to 0.

In step S23, the first acquisition unit 121 uses the trained model to extract (segmentation) an intervertebral region from the spinal image. An example of the extracted results is the same as that shown in FIG. 4(b). In other words, the image processing device 120 according to this embodiment extracts the intervertebral region not from the entire image but from inside the spine. Note that the trained model here can be generated by, for example, learning a combination of a spine image and an image in which an intervertebral region within the image is labeled as training data.

In step S24, the second acquisition unit 122 extracts a bone density analysis region of the lumbar vertebrae in the same manner as in Example 1 by combining the extraction results of the extracted spinal region and the extraction result of the intervertebral region. I can do it. Even in this case, by combining the extraction results of the spinal region and the intervertebral region, it is possible to extract and obtain the bone density analysis region of the lumbar vertebrae with high precision. In addition, by acquiring the intervertebral region from spinal images, the trained model has a narrower area for processing using image features, which reduces the amount of calculations, speeding up processing and reducing processing load. can be expected.

In step S25, the image processing device 120 extracts a background area. The method for extracting the background area is the same as step S13 of the first embodiment.

In the bone density analysis regions and background regions extracted as described above, the image processing device 120 according to the present embodiment calculates the bone density in each bone density analysis region as in the first embodiment.

(Example 3)
Next, Example 3 of the present invention will be described. The configuration example of the X-ray imaging system according to the present embodiment is the same as the configuration example of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. Note that the configuration example of the X-ray imaging system according to the present embodiment will be described using the same reference numerals as the configuration example of the X-ray imaging system 100 according to the first embodiment, and a description thereof will be omitted. Similar to the first embodiment, the image processing device 120 in this embodiment uses an image as shown in FIG. 2 as an input image and extracts a bone density analysis region. This example will be described below, focusing on the differences from Example 1.

FIG. 13 shows an image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When the image processing according to this embodiment is started, the process moves to step S31.

In step S31, the first acquisition unit 121 uses the learned model to extract (segmentation) the spinal region, intervertebral region, and pelvis region from the input image. Examples of the extracted results are shown in FIGS. 14(a) to 14(c). FIG. 14(a) shows an example of the extraction result of the spinal region, FIG. 14(b) shows an example of the extraction result of the intervertebral region, and FIG. 14(c) shows an example of the extraction result of the pelvis region.

The learned model may be stored in the storage unit 109 and then called and used. A trained model for extracting the spinal region, a trained model for extracting the intervertebral region, and a trained model for extracting the pelvic region may be provided separately. These trained models may have the same structure and differ only in weight. Further, the learned model here may be a common model, and the extraction results of the spinal region, intervertebral region, and pelvic region may be output from each channel.

The trained model here can be generated by, for example, learning a combination of an image near the lumbar vertebrae and an image in which the spinal region, intervertebral region, and pelvic region are labeled as training data. Note that if you have separate trained models for extracting the spinal region, intervertebral region, and pelvic region, the labeled images must be trained separately. Bye. Specifically, by learning a combination of an image near the lumbar vertebrae and an image in which the pelvic region is labeled as training data, a trained model for extracting the pelvic region can be generated. Note that the trained model for extracting the spinal region and the trained model for extracting the intervertebral region can be generated in the same manner as in the first embodiment.

The second acquisition unit 122 may perform correction processing on the extraction results based on these extraction results. An example of the extraction result correction process may be a process of extending the lower end of the vertebral region to reach the pelvic region when the extracted vertebral region and pelvic region are separated by a predetermined distance or more. As an example of the predetermined distance set here, it may be set to 10 mm, for example.

An example of the extraction result correction processing may be a process of shortening the lower end of the extracted spinal region to the upper boundary line of the pelvic region when the lower end of the extracted spinal region is inside the pelvic region.

An example of the extraction result correction process is a process in which the control unit 101 displays a warning message on the display unit 107 when the extracted intervertebral region and pelvic region are separated by a predetermined distance or more. May include. The message calling for attention may be, for example, a message informing the user that the bone density analysis region may be out of alignment, that the intervertebral region may not be extracted, or prompting the operator to make manual corrections. As an example of the predetermined distance set here, it may be set to 10 mm, for example.

Further, the second acquisition unit 122 can determine that the extracted intervertebral region and the pelvic region are separated by a predetermined distance or more, and that there is a possibility that the intervertebral region is not extracted. In this case, the second acquisition unit 122 may add a new intervertebral region at an intermediate position between the first intervertebral region counting upward from the pelvic region and the upper boundary line of the pelvic region. . The intervertebral region added at this time may be a line segment with a width of 10 mm and a height of 1 mm, similar to the example shown in FIG. 5(b). Also, an example of the center of gravity position of the intervertebral region added at this time is the center of gravity position of the first intervertebral region counting upward from the pelvic region, and the horizontal position of the upper boundary line of the pelvic region. (X coordinate) may be the midpoint between the point that is the X coordinate of the center of gravity position of the pelvic region.

An example of the extraction result correction processing may be a process of removing the extracted intervertebral region when the extracted intervertebral region is inside the pelvic region.

As described above, by referring to the extraction results of the pelvic region, it is possible to reduce the influence of segmentation errors in the spinal region and the intervertebral region.

Next, in step S32, the second acquisition unit 122 extracts a bone density analysis region of the lumbar vertebrae in the same manner as in Example 1 by combining the extraction results of the extracted spinal region and the extraction result of the intervertebral region. can do. Even in this case, by combining the extraction results of the spinal region and the intervertebral region, it is possible to extract and obtain the bone density analysis region of the lumbar vertebrae with high precision.

In step S33, the image processing device 120 extracts a background area. The method for extracting the background area is the same as step S13 of the first embodiment.

In the bone density analysis regions and background regions extracted as described above, the image processing device 120 according to the present embodiment calculates the bone density in each bone density analysis region as in the first embodiment.

(Example 4)
Next, Example 4 of the present invention will be described. The configuration example of the X-ray imaging system according to the present embodiment is the same as the configuration example of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. Note that the configuration example of the X-ray imaging system according to the present embodiment will be described using the same reference numerals as the configuration example of the X-ray imaging system 100 according to the first embodiment, and a description thereof will be omitted. Similar to the first embodiment, the image processing device 120 in this embodiment uses an image as shown in FIG. 2 as an input image and extracts a bone density analysis region. This example will be described below, focusing on the differences from Example 1.

FIG. 15 shows an image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When the image processing according to this embodiment is started, the process moves to step S41.

In step S41, the first acquisition unit 121 uses the learned model to extract (segmentation) a spinal region, an intervertebral region, and a rib region from the input image. Examples of the extracted results are shown in FIGS. 16(a) to 16(c). FIG. 16(a) shows an example of the extraction result of the spinal region, FIG. 16(b) shows an example of the extraction result of the intervertebral region, and FIG. 16(c) shows an example of the extraction result of the rib region.

The learned model may be stored in the storage unit 109 and then called and used. A trained model for extracting the spinal region, a trained model for extracting the intervertebral region, and a trained model for extracting the rib region may be provided separately. These trained models may have the same structure and differ only in weight. Further, the learned model here may be a common model, and the extraction results of the spinal region, intervertebral region, and rib region may be output from each channel.

The trained model here can be generated by, for example, learning a combination of an image near the lumbar vertebrae and an image in which the spinal region, intervertebral region, and rib region are labeled as training data. . Note that if you have separate trained models for extracting the spinal region, intervertebral region, and rib region, train the labeled images separately. Bye. Specifically, by learning a combination of an image near the lumbar vertebrae and an image in which the rib region within the image is labeled as training data, a trained model for extracting the rib region can be generated. Note that the trained model for extracting the spinal region and the trained model for extracting the intervertebral region can be generated in the same manner as in the first embodiment.

Furthermore, the sternum region may be extracted instead of or in addition to the rib region. In this case, for example, an image with the sternum region labeled without labeling the pelvic region may be used as training data, or an image with the spinal region, intervertebral region, pelvic region, and sternum region labeled. do it.

Next, in step S42, the second acquisition unit 122 extracts a bone density analysis region from the extraction result of step S41. Here, a case will be described in which the second to fourth lumbar vertebrae L2 to L4 are extracted as bone density analysis regions. Note that the combination of bone density analysis regions is not limited to this, and for example, the first lumbar vertebra L1 may also be extracted.

The second acquisition unit 122 extracts regions (vertebral bodies) separated by intervertebral spaces from the extracted spinal region. At this time, the second acquisition unit 122 regards the vertebral bodies in contact with the rib region as thoracic vertebrae. Then, as shown in FIG. 2, the vertebral body one below the thoracic vertebrae is extracted as the first lumbar vertebra L1. Furthermore, the vertebral bodies below the first lumbar vertebra L1 are sequentially extracted as the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4.

Further, as in Example 1, the second acquisition unit 122 sets corresponding reference regions SR1, SR2, and SR3 from the intervertebral regions above and below the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4. You may. Furthermore, as in Example 1, the second acquisition unit 122 selects the spinal region within these reference regions as the final bone density analysis region (second lumbar vertebra L2, third lumbar vertebra L3, fourth lumbar vertebra L4). May be set.

Further, as in the first embodiment, the second acquisition unit 122 acquires the second to fourth vertebral bodies counting from the bottom (from the pelvis side), the fourth lumbar vertebra L4, the third lumbar vertebra L3, and It may be extracted as the second lumbar vertebra L2. At this time, the second acquisition unit 122 may perform the correction process by extracting the fifth vertebral body counting from the bottom as the first lumbar vertebra L1 and comparing this with the region extracted as the ribs. .

An example of the correction processing here is that when the first lumbar vertebrae L1 and the rib region are in contact with each other, the control unit 101 detects the possibility that the extraction results of the bone density analysis region are deviated, and sends a message prompting the operator to make manual corrections. may include processing for displaying on the display unit 107.

In addition, an example of the correction processing here is that when the vertebral body one above the first lumbar vertebra L1 and the rib region are not in contact, the extraction results of the bone density analysis region may be deviated, and the operator The process may include a process in which the control unit 101 causes the display unit 107 to display a message prompting manual correction.

Further, when the vertical position of the upper end of the extracted spinal region is lower than the lower vertical position by a predetermined distance from the lower end of the extracted rib region, the image processing device 120 A correction process may be performed to extend the upper end a predetermined distance from the lower end of this rib region to a lower vertical position. As an example of the predetermined distance here, it may be set to 5 mm, for example.

As described above, by referring to the rib extraction results, it is possible to reduce the influence of segmentation errors in the spinal region and intervertebral region on lumbar vertebrae extraction. Furthermore, by combining the extraction results of the spinal region and the intervertebral region, it is possible to extract and obtain the bone density analysis region of the lumbar vertebrae with high precision.

In step S43, the image processing device 120 extracts a background area. The method for extracting the background area is the same as step S13 of the first embodiment.

In the bone density analysis regions and background regions extracted as described above, the image processing device 120 according to the present embodiment calculates the bone density in each bone density analysis region as in the first embodiment.

(Modified example)
Below, a modification of at least one of Example 1, Example 2, Example 3, and Example 4 of the present invention will be described. The image processing device 120 may extract a region of an object that is neither bone nor soft tissue, such as metal, as an implant region. The implant region may be extracted by segmentation using a trained model by machine learning, or by rule-based segmentation based on the variance of brightness values. The extracted implant region may be excluded from the analysis target during bone density analysis. The trained model can be generated by training, for example, a combination of an image near the lumbar vertebrae and an image labeled with a region of an object that is neither bone nor soft tissue, such as metal, as training data. can be done.

In addition, by combining Example 3 and Example 4, in addition to the spinal region and intervertebral region, both the pelvic region and rib region were extracted, and by combining these results, the extraction accuracy of the bone density analysis region was improved. It's okay. For example, if the vertical position of the lower end of the extracted spinal region is higher than the upper end of the pelvic region by a threshold value or more, the second acquisition unit 122 determines the vertical position of the lower end of the spinal region from the pelvic region. It may extend to the vertical position of the upper edge of. Further, when the vertical position of the upper end of the extracted spinal region is lower than the lower end of the rib region by a threshold value or more, the second acquisition unit 122 determines the vertical position of the upper end of the spinal region from the rib region. It may extend to the vertical position of the bottom edge of the area. As an example of the threshold value mentioned here, it can be set to 10 mm. However, the value of the threshold value is not limited to this, and any value can be set.

Further, the second acquisition unit 122 vertically divides the lumbar region set in this way into six equal parts, and divides the fourth region from the bottom (from the pelvis side) into the second lumbar vertebra L2, and the third region from the bottom into the second lumbar vertebrae L2. The region may be set as the third lumbar vertebra L3, and the second region from the bottom may be set as the fourth lumbar vertebra L4.

Furthermore, the first acquisition unit 121 may acquire, as the outer frame region, a fusion of at least one of the pelvic region and the rib region to the spinal region. Furthermore, the second acquisition unit 122 may extract the bone density analysis region by using the outer frame region instead of the spinal region and combining it with the extraction result of the intervertebral region in the same manner as above.

In addition, the image processing device 120 extracts a bone density analysis region using a total of three images: two images related to different radiation energies acquired by the DXA method and an energy sub-image generated from these images. You may. For example, when extracting a spinal region and an intervertebral region using a trained model (such as step S11 in the first embodiment), the first acquisition unit 121 extracts a spinal region and an intervertebral region from three images. May be extracted. In this case, the second acquisition unit 122 may use only the regions in which the extraction results overlap as the final spinal region and intervertebral region. Further, the second acquisition unit 122 may use only the regions extracted as the spinal region and the intervertebral region from two or more of the three images as the final spinal region and the intervertebral region.

At this time, the X-ray imaging system 100 may be individually equipped with trained models for the three images. In this case, images corresponding to the types of each of the three images (first and second images related to different radiation energies, and energy sub-images generated from these) are learned as input data for each teacher data. Thus, each trained model can be generated. Note that as the output data of the teacher data, images in which spinal regions and intervertebral regions are labeled for each type of image may be used. Furthermore, each input channel of one trained model may correspond to these three images. Similarly, the image processing device 120 may extract the spinal region and the intervertebral region using two of these three images.

By performing the above-described correction processing, the influence of segmentation errors can be reduced.

Note that in Examples 1 to 4 described above, the image processing device 120 uses the learned model to acquire a spinal region and an intervertebral region from an image near the lumbar vertebrae, which is an input image, and The bone density analysis area was obtained using However, the image processing according to the present disclosure can also be applied to other parts.

The omission of regions and the acquisition of extra regions that occur in the segmentation using the trained model described above are considered to be due to the fact that similar structures are consecutively connected in the region to be acquired. Therefore, for example, a similar situation may occur when region extraction is performed on a region where cervical vertebrae, thoracic vertebrae, or other similar structures are connected in series. Even in such cases, by applying the image processing according to the present disclosure, it is possible to correct missing regions and acquired extra regions, and finally acquire the analysis region with high precision. be able to.

As in Examples 1 to 4 described above, the correction process includes the relationship between the acquired first area and a wider second area that includes the first area, and the relationship between the acquired first area and the second area that includes the first area, and the relationship between the first area and the second area that includes the first area. The first region may be corrected using the relationship between the regions. For example, the obtained first region is a segmentation result using a trained model using the distance between the centroids of the first regions or the distance between the centroid of the first region and the edge of the second region. All you have to do is correct it.

Furthermore, in Examples 1 to 4 described above, the lumbar region located between the intervertebral regions was set as the region (analysis region) to be finally acquired. On the other hand, the analysis region to be finally acquired may be a region acquired by segmentation using a trained model by the first acquisition unit 121 and corrected by the second acquisition unit 122.

(Example 5)
Example 5 of the present disclosure will be described below with reference to FIGS. 17 to 19. In Examples 1 to 4, the technology of the present disclosure was applied to an X-ray imaging system that takes medical images. In contrast, in this embodiment, the technology of the present disclosure is applied to a radiation imaging system used in in-line automatic inspection.

In in-line automatic inspection, a technique that uses tomographic images reconstructed from a plurality of projection images taken with X-rays is widely used. For example, in in-line automatic inspection, it is desirable to carry out the inspection with the object to be inspected (for example, a board having a flat plate shape) brought close to the X-ray source for magnified imaging. On the other hand, when X-rays are irradiated with the X-ray source close to the thickness direction of the substrate, it is difficult for the X-rays to penetrate in the width direction of the substrate, which has long dimensions compared to the thickness. There may be cases where desired test results are not obtained. In connection with this, a technique (for example, a technique called oblique CT, lamino CT, or planar CT) has been proposed in which the object to be inspected is irradiated with X-rays in an oblique direction. With technology that irradiates the object with X-rays in an oblique direction, it is possible to bring the object closer to the X-ray source, making it easier to adjust the imaging magnification and making the size of the inspection device more compact. is possible.

Here, a plurality of projection images taken by irradiating X-rays in an oblique direction on such an inspection object and a tomographic image reconstructed from the projection images are used as input to the trained model to calculate solder. It is possible to obtain a target area for analyzing and detecting the presence of substances such as substances. However, even in such cases, similar structures such as solder are connected continuously, so when segmentation is performed using a machine learning model, the area to be acquired may be omitted, resulting in the analysis area being There is a risk that the data may be acquired out of alignment.

Therefore, one of the purposes of this embodiment is to obtain with high precision an analysis region in which similar structures are consecutively connected. In relation to this, in this example, a trained model is applied to a plurality of projection images taken by irradiating the inspection object with radiation in an oblique direction and a tomographic image obtained by reconstructing the projection images. Perform segmentation using In addition, through segmentation, we can obtain a solder area (first area) and a wider chip area (second area) that includes the solder area, and examine the relationship between the solder area and the chip area and the relationships between areas within the solder area. Using the relationship, the solder area obtained by segmentation is corrected.

First, with reference to FIG. 17, the configuration of the radiographic system according to this embodiment will be described. FIG. 17 shows an example of the overall configuration of the radiographic system according to this embodiment. The radiation imaging system 1700 is provided with a control unit 1710, a radiation generating device 1701, a stage 1706, a radiation imaging device 1704, a robot arm 1705, and an imaging device support portion 1703. Note that the configurations of the radiation generating device 1701 and the radiation imaging device 1704 may be the same as the configurations of the X-ray tube 103 and FPD 106 according to the first embodiment, and a description thereof will be omitted.

Here, the radiation imaging apparatus 1704 is supported by an imaging apparatus support part 1703, and is configured to be movable as the imaging apparatus support part 1703 and the robot arm 1705 move. Furthermore, an object to be inspected (hereinafter also referred to as "work 1702") is placed on stage 1706. The stage 1706 is configured to move to a position designated for radiography or to stop at a predetermined position designated for radiography according to a control signal from a stage control unit 1718 of a control unit 1710. ing.

The object to be inspected can include, for example, various articles and human bodies. When various articles (for example, a board) are to be inspected, the present invention can be applied to determining the quality of electronic components mounted on the board, analysis processing inside the inspected object, and the like. When the human body is the subject of examination, it is applicable to image diagnosis using tomography.

The control unit 1710 includes a first acquisition unit 1711, a generation unit 1712, a second acquisition unit 1713, a third acquisition unit 1714, a display control unit 1715, a storage unit 1716, an imaging device control unit 1717, and a stage control unit 1718. , and a radiation control section 1719 are provided. The first acquisition unit 1711 can acquire images captured by the radiation imaging apparatus 1704 and images generated by the generation unit 1712. Further, the first acquisition unit 1711 can also acquire various images from an external device (not shown) connected to the control unit 1710 via a network such as the Internet.

The generation unit 1712 can reconstruct a three-dimensional image from the plurality of radiation images (projection images) acquired by the acquisition unit. Furthermore, the generation unit 1712 can reconstruct a tomographic image of an arbitrary cross section from the generated three-dimensional image. Note that details of the processing by the generation unit 1712 will be described later.

The second acquisition unit 1713 uses the learned model to obtain the tomographic image reconstructed by the generation unit 1712 from the input image, similarly to the first acquisition unit 121 of the image processing device 120 according to the first embodiment. Extract and obtain the chip area and solder area. The third acquisition unit 1714 uses the chip area and the solder area acquired by the second acquisition unit 1713 to acquire an analysis area that is a target area for analyzing the presence or absence of solder. Details of the processing by the second acquisition unit 1713 and the third acquisition unit 1714 will be described later. In the following, it is assumed that the acquisition process of the solder area, chip area, and analysis area includes acquiring images showing these areas and specifying these areas.

Note that in this embodiment, the second acquisition section 1713 and the third acquisition section 1714 are implemented as functional configurations within the control section 1710. However, as in the first embodiment, a device (image processing device) separate from the control section 1710 may be provided, and the second acquisition section 1713 and the third acquisition section 1714 may be configured in the separate device. .

Note that the control unit 1710 can analyze the presence of solder in the acquired analysis region and determine whether the electronic component is properly mounted or not. Furthermore, when an image processing device is provided separately, the image processing device can also perform the analysis processing. Note that any known method may be used to analyze the presence of solder.

Display control section 1715 controls the display on display section 1720. The display control unit 1715, for example, displays various images generated by the generation unit 1712, each area acquired by the second acquisition unit 1713, analysis results of the analysis area acquired by the third acquisition unit 1714, and patient information. etc. can be displayed on the display section 1720. The storage unit 1716 can function similarly to the storage unit 109 according to the first embodiment.

Here, the control unit 1710 can be configured by a computer equipped with a processor and a memory. Note that the control unit 1710 may be configured by a general computer or a computer dedicated to the radiation control system. Further, the control unit 1710 may be, for example, a personal computer, such as a desktop PC, a notebook PC, a tablet PC (portable information terminal), or the like. Further, the control unit 1710 may be configured as a cloud-type computer in which some components are placed in an external device.

Further, each component of the control unit 1710 other than the storage unit 1716 may be configured by a software module executed by a processor such as a CPU or an MPU. Note that the processor may be, for example, a GPU, an FPGA, or the like. Further, each component may be configured by a circuit that performs a specific function, such as an ASIC. The storage unit 1716 may be configured with any storage medium such as an optical disk such as a hard disk or a memory.

Further, a display section 1720 and an operation section 1750 are connected to the control section 1710. Note that the display section 1720 and the operation section 1750 may be the same as the display section 107 and the operation section 108 according to the first embodiment, and a description thereof will be omitted.

The radiation control unit 1719 can function similarly to the X-ray generator control unit 102 according to the first embodiment. The radiation control unit 1719 controls, for example, imaging conditions such as the radiation irradiation angle, radiation focal position, tube voltage, and tube current by the radiation generating device 1701 based on an operator's operation via the operation unit 1750. be able to.

The radiation generating device 1701 outputs radiation based on a control signal from the radiation control unit 1719, with the axis passing through the radiation focal point as the central axis. Note that the radiation generating device 1701 can be configured as a radiation generating device that is movable in the XYZΦ directions, for example. In this case, the radiation generating device 1701 is equipped with a drive unit such as a motor, and based on a control signal from the radiation control unit 1719, the radiation generating device 1701 can move to any position within a plane (in the XY plane) intersecting the rotation axis (Z axis). The robot can move to any position or stop at any position (for example, the position of the rotation axis (Z-axis)). The radiation generating device 1701 irradiates radiation from a direction oblique to the rotation axis while moving in a plane intersecting the rotation axis or while stopping at the rotation axis position.

In FIG. 17, the rotation axis is an axis (Z-axis) in the vertical direction of the page, and the angle θ indicates an inclination angle with respect to the rotation axis (Z-axis). Angle Φ indicates a rotation angle around the Z axis. For example, the X direction corresponds to the horizontal direction of the page, and the Y direction corresponds to the direction perpendicular to the page. Further, the Z direction corresponds to, for example, the vertical direction of the page. The setting of the coordinate system in FIG. 17 is the same in FIGS. 18A and 18B.

For example, as shown in FIG. 18A, the state in which the radiation generating device 1701 moves within a plane intersecting the rotation axis (Z axis) means that the radiation generating device 1701 moves within a plane (XY plane) intersecting the rotation axis (Z axis). It shows a state of movement along a predetermined trajectory 1840. Further, the state in which the radiation generating device 1701 is stopped at the position of the rotation axis refers to a state in which the radiation generating device 1701 is stopped at the position of the rotation axis (Z-axis), for example, as shown in FIG. 18B. Further, radiation irradiation from a direction inclined with respect to the rotation axis refers to radiation irradiation in a state inclined at an angle θ with respect to the rotation axis (Z-axis), for example, as shown in FIGS. 18A and 18B.

In the radiography system shown in FIG. 18A, the radiation generating device 1701 is configured to be movable within a plane (in the XY plane) intersecting the rotation axis (Z axis), and irradiates radiation from a direction oblique to the rotation axis. do. A stage 1706 (holding section) that holds the work 1702 holds the work 1702 while being stopped at the rotation axis (Z-axis). Furthermore, the radiographic apparatus 1704 is configured to be movable within a plane (XY plane) intersecting the rotation axis (Z axis), and detects radiation that has passed through the inspection object. In FIG. 18A, the radiation generating device 1701 outputs radiation from a radiation focal point position 1820 of the radiation generating device 1701, with an axis 1820A passing through the radiation focal point as the central axis, based on a control signal from the radiation control unit 1719. Similarly, the radiation generating device 1701 outputs radiation from a radiation focal point position 1821 of the radiation generating device 1701, with an axis 1821A passing through the radiation focal point as the central axis, based on a control signal from the radiation control unit 1719. Here, the angle formed by the axis 1820A (axis 1821A) and the rotation axis (Z-axis) is the inclination angle (angle θ).

On the other hand, in the radiographic system shown in FIG. 18B, the radiation generating device 1701 irradiates radiation from a direction oblique to the rotation axis while being stopped at the rotation axis (Z-axis). A stage 1706 holding the work 1702 is configured to be movable within a plane intersecting the rotation axis (in the XY plane), and holds the work 1702. Furthermore, the radiographic apparatus 1704 is configured to be movable within a plane (XY plane) intersecting the rotation axis (Z axis), and detects radiation that has passed through the inspection object. In FIG. 18B, the radiation generating device 1701 moves from the radiation focal point position 1820 (rotation axis (Z-axis) position) of the radiation generating device 1701 to an axis 1820B passing through the radiation focal point based on a control signal from the radiation control unit 1719. Outputs radiation as the central axis. Furthermore, the radiation generating device 1701 changes the radiation irradiation angle based on the control signal of the radiation control unit 1719, and emits radiation from the radiation focal point position 1820 of the radiation generating device 1701 with an axis 1820C passing through the radiation focal point as the central axis. Output. Here, the angle formed by the axis 1820B (axis 1820C) and the rotation axis (Z-axis) is the inclination angle (angle θ).

The stage control unit 1718 controls the position of the stage 1706 so that the stage 1706 is moved to a specified position for radiography or stopped at a predetermined position for radiography. Note that the stage control unit 1718 can control the position of the stage 1706 based on a program for a predetermined photographing operation or an operation by an operator.

A stage 1706 holds a workpiece 1702 that is an object to be inspected. The stage 1706 is configured, for example, as a stage movable in the XYZΦ directions. The stage 1706 includes a drive unit such as a motor, and based on a control signal from a stage control unit 1718, the stage 1706 can move to any position within a plane (within an XY plane) that intersects the rotation axis (Z axis). It is possible to stop at an arbitrary position (for example, the position of the rotation axis (Z-axis)). The stage 1706 functions as a holding unit that holds the workpiece 1702 and is configured to be movable within a plane (in the XY plane) intersecting the rotation axis (Z axis).

The stage 1706 is configured to be movable, for example, along a trajectory 1860 in the Φ direction around the rotation axis (Z axis) (FIG. 18B) or a linear trajectory in the XY plane. Further, the stage 1706 is positioned at a predetermined position within the XY plane and can be stopped based on a control signal from the stage control unit 1718. In other cases, the stage 1706 can be configured to move in one direction using a belt conveyor or the like to place the work 1702 at a position for inspection.

The imaging device control unit 1717 controls the position and operation of the radiation imaging device 1704. Further, the imaging device control unit 1717 controls the movement positions of the robot arm 1705 and the imaging device support unit 1703. The robot arm 1705 and the imaging device support section 1703 can move the radiation imaging device 1704 to a designated position by moving to a predetermined position based on a control signal from the imaging device control section 1717. For example, the robot arm 1705 and the imaging device support unit 1703 function as an operating mechanism that moves the radiographic imaging device 1704 with degrees of freedom in the XY directions and degrees of freedom in the rotational direction (Φ) around the Z axis (degrees of freedom in the XYΦ directions). Can be configured.

The radiation imaging device 1704 is held at a predetermined position on the imaging device support portion 1703. The imaging device control section 1717 acquires position information of the imaging device control section 1717 based on the movement positions of the robot arm 1705 and the imaging device support section 1703. The imaging device control unit 1717 transmits to the generation unit 1712 position information and rotation angle information of the radiation imaging device 1704 acquired based on the movement positions and rotation angles of the robot arm 1705 and the imaging device support unit 1703.

The radiation imaging device 1704 detects the radiation output by the radiation generating device 1701 and transmitted through the workpiece 1702, and sends image information of the projected image of the workpiece 1702 to the control unit 1710. The radiation imaging apparatus 1704 is configured to be movable in a plane intersecting the rotation axis (Z-axis) by the operation of the robot arm 1705 and the imaging apparatus support part 1703, which have degrees of freedom in the XYΦ directions, and ) is detected. Here, movement in a plane intersecting the rotation axis means, for example, as shown in FIG. 18A and FIG. This shows the state of movement on a trajectory 1850.

Here, the radiographic processing according to this embodiment will be described using FIGS. 18A and 18B. In the radiography process according to this embodiment, in order to generate a three-dimensional image of the workpiece 1702, which is the object to be inspected, radiation is irradiated diagonally to the workpiece 1702 while changing the imaging position of the workpiece 1702, and a plurality of Capture the projected image.

FIGS. 18A and 18B are diagrams for explaining an example of radiographic processing according to this embodiment. FIG. 18A shows an example of radiographic processing in a state in which the radiation generating device 1701 moves on a predetermined trajectory 1840 in a plane (in the XY plane) intersecting the rotation axis (Z axis). On the other hand, FIG. 18B shows an example of radiographic processing in a state where the radiation generating device 1701 is stopped at the rotation axis (Z-axis) position. Note that the radiographic processing according to this embodiment is not limited to the configurations shown in FIGS. 18A and 18B. In the radiographic processing according to this embodiment, at least two of the radiation generating device 1701, the stage 1706 that holds the inspection object, and the radiographic device 1704 are movable within a plane intersecting the rotation axis (for example, in conjunction with each other). It is sufficient if the structure is rotatable). Note that at least two of the above are rotated so that the radiation irradiated from the radiation generating device 1701 passes through the inspection object in a direction oblique to the rotation axis and satisfies a positional relationship that can be detected by the radiation imaging device 1704. It suffices if it is configured to be movable within a plane intersecting the axis. For example, the radiation generating device 1701, the stage 1706, and the radiation imaging device 1704 may be configured to be movable within a plane intersecting the rotation axis so as to satisfy the above positional relationship. Further, for example, in order to satisfy the above positional relationship, the radiation generating device 1701 and the stage 1706 are configured to be movable within a plane intersecting the rotation axis while the radiation imaging device 1704 is stopped at the rotation axis position. Good too.

Next, image generation processing and the like in the control unit 1710 according to this embodiment will be explained. The first acquisition unit 1711 acquires a radiation image (projection image) sent from the radiation imaging apparatus 1704 and sends it to the generation unit 1712. Note that the first acquisition unit 1711 can also acquire generated three-dimensional images and tomographic images, which will be described later. Further, the first acquisition unit 1711 may acquire these image information and various images from an external device connected to the control unit 1710.

In the radiography operation according to this embodiment, in order to reconstruct a three-dimensional image from a projection image, as described above, radiation is irradiated obliquely to the workpiece 1702, and the radiography is performed for each imaging position of the workpiece 1702. be exposed. The generation unit 1712 can reconstruct a three-dimensional image from the plurality of projection images taken in this manner. More specifically, the generation unit 1712 uses the position information and rotation angle information of the radiation imaging device 1704 received from the imaging device control unit 1717 and the projected image of the workpiece 1702 photographed by the radiation imaging device 1704 to reproduce the image. Performs configuration processing and generates a three-dimensional image. Note that the generation unit 1712 reconstructs three-dimensional images of different energies using projection images based on radiation of different energies, such as images taken by X-rays with two types of tube voltages using the kV switching method described above. You can also do that.

Furthermore, the generation unit 1712 can reconstruct a tomographic image of an arbitrary cross section from the generated three-dimensional image. Note that any known method may be used to reconstruct the three-dimensional image and tomographic image. Here, the cross section from which the tomographic image is cut out from the three-dimensional image may be set based on predetermined initial settings, or may be set according to instructions from the operator. Further, the cross section may be automatically set by the control unit 1710 based on the detection result of the state of the inspection object detected based on the projected image or information from various sensors (not shown). , may be automatically set according to the selection of the inspection purpose based on the operator's operation or the like. Note that in this embodiment, the generation unit 1712 reconstructs tomographic images for three cross sections: an XY cross section, a YZ cross section, and an XZ cross section, for example.

The second acquisition unit 1713 performs segmentation of the tomographic image using the reconstructed tomographic image as input data for the learned model. The second acquisition unit 1713 extracts and acquires, as a segmentation result, a solder area to which solder is applied in the tomographic image, and a chip area that includes a solder area and is an area to which a chip such as an IC chip is applied. be able to.

Here, as the learning data for the trained model according to this example, the input data is a reconstructed tomographic image, and the output data is an image in which the chip area and solder area are labeled for the reconstructed tomographic image. Bye. Note that a trained model for extracting a chip area and a trained model for extracting a solder area may be provided separately. In this case, the labeled images may be trained separately. Specifically, a combination of a tomographic image and an image labeled with a chip area within that tomographic image, and a combination of a tomographic image and an image labeled with a solder area within that tomographic image are respectively used as training data. Through learning, each trained model can be generated.

The third acquisition unit 1714 corrects the solder area acquired by the second acquisition unit 1713 using the relationship between the solder area and the chip area and the relationship between the areas within the solder area. Note that the correction process by the third acquisition unit 1714 may be performed in the same manner as the correction process according to the first embodiment. Here, the correction processing according to this embodiment will be described in more detail with reference to FIGS. 19(a) to 19(c).

FIG. 19A shows an example of a tomographic image 1900 of a substrate including chip regions 1901, 1902, and 1903. Although this example describes an example in which the substrate includes three chip regions, the number of chip regions is not limited to this, and one or more regions may be provided depending on the desired configuration. Further, in this example, solder regions are provided within chip regions 1901, 1902, and 1903 in separate patterns (solder patterns 1910, 1920, and 1930), respectively. However, the solder patterns for each chip region 1901, 1902, 1903 may be the same or different. Note that the solder pattern can define not only the relationship between the solder areas but also the relationship between the chip area and the solder area. Note that the shape, number, and position of the chip area and the solder area may be arbitrarily set depending on the desired configuration.

The second acquisition unit 1713 uses the tomographic image 1900 as input data for the trained model to acquire the chip region extraction result and the solder region extraction result from the trained model. FIG. 19(b) shows an example of each chip region 1901, 1902, and 1903 obtained by segmentation using the tomographic image 1900 as input data of a trained model. Further, FIG. 19(c) shows an example of solder regions 1911 to 1919 in the chip region 1901 regarding the solder regions obtained by segmentation using the tomographic image 1900 as input data of the trained model.

In the example of FIG. 19(c), solder regions 1911 to 1915, 1918, and 1919 are extracted, but solder regions 1916 and 1917 are not extracted. In this case, the third acquisition unit 1714 acquires the solder area based on the relationship between the chip area 1901 and the solder area defined in the predetermined solder pattern 1910 regarding the solder areas 1911 to 1919 and the relationship between the areas within the solder area. Correct 1916 and 1917.

For example, with respect to the solder region 1913, if the distance to the next solder region in the region below the solder region 1913 is greater than a threshold, the third acquisition unit 1714 determines the distance between the solder region 1913 and the next solder region. Set up additional solder areas. In the example of FIG. 19(c), when the distance between the solder area 1913 and the next solder area 1919 in the area below the solder area 1913 is greater than the threshold, the third acquisition unit 1714 determines the intermediate position between these solder areas. A solder area 1916 is set. The size of the solder area to be set may be set depending on the desired configuration, for example, it may be the same as the size of other solder areas, or the size of the solder area 1916 defined in the solder pattern 1910 may be set. It can also be the size.

Further, with respect to the solder region 1914, if the distance between the lower end of the chip region 1901 and the solder region in the region above the lower end is greater than a threshold, the third acquisition unit 1714 detects the distance between the lower end of the chip region 1901 and the solder region. Set up additional solder area between. In the example of FIG. 19(c), when the distance from the lower end of the chip region 1901 to the solder region 1914 in the region above the lower end is greater than the threshold, the third acquisition unit 1714 acquires solder at an intermediate position between these regions. A region 1917 is set. Note that the size of the solder area to be set may be set in the same manner as described above.

Note that in the above example, the third acquisition unit 1714 corrected the solder area based on the distance in the vertical direction of the image, but the third acquisition unit 1714 corrected the solder area based on the distance in the horizontal direction and the distance in the diagonal direction of the image. Area correction may also be performed. Further, as in the first embodiment, the threshold value serving as a reference for the correction process may be dynamically set based on the average distance between the acquired solder areas, or may be set in advance based on the solder pattern 1910. It's okay. Note that the position of the solder area may be determined based on the center of gravity of the solder area.

Furthermore, since the solder pattern may be arbitrarily set, the solder regions may not be arranged at the same distance from each other, as shown in solder patterns 1920 and 1930, for example. Therefore, when correcting the solder area, the third acquisition unit 1714 can determine whether correction of the solder area is necessary based on the predetermined solder pattern. For example, the third acquisition unit 1714 can determine that the lower left area of the solder pattern 1920 and the center area of the solder pattern 1930 are not to be corrected.

In addition, in the above example, an example was described in which a solder area is missing, but as in the first embodiment, when an extra solder area is acquired, the third acquisition unit 1714 detects the missing solder area. Solder areas may be excluded. In this case as well, the third acquisition unit 1714 can use the relationship between the solder areas and the relationship between the solder area and the chip area to identify and exclude the extra solder area. The method for identifying the extra solder area may be the same as in Example 1. For example, if the distance between the acquired solder areas is smaller than a threshold, one of the corresponding solder areas may be identified as an outlier (or an inference error). ) and may be excluded from the solder area.

The display control unit 1715 can cause the display unit 1720 to display the presence or absence of solder in the solder area and the quality of mounting of the electronic component based on the analysis results performed on the analysis area by the control unit 1710. Note that the analysis results may be displayed superimposed on the tomographic image, or the tomographic image and the tomographic image on which the analysis results are superimposed may be switched and displayed according to an operator's instruction or the like. In addition, when analysis results have been acquired for tomographic images for multiple cross sections, the display control unit 1715 can display the analysis results for each tomographic image side by side on the display unit 1720, or display them in a switched manner. can. In these cases, the display control unit 1715 collectively switches the display between the plurality of tomographic images and the analysis results or the plurality of tomographic images on which the analysis results are superimposed, in response to instructions from the operator via the operation unit 1750. You can go.

Furthermore, the display control unit 1715 can cause the display unit 1720 to display the analysis region acquired for the tomographic image. Further, the display control unit 1715 can also display the superimposed analysis region on the tomographic image on the display unit 1720. Note that when analysis regions have been acquired for tomographic images of multiple cross sections, the display control section 1715 displays images etc. in which these analysis regions are superimposed on the corresponding tomographic images side by side on the display section 1720, or switches between them. It can also be displayed. Further, the display control unit 1715 may switch between displaying the image on which the analysis region is superimposed and the original tomographic image. In this case, the display control unit 1715 may collectively switch the display of a plurality of tomographic images in which a plurality of tomographic images and an analysis region are superimposed in accordance with an instruction from an operator via the operation unit 1750.

In this example, the second acquisition unit 1713 acquired the solder area and the chip area by segmentation, and the third acquisition unit 1714 corrected the solder area, which is the analysis area. However, the regions obtained by segmentation are not limited to these, and may be any region related to an object in which similar structures are connected in series, as described above. Therefore, similarly to Examples 1 to 4, regions such as the lumbar vertebrae, cervical vertebrae, and thoracic vertebrae may be used as analysis regions, and these regions and spinal regions may be obtained by segmentation, and the above-described correction processing may be applied. Even in such a case, by applying the image processing according to the present disclosure, it is possible to correct missing areas that should be acquired and extra areas that have been acquired, and ultimately improve the analysis area that should be acquired. can be obtained with high precision.

Note that the series of processes according to this embodiment is the same as the series of processes according to the first embodiment, and therefore will be omitted. However, it should be noted that in this embodiment, radiation imaging is performed by obliquely irradiating radiation onto the workpiece 1702 while changing the imaging position of the workpiece 1702 as described above to photograph a plurality of projection images.

As described above, the control unit 1710 according to this embodiment functions as an example of an image processing device including the first acquisition unit 1711, the second acquisition unit 1713, and the third acquisition unit 1714. The first acquisition unit 1711 functions as an example of a first acquisition unit that acquires an image obtained by irradiating the inspection object with radiation in an oblique direction. In addition, the second acquisition unit 1713 uses the image acquired by the first acquisition unit 1711 as input data for the trained model, and the second acquisition unit 1713 includes a first area and a wider area than the first area. It functions as an example of a second acquisition means that acquires the second area. Here, the solder regions 1911 to 1915 and 1918 to 1919 are examples of the acquired first regions, and the chip region 1901 is an example of the second region. The third acquisition unit 1714 uses at least one of the relationship between the acquired first area and the acquired second area and the relationship between the plurality of areas within the acquired first area, to obtain the acquired first area. It functions as an example of a third acquisition means that corrects the area and uses the corrected first area to acquire an analysis area for performing image analysis processing. Note that the analysis area can be the corrected first area.

More specifically, the third acquisition unit 1714 acquires a predetermined pattern regarding the first area, the relationship between the acquired first area and the acquired second area, and the plurality of patterns in the acquired first area. The acquired first region is corrected using at least one of the relationships between the regions. Here, the solder pattern 1910 is an example of a predetermined pattern. Further, the third acquisition unit 1714 can use a predetermined pattern regarding the first area to determine whether or not the acquired first area needs to be corrected.

According to the image processing apparatus according to the present embodiment, an analysis region in which similar structures such as a solder region or a lumbar vertebrae region are consecutively connected can be acquired with high precision. In addition, since radiation is irradiated obliquely to the object to be inspected, the object to be inspected can be brought closer to the radiation source, making it easier to adjust the imaging magnification and making the size of the inspection device more compact. can.

Furthermore, the control unit 1710 according to the present embodiment further includes a display control unit 1715 that functions as an example of a display control unit that causes a display unit 1720 (display unit) to display the analysis region acquired by the third acquisition unit 1714. . Further, the second acquisition unit 1713 uses the trained model to acquire a plurality of first regions and a plurality of second regions based on a plurality of tomographic images corresponding to at least two cross sections of the inspection object. can do. Further, the third acquisition unit 1714 can correct the acquired plurality of first regions and acquire a plurality of analysis regions using the corrected plurality of first images. In this case, the display control unit 1715 can cause the display unit 1720 to display the plurality of acquired analysis regions or the plurality of tomographic images in which the corresponding analysis regions are superimposed side by side. In this case, it is possible to confirm analysis regions related to a plurality of cross-sections, and the inspection of the inspection object can be performed more efficiently.

Furthermore, the display control unit 1715 displays a plurality of tomographic images corresponding to at least two cross-sections, a plurality of analysis regions corresponding to the plurality of tomographic images, or a plurality of superimposed corresponding analysis regions according to an instruction from the operator. The tomographic images can be switched all at once and displayed on the display unit 1720. In this case, the acquired analysis region and the reconstructed tomographic image can be easily compared. Therefore, the inspection on the inspection object can be performed more efficiently.

Note that the cross sections related to the plurality of tomographic images can be set according to at least one of initial settings, operator instructions, detection results of the state of the inspection object, and selection of the inspection purpose. Therefore, a tomographic image of a cross section according to the desired settings is generated, and an analysis region corresponding to the tomographic image can be acquired with high accuracy. Thereby, the inspection regarding the inspection object can be performed more efficiently.

Note that in this example, the second acquisition unit 1713 used the learned model to acquire the first region and the second region based on the tomographic image. On the other hand, the second acquisition unit 1713 may acquire the first region and the second region based on the projection image, three-dimensional image, or energy sub-image. In this case, a projected image, a three-dimensional image, or an energy sub-image may be used as the input data of the learning data of the learned model. Further, as the output data of the learning data, an image in which the first region and the second region are labeled for a projected image, a three-dimensional image, or an energy sub-image may be used. Note that when a three-dimensional image is used as input data for the learning data, an image obtained by labeling a tomographic image of a predetermined cross section with a first region and a second region may be used as output data for the learning data. .

(Modified example)
The learning data of the trained models according to the first to fifth embodiments and the modified examples are not limited to data obtained using the radiation imaging apparatus itself that performs actual imaging. The image data may be data obtained using the same type of radiographic apparatus, data obtained using the same type of radiographic apparatus, etc., depending on the desired configuration.

Furthermore, when using a plurality of images as input to a trained model, the plurality of input images can be input to a plurality of input channels of the trained model, respectively. On the other hand, a plurality of input images may be combined into one image and input into one channel of the trained model. In this case, one image obtained by combining input images may be used as input data for the learning data of the trained model as well.

Further, in the above embodiments 1 to 5 and modified examples, the control units 101 and 1710 superimpose and display segmentation results such as analysis regions on the display image in response to instructions from the operator such as operating a button (not shown). It may be displayed on the sections 107 and 1720. Note that when there are a plurality of display images and corresponding segmentation results, the segmentation results may be superimposed on each display image and displayed on the display units 107 and 1720 at once according to an instruction from the operator. Further, the control units 101 and 1710 may display the acquired results of the bone density analysis region and the background region, or the solder region and the chip region, in different colors, respectively. In this case as well, a plurality of images may be displayed in different colors at once according to the operator's instructions.

In addition, in the trained models according to the above-mentioned Examples 1 to 5 and the modified examples, the magnitude of the brightness value of the image of input data, the order and slope of bright and dark areas, position, distribution, continuity, etc. are used as feature quantities. It is thought that it is extracted as a part and used for inference processing.

Further, various learned models according to the above-described embodiments 1 to 5 and modifications can be provided in the control units 101 and 1710. The trained model may be configured with a software module executed by a processor such as a CPU, MPU, GPU, or FPGA, or may be configured with a circuit that performs a specific function such as an ASIC. Further, the learned model may be provided in another server device connected to the control unit 101, 1710. In this case, the control units 101 and 1710 can use the trained model by connecting to a server or the like provided with the trained model via an arbitrary network such as the Internet. Here, the server provided with the trained model may be, for example, a cloud server, a fog server, an edge server, or the like. Note that when configuring a network within a facility, within a site that includes a facility, within an area that includes multiple facilities, etc. to enable wireless communication, for example, if a network is configured to enable wireless communication, The reliability of the network may be improved by configuring it to use radio waves in a dedicated wavelength band. Furthermore, the network may be constructed using wireless communication that is capable of high speed, large capacity, low delay, and multiple simultaneous connections.

Further, the present invention can also be realized by executing the following processing. That is, software (programs) that implement one or more functions of the various embodiments and modifications described above is supplied to a system or device via a network or various storage media, and the computer (or CPU or (MPU, etc.) reads and executes a program.

Furthermore, the present invention provides software (programs) for realizing one or more functions of the various embodiments and modifications described above to a system or device via a network or a storage medium, and a computer of the system or device This can also be achieved by reading and executing a program. It can also be realized by a circuit (for example, an ASIC) that realizes one or more functions. A computer has one or more processors or circuits and may include separate computers or a network of separate processors or circuits for reading and executing computer-executable instructions.

The processor or circuit may include a central processing unit (CPU), microprocessing unit (MPU), graphics processing unit (GPU), application specific integrated circuit (ASIC), or field programmable gateway (FPGA). The processor or circuit may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

The above disclosure includes the following configurations and methods.
(Configuration 1)
a first acquisition means for respectively acquiring a spinal region and an intervertebral region in the input image;
a second acquisition means for acquiring a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae using the acquired spinal region and the acquired intervertebral region;
An image processing device having:
(Configuration 2)
The image processing device according to configuration 1, wherein the first acquisition means acquires the spinal region and the intervertebral region in the input image, respectively, using a learned model.
(Configuration 3)
The image processing device according to configuration 1 or 2, wherein the input image is an energy subtraction image.
(Configuration 4)
The image processing device according to any one of configurations 1 to 3, wherein the input image is an image acquired using X-rays with two types of tube voltages.
(Configuration 5)
The image processing device according to any one of configurations 1 to 3, wherein the input image is an image acquired using a sensor having two types of materials with different X-ray absorption rates.
(Configuration 6)
The image processing device according to configuration 1 or 2, wherein the input image is a simple X-ray image.
(Configuration 7)
The image processing device according to configuration 2, wherein the first acquisition means separately includes a trained model for acquiring the spinal region and a trained model for acquiring the intervertebral region.
(Configuration 8)
The image processing device according to configuration 2, wherein the first acquisition means includes one trained model that acquires the spinal region and the intervertebral region, respectively.
(Configuration 9)
The image processing device according to configuration 1, wherein the first acquisition means acquires the spinal region in the input image, and acquires the intervertebral region from a spinal image generated based on the acquisition result.
(Configuration 10)
The second acquisition means acquires a region inside the acquired spinal region as a vertebral body within a reference region sandwiched by straight lines passing through the acquired intervertebral region,
The image processing device according to any one of configurations 1 to 9, wherein the bone density analysis region of the lumbar vertebrae is acquired by counting the acquired vertebral bodies from the pelvic side.
(Configuration 11)
The image processing device according to any one of configurations 1 to 10, wherein the second acquisition means performs a process of excluding a region outside the acquired spinal region from the acquired intervertebral region. .
(Configuration 12)
The image processing device according to any one of configurations 1 to 11, wherein the second acquisition means performs rotation correction of the input image based on the acquired shape of at least one of the spinal region and the intervertebral region. .
(Configuration 13)
11. The image processing device according to configuration 10, wherein the second acquisition means draws a straight line constituting the reference region diagonally based on the acquired shape of at least one of the spinal region and the intervertebral region.
(Configuration 14)
14. The image processing apparatus according to any one of configurations 1 to 13, wherein the second acquisition means sets a background area based on a variance of brightness values of the input image.
(Configuration 15)
When the number of acquired intervertebral regions is less than a predetermined number, the second acquisition means divides the acquired spinal regions into the predetermined number, thereby determining the bone density analysis region. The image processing device according to any one of acquiring configurations 1 to 14.
(Configuration 16)
The image according to any one of configurations 1 to 15, wherein the second acquisition means performs a correction process based on a distance between an upper end or a lower end of the acquired spinal region and an intervertebral region one inside. Processing equipment.
(Configuration 17)
The first acquisition means acquires the spinal region, the intervertebral region, and the pelvic region in the input image, and the second acquisition means acquires the spinal region, the intervertebral region, and the pelvic region in the input image. 17. The image processing device according to any one of configurations 1 to 16, which acquires the bone density analysis region of the lumbar vertebrae based on the acquisition results.
(Configuration 18)
The first acquisition means acquires the spinal region, the intervertebral region, and the rib region in the input image, and the second acquisition means acquires the spinal region, the intervertebral region, and the rib region in the input image. 18. The image processing device according to any one of configurations 1 to 17, which acquires the bone density analysis region of the lumbar vertebrae based on the acquisition results.
(Configuration 19)
The second obtaining means determines the spinal region based on at least one of the acquired distance from the upper end of the spinal region to the lower end of the rib region, or the distance from the lower end of the spinal region to the upper end of the pelvic region. 19. The image processing apparatus according to any one of configurations 1 to 18, which performs a correction process to extend or shorten.
(Configuration 20)
The image processing according to any one of configurations 1 to 19, wherein the first acquisition means acquires an outer frame area that is a fusion of the spinal region and at least one of a pelvic region and a rib region in the input image. Device.
(Configuration 21)
The first acquisition means acquires at least two of the three images, two images acquired using the DXA method and an energy subtraction image generated from the two images. and obtaining the spinal region and the intervertebral region, respectively,
The image processing device according to configuration 1 or 2, wherein the second acquisition means acquires the bone density analysis region based on the acquisition results of the spinal region and the intervertebral region for the at least two images.
(Configuration 22)
22. The image processing apparatus according to any one of configurations 1 to 21, further comprising a display unit that displays the acquisition result or acquisition process by the second acquisition unit to an operator.
(Configuration 23)
23. The image processing apparatus according to any one of configurations 1 to 22, further comprising an operation means for an operator to operate while viewing the acquisition result or acquisition process by the second acquisition means.
(Configuration 24)
a first acquisition means for acquiring an image obtained by irradiating radiation in an oblique direction to the inspection object;
a second acquisition means for acquiring a first region and a second region that includes the first region and is wider than the first region, using the acquired image as input data for a trained model; ,
Correcting the acquired first area using at least one of the relationship between the acquired first area and the acquired second area and the relationship between a plurality of areas in the acquired first area. and a third acquisition means for acquiring an analysis area for performing image analysis processing using the corrected first area;
An image processing device having:
(Configuration 25)
The third acquisition means includes a predetermined pattern regarding the first area, a relationship between the acquired first area and the acquired second area, and a plurality of areas within the acquired first area. 25. The image processing device according to configuration 24, wherein the acquired first region is corrected using at least one of the relationships between.
(Configuration 26)
26. The image processing apparatus according to configuration 25, wherein the third acquisition means uses the predetermined pattern to determine whether correction of the acquired first area is necessary.
(Configuration 27)
27. The image processing device according to any one of configurations 24 to 26, wherein the analysis area is the corrected first area.
(Configuration 28)
further comprising display control means for displaying the acquired analysis region on a display means,
The second acquisition means uses the learned model to acquire the plurality of first regions and the plurality of second regions based on a plurality of tomographic images corresponding to at least two cross sections of the inspection object. get
The third acquisition means corrects the acquired plurality of first regions and uses the corrected plurality of first regions to acquire the plurality of analysis regions,
28. The image processing according to any one of 24 to 27, wherein the display control means displays the plurality of acquired analysis regions or the plurality of tomographic images on which the corresponding analysis regions are superimposed on the display means side by side. Device.
(Configuration 29)
The display control means displays the plurality of tomographic images by collectively switching between the plurality of tomographic images and the plurality of analysis regions or the plurality of tomographic images on which the corresponding analysis regions are superimposed, according to an instruction from an operator. 29. The image processing device according to 28, wherein the image processing device is displayed on a means.
(Configuration 30)
The cross sections related to the plurality of tomographic images are set according to any one of configurations 28 to 29, which are set according to at least one of initial settings, operator instructions, detection results of the state of the inspection target, and selection of the inspection purpose. The image processing device described.
(Method 1)
a first acquisition step of respectively acquiring a spinal region and an intervertebral region in the input image;
a second acquisition step of acquiring a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae using the acquired spinal region and the acquired intervertebral region;
A method of operating an image processing device including:
(Method 2)
a first acquisition step of acquiring an image obtained by irradiating radiation in an oblique direction to the inspection object;
a second acquisition step of acquiring a first region and a second region that includes the first region and is wider than the first region, using the acquired image as input data of the trained model; ,
Correcting the acquired first area using at least one of the relationship between the acquired first area and the acquired second area and the relationship between a plurality of areas in the acquired first area. and a third acquisition step of acquiring an analysis region for performing image analysis processing using the corrected first region;
A method of operating an image processing device including:
(Program 1)
A program that, when executed by a computer, causes the computer to execute each step of the method for operating an image processing apparatus according to method 1 or 2.

Although the present invention has been described above with reference to the embodiments and modified examples, the present invention is not limited to the above embodiments and modified examples. The present invention includes inventions modified within the scope of the spirit of the present invention and inventions equivalent to the present invention. Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of the spirit of the present invention.

120: Image processing device, 121: First acquisition unit, 122: Second acquisition unit

Claims (33)

a first acquisition means for respectively acquiring a spinal region and an intervertebral region in the input image;
a second acquisition means for acquiring a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae using the acquired spinal region and the acquired intervertebral region;
An image processing device having: The image processing apparatus according to claim 1, wherein the first acquisition means acquires the spinal region and the intervertebral region in the input image, respectively, using a learned model. The image processing device according to claim 1 or 2, wherein the input image is an energy subtraction image. 3. The image processing apparatus according to claim 1, wherein the input image is an image acquired using X-rays with two types of tube voltages. The image processing apparatus according to claim 1 or 2, wherein the input image is an image acquired using a sensor having two types of materials with different X-ray absorption rates. The image processing apparatus according to claim 1 or 2, wherein the input image is a simple X-ray image. The image processing apparatus according to claim 2, wherein the first acquisition means separately includes a trained model for acquiring the spinal region and a trained model for acquiring the intervertebral region. The image processing apparatus according to claim 2, wherein the first acquisition means includes one trained model that acquires the spinal region and the intervertebral region, respectively. The image processing device according to claim 1 or 2, wherein the first acquisition means acquires the spinal region in the input image, and acquires the intervertebral region from a spinal image generated based on the acquisition result. The second acquisition means acquires a region inside the acquired spinal region as a vertebral body within a reference region sandwiched by straight lines passing through the acquired intervertebral region,
The image processing device according to claim 1 or 2, wherein the bone density analysis region of the lumbar vertebrae is acquired by counting the acquired vertebral bodies from the pelvic side. The image processing apparatus according to claim 1 or 2, wherein the second acquisition means performs a process of excluding a region outside the acquired spinal region from the acquired intervertebral region. 3. The image processing apparatus according to claim 1, wherein the second acquisition means performs rotational correction of the input image based on the acquired shape of at least one of the spinal region and the intervertebral region. 11. The image processing apparatus according to claim 10, wherein the second acquisition means diagonally draws a straight line constituting the reference area based on the acquired shape of at least one of the vertebral region and the intervertebral region. 3. The image processing apparatus according to claim 1, wherein the second acquisition means sets a background area based on a variance of brightness values of the input image. When the number of acquired intervertebral regions is less than a predetermined number, the second acquisition means divides the acquired spinal regions into the predetermined number, thereby determining the bone density analysis region. The image processing apparatus according to claim 1 or 2, wherein the image processing apparatus acquires an image. 3. The second acquisition means performs the correction process based on the distance between the acquired upper end or lower end of the spinal region and an intervertebral region one position inside from the upper end or lower end. Image processing device. The first acquisition means acquires the spinal region, the intervertebral region, and the pelvic region in the input image, and the second acquisition means acquires the spinal region, the intervertebral region, and the pelvic region in the input image. The image processing apparatus according to claim 1 or 2, wherein the bone density analysis region of the lumbar vertebrae is acquired based on the acquisition result of. The first acquisition means acquires the spinal region, the intervertebral region, and the rib region in the input image, and the second acquisition means acquires the spinal region, the intervertebral region, and the rib region in the input image. The image processing apparatus according to claim 1 or 2, wherein the bone density analysis region of the lumbar vertebrae is acquired based on the acquisition result of. The second obtaining means determines the spinal region based on at least one of the acquired distance from the upper end of the spinal region to the lower end of the rib region, or the distance from the lower end of the spinal region to the upper end of the pelvic region. 3. The image processing apparatus according to claim 1, wherein the image processing apparatus performs a correction process to extend or shorten. 3. The image processing apparatus according to claim 1, wherein the first acquisition means acquires an outer frame area that is a fusion of the spinal region and at least one of a pelvic region and a rib region in the input image. The first acquisition means acquires at least two of the three images, two images acquired using the DXA method and an energy subtraction image generated from the two images. and obtaining the spinal region and the intervertebral region, respectively,
The image processing apparatus according to claim 1 or 2, wherein the second acquisition means acquires the bone density analysis region based on the acquisition results of the spinal region and the intervertebral region for the at least two images. 3. The image processing apparatus according to claim 1, further comprising display means for displaying the acquisition result or acquisition process by the second acquisition means to an operator. 3. The image processing apparatus according to claim 1, further comprising operation means for an operator to operate while viewing the acquisition result or acquisition process by the second acquisition means. a first acquisition means for acquiring an image obtained by irradiating radiation in an oblique direction to the inspection object;
a second acquisition means for acquiring a first region and a second region that includes the first region and is wider than the first region, using the acquired image as input data for a trained model; ,
Correcting the acquired first area using at least one of the relationship between the acquired first area and the acquired second area and the relationship between a plurality of areas in the acquired first area. and a third acquisition means for acquiring an analysis area for performing image analysis processing using the corrected first area;
An image processing device having: The third acquisition means includes a predetermined pattern regarding the first area, a relationship between the acquired first area and the acquired second area, and a plurality of areas within the acquired first area. 25. The image processing apparatus according to claim 24, wherein the acquired first region is corrected using at least one of the relationships between. 26. The image processing apparatus according to claim 25, wherein the third acquisition means uses the predetermined pattern to determine whether correction of the acquired first area is necessary. The image processing device according to any one of claims 24 to 26, wherein the analysis area is the corrected first area. further comprising display control means for displaying the acquired analysis region on a display means,
The second acquisition means uses the learned model to acquire the plurality of first regions and the plurality of second regions based on a plurality of tomographic images corresponding to at least two cross sections of the inspection object. get
The third acquisition means corrects the acquired plurality of first regions and uses the corrected plurality of first regions to acquire the plurality of analysis regions,
25. The image processing apparatus according to claim 24, wherein the display control means causes the display means to display the plurality of acquired analysis regions or the plurality of tomographic images in which the corresponding analysis regions are superimposed side by side on the display means. The display control means displays the plurality of tomographic images by collectively switching between the plurality of tomographic images and the plurality of analysis regions or the plurality of tomographic images on which the corresponding analysis regions are superimposed, according to an instruction from an operator. The image processing device according to claim 28, wherein the image processing device displays the image on a means. 29. The image processing apparatus according to claim 28, wherein the cross sections related to the plurality of tomographic images are set according to at least one of initial settings, an operator's instruction, a detection result of a state of the inspection object, and a selection of an inspection purpose. a first acquisition step of respectively acquiring a spinal region and an intervertebral region in the input image;
a second acquisition step of acquiring a bone density analysis region that is a target region for bone density measurement of the lumbar vertebrae using the acquired spinal region and the acquired intervertebral region;
A method of operating an image processing device including: a first acquisition step of acquiring an image obtained by irradiating radiation in an oblique direction to the inspection object;
a second acquisition step of acquiring a first region and a second region that includes the first region and is wider than the first region, using the acquired image as input data of the trained model; ,
Correcting the acquired first area using at least one of the relationship between the acquired first area and the acquired second area and the relationship between a plurality of areas in the acquired first area. and a third acquisition step of acquiring an analysis region for performing image analysis processing using the corrected first region;
A method of operating an image processing device including: A program that, when executed by a computer, causes the computer to execute each step of the method for operating an image processing apparatus according to claim 31 or 32.