JP2021058480A - Target outer shape estimation device and treatment apparatus - Google Patents

Abstract

To quickly estimate a target object portion in comparison to a technique of estimating the target object portion by deriving a three-dimensional position from a plurality of photographed images.SOLUTION: A target outer shape estimation device comprises: photographing means (6a, 6b) which photographs an image of a region including a target in a body of a subject (2) from the second direction different from the first direction with respect to the first direction of irradiating the target in the body of the subject (2) with the first radiation; learning means (C9) which outputs correlation information by learning the correlation between the outer shape of the target reflected in the photographed image and the outer shape of the target in the first direction on the basis of the outer shape of the target reflected in the photographed image from the second direction; and outer shape estimation means (C13) which estimates the outer shape of the target in the first direction on the basis of the outer shape of the target reflected in the photographed image from the second direction that is photographed immediately before the treatment and the correlation information.SELECTED DRAWING: Figure 2

Description

The present invention relates to a target contour estimation device and a treatment device.

When treating a tumor by irradiating it with radiation such as radiated electromagnetic waves such as X-rays or particle beams such as positron beams, the tumor should be treated so that the radiation does not irradiate other than the tumor part of the subject (patient). It is necessary to specify the outer shape. In addition, since the position of the target target (tumor, etc.) for treatment moves due to the patient's breathing or beating, etc., in order to prevent radiation from irradiating a site other than the target target (normal site), X is used in real time. It is necessary to take a line-through image to track and locate the target object and irradiate the target object only when it moves to the radiation irradiation position. In addition, the technique of taking an image to identify and track the position of the target target and guide the irradiation position of radiation, that is, the target tracking technique by so-called image guidance, is an X-ray fluoroscopic image taken by using X-rays on the order of kV. It is common, but not limited to, used. For example, not only X-ray fluoroscopic images, but also X-ray images and ultrasonic images taken using MV-order X-rays, MRI (Magnetic Resonance Imaging) images, CT images (Computed Tomography), It is also possible to use PET (Positron Emission Tomography) images. It is also possible to use an X-ray backscattered image using backscattering instead of the fluoroscopic image.
The following techniques are known as techniques for identifying and tracking the position of a target target.

Patent Document 1 (International Publication No. 2018/159975) describes a technique for tracking a tracking target such as a tumor that moves due to a patient's respiration or the like. In the technique described in Patent Document 1, a bone structure DRR image such as a bone and a soft tissue DRR image such as a tumor are separated from an image including a tracking target (tumor or the like), and the bone structure DRR image is randomly linearly converted. Create a plurality of superimposed images (random overlays) on which the objects are superimposed on the soft tissue DRR image. Then, learning is performed using deep learning using a plurality of superimposed images. At the time of treatment, a region to be tracked is specified from the fluoroscopic image at the time of treatment and the learning result, and therapeutic X-rays are irradiated.

In Patent Document 2 (Japanese Patent No. 3053389), a tumor marker embedded in the vicinity of a tumor to be treated is photographed by seeing through from a plurality of directions with X-rays, and a pre-registered follow-up target (tumor) is obtained. A technique for identifying the position of a tumor marker in three dimensions by performing template matching between a marker template image and a shade normalization cross-correlation is described.

International Publication No. 2018/159775: WO2018 / 159775A1 Publication (“0031” to “0053”, FIGS. 1 to 7) Japanese Patent No. 3053389 ("0035" to "0046", FIGS. 1 and 2)

(Problems of conventional technology)
In the techniques described in Patent Documents 1 and 2, the direction of irradiating the patient with X-rays for fluoroscopic imaging of the target object and the direction of irradiating the therapeutic beam toward the patient are different. The shape of the tumor is not a three-dimensionally homogeneous and isotropic shape such as a spherical surface, but may be elongated or uneven. Therefore, the shape of the tumor projected in each direction is generally different between the perspective direction and the treatment beam irradiation direction, and when the treatment beam is simply irradiated using the image in the perspective direction, the entire tumor is X-rayed. There is a problem that the line is not irradiated or the normal part is irradiated with X-ray.

In the technique described in Patent Document 2, the three-dimensional coordinates of the tumor marker are calculated from the fluoroscopic images taken from two directions. Then, the position to irradiate the therapeutic beam is calculated from the three-dimensional position coordinates and irradiated. Therefore, the technique described in Patent Document 2 calculates the three-dimensional coordinates (three-dimensional) of the tumor from the two captured images (two-dimensional), and determines the irradiation field (two-dimensional) in the irradiation direction of the therapeutic beam. It is a technology to derive.
When operations are performed in order such as two-dimensional, three-dimensional, and two-dimensional as in the technique described in Patent Document 2, processing takes time, and there is a problem that it is difficult to respond to real-time tracking and changes in the shape of the outer shape. .. Further, the technique of embedding a metal marker in the body, such as the technique described in Patent Document 2, becomes an invasive method and has a problem of imposing a burden on the patient.

It is a technical subject of the present invention to estimate a target target site more quickly than a technique of estimating a target target site by deriving a three-dimensional position from a plurality of captured images.

In order to solve the technical problem, the target outer shape estimation device of the invention according to claim 1 is used.
It has a first irradiation means for irradiating a target in the body of a subject with a first radiation, and a second direction different from the first direction with respect to the first direction for irradiating the first radiation. An imaging means for capturing an image of the area including the target in the subject's body from the direction of
Based on the outer shape of the target reflected in the fluoroscopic image from the second direction, the classifier which is the correlation information by learning the correlation between the outer shape of the target reflected in the fluoroscopic image and the outer shape of the target in the first direction is obtained. Learning means to output and
An outer shape estimating means for estimating the outer shape of the target in the first direction based on the outer shape of the target reflected in the fluoroscopic image from the second direction taken most recently in which the treatment is performed and the discriminator.
It is characterized by being equipped with.

The invention according to claim 2 is the target outer shape estimation device according to claim 1.
The first radiation is an X-ray or a particle beam,
The fluoroscopic image is characterized in that it is any one or a combination of an X-ray image, a nuclear magnetic resonance image, an ultrasonic inspection image, a positron emission tomography image, a body surface shape image, and a photoacoustic imaging image.

The invention according to claim 3 is the target outer shape estimating device according to claim 1 or 2.
Based on the outer shape of the target shown in the fluoroscopic image, the learning means that also learns the correlation between the outer shape of the target shown in the fluoroscopic image and the distance from the first irradiation means in the first direction.
A distance estimating means for estimating the distance of the target from the first irradiation means in the first direction based on the outer shape of the target reflected in the fluoroscopic image taken most recently in which the treatment is performed and the discriminator. When,
It is characterized by being equipped with.

The invention according to claim 4 is the target outer shape estimating device according to any one of claims 1 to 3.
An imaging means for capturing an image of a region containing a target in the subject's body from a third direction different from the second direction.
Based on the outer shape of the target reflected in the fluoroscopic image from the second direction and the outer shape of the target reflected in the fluoroscopic image from the third direction, the outer shape of the target reflected in each fluoroscopic image and the outer shape of the target in the first direction. The learning means that learns the correlation with the outer shape of the target and outputs a classifier,
It is characterized by being equipped with.

The invention according to claim 5 is the target outer shape estimating device according to any one of claims 1 to 4.
A superimposed image creating means for creating a plurality of superimposed images in which each of a plurality of non-tracked target images not including the image feature is superimposed on the tracked portion image including the target image feature.
Based on the outer shape of the target reflected in the perspective image from the second direction, the correlation between the outer shape of the target reflected in the perspective image and the outer shape of the target in the first direction is learned, and the plurality of superimposed images are displayed. Based on the learning means for creating a classifier by learning at least one of the image feature and the position information of the target.
It is characterized by being equipped with.

The invention according to claim 6 is the target outer shape estimating device according to claim 5.
An input unit for inputting a teacher image for specifying the image feature of the target in advance,
The learning means for creating a classifier by learning at least one of the image feature and the position information of the target based on the plurality of superimposed images and the teacher image.
It is characterized by being equipped with.

The invention according to claim 7 is the target outer shape estimating device according to claim 5 or 6.
Based on the size of the image feature of the target, the resolution of the image, and the preset tracking accuracy, the number of the non-tracking target images is derived, and the non-tracking target image according to the derived number of images is obtained. Non-tracked image editing means to create,
It is characterized by being equipped with.

The invention according to claim 8 is the target outer shape estimating device according to any one of claims 5 to 7.
An image separation means for separating and extracting a tracking target part image including the target image feature and a separate non-tracking target image not including the tracking target image feature from a learning original image including the target image feature. ,
A non-tracking target image editing means for editing and creating the non-tracking target image composed of a plurality of separated non-tracking target images, and a non-tracking target image editing means.
With the superimposed image creating means for creating the superimposed image based on the tracked portion image and the non-tracked target image.
It is characterized by being equipped with.

The invention according to claim 9 is the target outer shape estimating device according to any one of claims 5 to 7.
An image separation means for separating an original image including the image feature of the target into a tracking target part image including the image feature of the target and a separation non-tracking target image not including the image feature of the target.
An obstacle image acquisition means for acquiring an image of an obstacle that is not included in the original image and is included in the non-tracking target image.
A non-tracking target image editing means for editing and creating a plurality of the non-tracking target image by editing at least one of the separated non-tracking target image and the obstacle image.
The superimposed image creating means for creating the superimposed image based on the tracked portion image and the non-tracked target image, and
It is characterized by being equipped with.

The invention according to claim 10 is the target outer shape estimating device according to claim 8 or 9.
The image separation means for separating and extracting the tracking target portion image and the separation non-tracking target image from the original image for learning including the target image feature based on the image contrast information of the target image feature.
It is characterized by being equipped with.

The invention according to claim 11 is the target outer shape estimating device according to any one of claims 5 to 10.
A tracking accuracy evaluation means for evaluating the tracking accuracy of the target based on the image for evaluation in which at least one of the image feature and the position information of the target is preset and the classifier.
It is characterized by being equipped with.

The invention according to claim 12 is the target outer shape estimating device according to claim 11.
The learning means, which recreates the classifier by increasing the number of non-tracked images when the tracking accuracy by the tracking accuracy evaluation means does not reach a preset accuracy.
It is characterized by being equipped with.

In order to solve the technical problem, the therapeutic device of the invention according to claim 13 is
The target outer shape estimation device according to any one of claims 1 to 12,
An irradiation means that irradiates therapeutic radiation from the first direction based on the outer shape of the target estimated by the target outer shape estimation device.
It is characterized by being equipped with.

According to the inventions of claims 1 and 13, the target target site can be estimated more quickly than the technique of deriving a three-dimensional position from a plurality of captured images to estimate the target target site.
According to the second aspect of the invention, any one or a combination of an X-ray image, a nuclear magnetic resonance image, an ultrasonic inspection image, a positron emission tomography image, a body surface shape image, and a photoacoustic imaging image. It can be used for treatment with X-rays or particle beams.
According to the invention of claim 3, the distance to the target can also be estimated, and it can be utilized for treatment with a particle beam such as a proton beam.

According to the invention of claim 4, the outer shape of the tumor is estimated based on the fluoroscopic images from two directions, and the estimation accuracy of the outer shape can be improved as compared with the case from one direction. ..
According to the fifth aspect of the present invention, at least one of the region and the position of the target site for treatment can be easily and quickly identified as compared with the conventional image tracking method by template matching.
According to the invention of claim 6, the processing speed can be increased as compared with the case where the teacher image is not used. According to the description of claim 6, when there is a teacher image, the position and shape of the target image feature region in the teacher image are the same as the position and shape of the target image feature region included in the plurality of superimposed images. There is a correlation, but there is no correlation between the position and shape of image features of non-target obstacles. The presence or absence of this correlation has the effect of learning by distinguishing whether or not the information is necessary for tracking.

According to the invention of claim 7, a necessary and sufficient number of non-tracking target images can be created according to the size and resolution of the image and the tracking system.
According to the invention of claim 8, the background is different for each subject such as a patient as compared with the case where a separated non-tracking target image including a background object / obstacle that hinders tracking such as bone is not used. You can learn according to the difference between obstacles and obstacles. That is, in the conventional technique, only a classifier created based on a large number of subjects can be realized, but as compared with those conventional techniques, it is possible to perform tracking suitable for each subject.
According to the invention of claim 9, by adding an image of an obstacle not included in the original image, when an obstacle not included in the original image is mixed in the non-tracked target image being tracked. However, the accuracy of target tracking can also be improved. That is, in the past, when an obstacle not included in the original image of learning was mixed in the non-tracked target image of the target, the accuracy of target tracking was significantly reduced. It is possible to perform tracking suitable for the image to be tracked.

According to the invention of claim 10, the target and the non-target can be automatically separated based on the contrast information, and the separation process can be performed as compared with the case where the target is manually specified. Easy to do.
According to the invention of claim 11, the discriminator can be evaluated before the actual treatment.
According to the invention of claim 12, when the accuracy of the classifier is insufficient, the classifier can be recreated by using the superimposed image with an increased number of images, and the accuracy of the classifier can be improved. Can be done.

FIG. 1 is an explanatory view of a radiotherapy machine to which the tumor shape estimation device of Example 1 of the present invention is applied. FIG. 2 is an explanatory view of a main part of the radiotherapy machine of the first embodiment. FIG. 3 is a block diagram showing each function provided in the control unit of the radiotherapy machine of the first embodiment. FIG. 4 is an explanatory diagram of an example of processing in the control unit of the first embodiment. FIG. 5 is an explanatory diagram of an example of the superimposed image 26 and the teacher image 30 of the first embodiment, FIG. 5A is an explanatory diagram of an example of the superimposed image by a DRR image pseudo-transparent from one direction, and FIG. 5B is shown in FIG. 5A. An explanatory diagram of an example of a corresponding teacher image, FIG. 5C is an explanatory diagram of an example of a superimposed image by a pseudo perspective DRR image from two directions, and FIG. 5D is an explanatory diagram of an example of a teacher image corresponding to FIG. 5C. FIG. 6 is an explanatory diagram of an example of learning the position of the tracking target, FIG. 6A is an explanatory diagram of an example of a plurality of superimposed images, and FIG. 6B is an explanatory diagram when the superimposed images are superimposed. FIG. 7 is an explanatory diagram of an example of how to use the radiotherapy machine of Example 1. FIG. 8 is an explanatory diagram of the second embodiment and is a diagram corresponding to FIG. 2 of the first embodiment. FIG. 9 is a block diagram of the second embodiment and is a diagram corresponding to FIG. 3 of the first embodiment. FIG. 10 is an explanatory diagram of an example of the superimposed image and the teacher image of the second embodiment, FIG. 10A is an explanatory diagram of an example of the superimposed image by a DRR image pseudo-transparent from one direction, and FIG. 10B corresponds to FIG. 10A. An explanatory diagram of an example of a teacher image, FIG. 10C is an explanatory diagram of an example of a superimposed image by a pseudo perspective DRR image from two directions, and FIG. 10D is an explanatory diagram of an example of a teacher image corresponding to FIG. 10C.

Next, an embodiment which is a specific example of the embodiment of the present invention will be described with reference to the drawings, but the present invention is not limited to the following examples.
In addition, in the explanation using the following drawings, the illustrations other than the members necessary for the explanation are omitted as appropriate for the sake of easy understanding.

FIG. 1 is an explanatory view of a radiotherapy machine to which the tumor shape estimation device of Example 1 of the present invention is applied.
FIG. 2 is an explanatory view of a main part of the radiotherapy machine of the first embodiment.
In FIGS. 1 and 2, the radiotherapy machine (an example of the treatment device) 1 to which the target tracking device of the first embodiment of the present invention is applied has a bed 3 on which the patient 2 who is the subject of treatment sleeps. An X-ray irradiation device 4 for fluoroscopic photography is arranged below the bed 3. The X-ray irradiation apparatus 4 for fluoroscopy according to the first embodiment has two X-ray irradiation units (first channel 4a and second channel 4b). Each X-ray irradiation unit (second irradiation means and third irradiation means) 4a, 4b can irradiate the patient 2 with X-rays as an example of radiation for fluoroscopy.
Imaging devices (imaging means) 6a and 6b are arranged on opposite sides of the X-ray irradiation units 4a and 4b with the patient 2 in between. The imaging devices 6a and 6b receive X-rays transmitted through the patient and capture an X-ray fluoroscopic image. The image taken by the image pickup apparatus 6 is converted into an electric signal by the image generator 7 and input to the control system 8. In the following description, the direction from the first channel X-ray irradiation unit 4a toward the image pickup apparatus 6a is defined as the 0-direction. In that case, in this embodiment, the X-ray irradiation units 4a and 4b and the image pickup devices 6a and 6b are arranged so that the direction from the second channel X-ray irradiation unit 4b toward the image pickup device 6b is 90-. Is. This arrangement can be installed at an arbitrary angle to carry out the present invention according to the actual treatment site.

Further, a therapeutic radiation irradiator (first irradiation means) 11 is arranged above the bed 3. In this embodiment, the irradiation direction (Beam's Eye View direction) of the first irradiation means is arranged in the 45-direction from the therapeutic irradiation device 11 toward the tracking target 9 (target target for treatment such as tumor). .. This arrangement can also be installed at an arbitrary angle according to the actual treatment site to carry out the present invention. The therapeutic radiation irradiator 11 is configured so that a control signal can be input from the control system 8. The therapeutic radiation irradiator 11 is configured to be capable of irradiating X-rays as an example of therapeutic radiation toward a preset position (affected portion of patient 2) in response to input of a control signal. In Example 1, in the therapeutic irradiation device 11, an X-ray passing region (shape of an opening through which X-rays pass) is formed outside the X-ray source according to the outer shape of the tracking target 9 (tumor or the like). An MLC (Multi Leaf Collimator, not shown) is installed as an example of an adjustable diaphragm. The MLC is conventionally known and a commercially available one can be used, but the MLC is not limited to the MLC, and an arbitrary configuration in which the X-ray passing region can be adjusted can be adopted.
In Example 1, X-rays having an acceleration voltage on the order of kV (kilovolt) are irradiated as X-rays for fluoroscopy, and X-rays having an acceleration voltage on the order of MV (megavolt) are irradiated as X-rays for treatment. To.

(Explanation of the control system (control unit) of the first embodiment)
FIG. 3 is a block diagram showing each function provided in the control unit of the radiotherapy machine of the first embodiment.
In FIG. 3, the control unit C of the control system 8 has an input / output interface I / O that inputs / outputs signals to / from the outside. Further, the control unit C has a ROM: read-only memory in which a program, information, and the like for performing necessary processing are stored. Further, the control unit C has a RAM: random access memory for temporarily storing necessary data. Further, the control unit C has a CPU: a central processing unit that performs processing according to a program stored in a ROM or the like. Therefore, the control unit C of the first embodiment is composed of a small information processing device, a so-called microprocessor. Therefore, the control unit C can realize various functions by executing the program stored in the ROM or the like.

(Signal output element connected to control unit C)
The control unit C is input with output signals from a signal output element such as an operation unit UI, an image generator 7, and a sensor (not shown).
The operation unit (user interface) UI has a touch panel UI 0 as an example of a display unit and an example of an input unit. Further, the operation unit UI includes various input members such as a button UI 1 for starting learning processing, a button UI 2 for inputting teacher data, and a button UI 3 for starting fluoroscopic photography.
The image generator 7 inputs the image captured by the image pickup device 6 to the control unit C.

(Controlled element connected to control unit C)
The control unit C is connected to an X-ray irradiation device 4 for fluoroscopy, a radiation irradiator 11 for treatment, and other control elements (not shown). The control unit C outputs control signals thereof to the fluoroscopic X-ray irradiation device 4, the therapeutic radiation irradiator 11, and the like.
The fluoroscopic X-ray irradiation device 4 irradiates the patient 2 with X-rays for capturing an X-ray fluoroscopic image during learning or treatment.
The therapeutic radiation irradiator 11 irradiates the patient 2 with therapeutic radiation (X-rays) at the time of treatment.

(Function of control unit C)
The control unit C has a function of executing processing according to an input signal from the signal output element and outputting a control signal to each of the control elements. That is, the control unit C has the following functions.

FIG. 4 is an explanatory diagram of an example of processing in the control unit of the first embodiment.
C1: Learning image reading means The learning image reading means C1 reads (reads) an image input from the image generator 7. The learning image reading means C1 of the first embodiment reads the input image from the image generator 7 when the button UI1 for starting the learning process is input. In the first embodiment, the CT image is read for a preset learning period after the input of the button UI1 for starting the learning process is started. The learning image reading means C1 of Example 1 includes a tracking target image region 21 (a tumor image region as an example of the tracking target 9 which is a target of treatment) shown in FIG. 4, and a plurality of original images for learning (a plurality of original images). A vertical cross-sectional image (not shown) is constructed from the cross-sectional image) 22 to perform the following operations. Therefore, in the first embodiment, the learning-related means C2 to C10 execute image separation, editing, superimposition, and the like based on each image along the time series read and stored by the learning image reading means C1. .. That is, in the first embodiment, the learning process is not performed in real time for the acquisition of the CT image, but if the processing speed is improved by increasing the speed of the CPU or the like and the processing can be performed in real time, the learning process is performed in real time. Is also possible.

C2: Image separating means The image separating means C2 is a tracking target part image 23 including the tracking target image area 21 (a soft part as an example) based on the original image 22 including the learning three-dimensional information including the tracking target image area 21. Tissue DRR (Digitally Reconstructed Radiography) image) and separated background image (non-tracked image, separated non-tracked image) 24 (bone structure DRR image including image area of bone structure as an example) that does not include the tracking target image area 21. (Bone image)) and separate and extract. The image separation means C2 of the first embodiment has a tracking target image 23 (first image, soft tissue DRR image) and a separation background image 24 (bone structure DRR image) based on the CT value which is the contrast information of the CT image. Separate into and. In Example 1, as an example, a region having a CT value of 200 or more is used as a bone structure DRR image to form a separated background image 24, and a region having a CT value of less than 200 is used as a soft tissue DRR image to form a tracked portion image 23. ..
In Example 1, as an example, a tumor generated in the lung (tracking target 9), that is, a target target for treatment is shown as a soft tissue DRR image in the tracking target image 23, but for example, a tracking target. When is an abnormal part of the bone or the like, a bone structure DRR image is selected as the tracking target part image 23, and a soft tissue DRR image is selected as the separated background image 24. In this way, the selection of the tracking target portion image and the background image (non-tracking target image) is appropriately selected according to the background image including the tracking target and the obstacle.

C3: Obstacle image acquisition means The obstacle image acquisition means C3 includes an obstacle image 25 that is included in the case of acquiring the non-tracking target image 28 in the therapeutic radiation irradiator 11 and is different from the separated background image 24. get. In the first embodiment, as the obstacle image 25, an X-ray fluoroscopic image of the frame (couch frame) of the bed 3 and the fixture for fixing the patient to the bed 3 is stored in the storage medium in advance, and the obstacle image acquisition means C3. Reads and acquires an image (obstacle image 25) of a couch frame or a fixture stored in a storage medium.
C4: Random number generation means Random number generation means C4 generates random numbers.

C5: Background image editing means (non-tracking target image editing means, non-tracking target image creating means)
The background image editing means C5 edits at least one of the position, scaling, rotation, and light / darkness of the separated background image 24 and the obstacle image 25 with respect to the tracking target image 23 to create the background image (non-tracking target image) 29. This is an example of The background image editing means C5 of the first embodiment edits at least one of position, scaling, rotation, and light / darkness based on a random number. In the first embodiment, specifically, the affine transformation is performed to translate the separated background image 24 and the obstacle image 25, or the linear transformation (rotation, shearing, enlargement, reduction) is performed, and the affine is based on a random number. By changing each value of the transformation matrix, the separated background image 24 and the like are edited to create the background image 29. As for the light and darkness (contrast), the light and darkness of the separated background image 24 and the like is changed in the light or dark direction by an amount according to a random number. In the first embodiment, the background image editing means C5 creates 100 edited background images as an example of a preset number for one separated background image 24 and the like. That is, 100 background images 29 are created in which the positions of the separated background images 24 and the like are randomly edited by random numbers.

It is desirable that the number N of the background images 29 to be created is set based on the size of the image region of the tracking target image 23, the resolution of the image, and the preset tracking accuracy. As an example, when the image of the tracking target image 23 is 10 cm × 10 cm, the resolution is 1 mm, and the required tracking accuracy is 10 times the resolution, {10 (cm) / 1 (mm)} × 10 (times). ) = 1000 (sheets), it is also possible to create 1000 background images 29.

C6: Superimposed image creating means The superposed image creating means C6 superimposes a background image 29 (an image obtained by editing such as rotation on a separated background image 24 or an obstacle image 25) on a tracking target image 23, respectively. To create.
C7: Teacher image input receiving means The teacher image input receiving means C7 is a teacher as an example of an image that teaches an object to be tracked in response to an input to the touch panel UI0 or a button UI2 for inputting teacher data. The input of the teacher image 30 including the image area 27 is accepted. In the first embodiment, the original image (CT image) 22 for learning is displayed on the touch panel UI0, and the doctor inputs it on the screen so as to surround the image area of the tracking target which is the target target of the treatment, and the teacher It is configured so that the image area 27 can be determined.

FIG. 5 is an explanatory diagram of an example of the superimposed image 26 and the teacher image 30 of the first embodiment, FIG. 5A is an explanatory diagram of an example of the superimposed image by a DRR image pseudo-transparent from one direction, and FIG. 5B is shown in FIG. 5A. An explanatory diagram of an example of a corresponding teacher image, FIG. 5C is an explanatory diagram of an example of a superimposed image by a pseudo perspective DRR image from two directions, and FIG. 5D is an explanatory diagram of an example of a teacher image corresponding to FIG. 5C.
5A and 5B show, as an example, an X-ray transmission image taken in the 0-degree direction (0-degree direction from the upper X-ray irradiation means 4a in FIG. 2 toward the lower image pickup device 6a) for treatment. This corresponds to the case of irradiating X-rays in a direction tilted by 45 degrees (45 degree direction in FIG. 2, Beam's Eye View, BEV direction). With respect to the DRR superimposed image 26 (pseudo) fluoroscopically seen in the 0 degree direction shown in FIG. 5A, the tracking target 9 (tumor, etc.) in the region of the two-dimensional image in the 45 degree direction as shown in FIG. 5B. (Teacher image area 27) can be input as the teacher image 30.

5C and 5D are examples of taking an X-ray transmission image not only from the 0-degree direction but also from the 90-degree direction (horizontal direction in FIG. 2). As shown in FIG. 5D, as shown in FIG. 5D, the tumor in the region of the two-dimensional image in the 45-degree direction is compared with the pair of DRR superimposed images 26 that are pseudo-transparent from the 0-degree direction and the 90-degree direction shown in FIG. 5C. The contour (teacher image area 27) can be input as the teacher image 30.
In the configuration shown in FIGS. 5C and 5D, a DRR superimposed image and a teacher image can be set for each of the two fluoroscopic (photographing) directions of the 0 degree direction and the 90 degree direction, but two (or more) In the method of creating a set of superimposed images / teacher images (many), the relationship between the background images in each direction may be created independently, or may be created in conjunction with each other.

By creating the background images in each direction in conjunction with each other, it is possible to learn by partially interlocking the degrees of freedom of coordinates in a set of superimposed images and teacher images in different directions, and estimating the position and shape of the tracking target. It is possible to improve the accuracy. For example, in FIGS. 1 and 2, in the two imaging directions of the 0 degree direction and the 90 degree direction, the coordinates of the patient's body axis direction (direction from head to foot) are common to the two imaging means, so that the background image In the case of random editing, it is preferable to randomly operate both the body axis directions of the 0-degree direction image and the 90-degree direction image in conjunction with each other.
Therefore, in the first embodiment, shooting is performed from two directions, and processing is performed as shown in FIGS. 5C and 5D, but the present invention is not limited to this, and is also applicable to shooting from one direction as shown in FIGS. 5A and 5B. It is possible.
C8: Teacher image adding means The teacher image adding means C8 adds (further superimposes) the teacher image 30 including the input teacher image area 27 to the superimposed image 26.

FIG. 6 is an explanatory diagram of an example of learning the position of the tracking target, FIG. 6A is an explanatory diagram of an example of a plurality of superimposed images, and FIG. 6B is an explanatory diagram when the superimposed images are superimposed.
C9: Learning means (identifier creation means, correlation learning means)
The learning means C9 learns at least one of the area information and the position information of the tracking target image area 21 in the image based on the plurality of learning images 51 in which the teacher image 30 is added to the plurality of superimposed images 26, and is a classifier. To create. In the first embodiment, both the region and the position of the tracking target image region 21 are learned, and the fluoroscopic image from the X-ray irradiation direction (imaging direction, second direction, third direction) for transmission imaging is performed. Based on the outer shape of the tumor shown in, the correlation between the outer shape of the tumor shown in the fluoroscopic image and the outer shape of the tumor in the irradiation direction of X-rays for treatment (first direction, hereinafter referred to as the treatment direction) is learned and identified. Output the device (correlation information).

Further, in the first embodiment, the position of the center of gravity of the area of the tracking target image area 21 is set as the position of the tracking target image area 21, but the upper end, the lower end, the right end, the left end, etc. of the area are arbitrary depending on the design, specifications, and the like. It can be changed to the position.

In FIG. 6, in the learning means C9 of the first embodiment, in each of the plurality of superimposed images (see FIG. 6A) in which the images of the obstacle image region 31 whose positions and the like are randomly edited are superimposed on the tracking target image region 32. When the images are further superimposed, as shown in FIG. 6B, the tracked image area 32 whose position has not been edited is relatively emphasized (amplified), and the obstacle image area 31 whose position and the like are random is relatively emphasized. It will be suppressed (damped). Therefore, it is possible to learn the position and outer shape of the tracking target image area 32 in the CT image.

Further, the learning means C9 of the first embodiment captures a fluoroscopic image from the treatment direction in addition to the imaging direction before the treatment is started, for example, at the stage of formulating the treatment plan. Then, the correlation between the imaging direction and the treatment direction is learned in advance based on the images from the imaging directions (two directions) and the treatment direction. That is, when the tumor looks like an outer shape from the imaging direction, what the outer shape looks like in the treatment direction (correlation) is learned. The learning means C9 can adopt any conventionally known configuration, but it is preferable to use so-called deep learning (multi-layered neural network), and in particular, CNN (Convolutional Neural Network) is used. It is preferable to do so. In the first embodiment, Caffe is used as an example of deep learning, but the present invention is not limited to this, and any learning means (framework, algorithm, software) can be adopted.

When the number of samples is small, for example, as described in each means C2 to C9, the image of the soft tissue such as a tumor and the image of the bone are separated, and the image of the bone or the like is rotated, enlarged, or reduced. It has been explained that learning is performed by increasing the number of samples by generating an image in which the above is randomly performed and superimposing it on an image of a tumor or the like (random overlay), but the present invention is not limited to this. For example, it is possible to learn by using a sample whose outer shape is known in advance, and add an image of the tumor of patient 2 to deepen the learning.
It is also possible to separately generate a discriminator for tumor positioning and a discriminator for correlation, and use two discriminators.

C10: Learning result storage means The learning result storage means C10 stores the learning result of the learning means C9. That is, the learning-optimized CNN is stored as a discriminator.

C11: Fluoroscopic image reading means The fluoroscopic image reading means C11 reads (reads) an image input from the image generator 7. The fluoroscopic image reading means C11 of the first embodiment reads the image input from the image generator 7 when the button UI3 for starting fluoroscopic imaging is input. In the first embodiment, the X-rays emitted from the X-ray irradiation units 4a and 4b pass through the patient 2 and read the images taken by the imaging devices 6a and 6b.

C12: Tumor identification means (target identification means)
The tumor identifying means C12 identifies the outline of the target tumor based on the captured image. The outer shape of the tumor identifies the tumor based on the CT value, which is the contrast information of the CT image. As the specific method, it is possible to adopt a conventionally known image analysis technique, or it is also possible to read images of a plurality of tumors and perform learning to discriminate them. It is also possible to use the techniques described. The "outer shape" of the target is not limited to the outer shape of the tumor itself (= the boundary between the normal site and the tumor), and includes the case where it is set inside or outside the tumor at the discretion of a doctor or the like. That is, the "outer shape" can be a user-defined area. Therefore, the "target" is not limited to the tumor, and can be an area that the user wants to track.
The tumor identifying means C12 of Example 1 identifies the tumors appearing in the two captured images taken by the imaging devices 6a and 6b, respectively.

C13: External shape estimation means The external shape estimation means C13 estimates the external shape of the tumor in the treatment direction based on the correlation information and the external shape of the tumor reflected in the fluoroscopic image taken from the imaging direction most recently taken when the treatment is performed. Output the result.
C14: Irradiation means The irradiation means C14 controls the therapeutic radiation irradiator 11, and when the region and position of the tumor estimated by the external shape estimation means C13 are included in the irradiation range of the therapeutic X-ray. Irradiate with therapeutic X-rays. The irradiation means C14 of Example 1 controls the MLC according to the tumor area (outer shape) estimated by the outer shape estimation means C13, and the irradiation area (irradiation field) of the therapeutic X-ray is the tumor. Adjust so that it has an outer shape. When the therapeutic X-ray is irradiated, the irradiation means C14 controls the MLC in real time with respect to the output of the estimation result of the outer shape of the tumor which changes with the passage of time.

C15: Tracking accuracy evaluation means The tracking accuracy evaluation means C15 is based on an evaluation image (test image) in which at least one of the tracked area and the position is set in advance and a discriminator, and the tracking accuracy of the tracking target. To evaluate. In Example 1, as a test image, an image whose track target area and position are known is used, the track target is tracked by the classifier, and the track target area and position identified by the classifier and the known area are used. And evaluate the deviation from the position. For the area, for example, each pixel (pixel) on the outer edge of the area specified by the classifier and the pixel on the outer edge to be tracked in the test image are evaluated as (the number of pixels whose positions match) / (total pixel of the outer edge). If the number) is calculated and it exceeds a threshold (for example, 90%), it can be evaluated that the accuracy of the discriminator is sufficient for specifying the region. Similarly, regarding the position, if the position of the tracking target (center of gravity position) specified by the classifier is within the threshold value (for example, 5 pixels) with respect to the position of the tracking target in the test image, the classifier It can be evaluated that the accuracy of is sufficient for specifying the position. The evaluation method is not limited to the above, and any evaluation method can be adopted, for example, the region is evaluated by deriving the correlation coefficient between the outer shapes of the regions.

In the first embodiment, when the tracking accuracy evaluation means C15 evaluates that the accuracy is insufficient, the background image editing means C5 creates an edited image in which the number of images (2N) is twice the number N up to that point. Then, the superimposed image is additionally created using the added edited image, and the classifier is recreated by the learning means C9 using the increased superimposed image. Therefore, the accuracy of the recreated classifier is improved. It is also possible to automatically increase the number of edited images until the accuracy of the classifier reaches a preset threshold value, and to continue recreating the classifier, or to manually increase the number of edited images. And the classifier can be recreated.
The target external shape estimation device of the first embodiment is configured by the X-ray irradiation device 4 for fluoroscopic photography, the image pickup device 6, the image generator 7, the control system 8, and the means C1 to C15.

(Action of Example 1)
In the radiotherapy machine 1 of Example 1 having the above configuration, the correlation information between the image in the photographing direction and the image in the treatment direction is learned in advance, and at the time of treatment, the outer shape of the tumor in the treatment direction is estimated from the photographed image. Then, therapeutic X-rays are irradiated according to the estimated outer shape of the tumor.
In the conventional technique, a three-dimensional image is created from a two-dimensional photographed image, and a projected image in the treatment direction is derived from the three-dimensional image. It can be used to derive an image in the treatment direction from a two-dimensional captured image, and the processing time can be shortened. Therefore, it is possible to quickly estimate the position of the tumor (tracking target site), and the real-time property is improved. Therefore, the time lag during treatment is shorter than that of the prior art, and the tumor can be irradiated with X-rays with higher accuracy.

In particular, in Example 1, the correlation between the images taken from the two directions and the images in the treatment direction is used. It is also possible to use the correlation information between the image taken in one direction and the image in the treatment direction, but in Example 1, the outer shape of the tumor in the treatment direction is estimated as compared with the case of the image taken in one direction. The accuracy of is improved.
Further, in the first embodiment, since the learning is performed using the photographed image of the patient 2 to be treated, the accuracy is improved as compared with the case of using the photographed image of the third party.

Further, in Example 1, the number of samples to be learned in the patient 2 to be treated is increased by separating the images of the soft tissue and the bone and performing random overlay. In the conventional configuration of creating a three-dimensional image from a two-dimensional photographed image, such a thing is not performed, and a fixture such as a bed, a structure, or the like is reflected in the two-dimensional photographed image. If the structure or the like overlaps with the tumor, the estimation accuracy of the outer shape of the target such as the tumor will decrease, and the position of the bed or the patient will be different between the time of photography before the treatment day and the time of photography on the day of treatment. , The alignment may be performed based on the position of the bed, etc., and the estimation accuracy of the outer shape and position of the tumor, etc. may decrease (low robustness). On the other hand, in the present application, by increasing the number of samples and learning by random overlay, the influence of the bed or the like can be reduced, and the estimation accuracy of the outer shape and position of the tumor can be improved.
In addition, by increasing the number of samples with random overlay, the number of images taken at the time of treatment planning before the treatment date can be reduced. Therefore, it is also possible to reduce the dose of radiation received by the patient during the entire treatment of the day from prior imaging.

Further, conventionally, if it is attempted to perform learning using only the captured image of the patient himself / herself, the sample tends to be insufficient, and in some cases, the learning is performed using the captured image of the patient himself / herself. In this case, the discriminator created is not dedicated to the patient himself / herself, but becomes a general-purpose one in which the situations of many people are averaged, and there is a problem that the accuracy is lowered for the tumor of the patient himself / herself. On the other hand, in Example 1, it is possible to increase the number of samples by random overlay from the photographed image of the patient himself / herself, and since the classifier is created from the photographed image of the patient himself / herself, it is possible to improve the accuracy. Is.

Further, in the first embodiment, the learning is performed using the teacher image, and the time required for learning can be shortened as compared with the case where the teacher image is not used. When the teacher image is not used, if the number of samples is increased by random overlay, the same result can be obtained only by taking a long time. Therefore, it is possible to configure the configuration without using the teacher image.

Further, in the first embodiment, the tracking target image 23 (soft tissue DRR image) and the separated background image 24 (bone structure DRR image) are separated from the original image 22 based on the CT value, and edited and processed. , Learning according to the patient is possible.
Further, in the first embodiment, the brightness (contrast) is also edited when the separated background image 24 is edited. Therefore, even if the contrast is different between the time when the original image 22 for learning is taken and the time when the X-ray fluoroscopic image for treatment is taken, the situation where the contrast is different has already been learned, and it is easy to track. It is possible to further improve the tracking accuracy by learning using not only the contrast of the separated background image 24 but also the overall contrast of the original image 22.

Further, in the first embodiment, the separated background image 24 and the obstacle image 25 are randomly edited. For highly accurate learning, it is sufficient to have sufficient data evenly distributed, but it is troublesome and troublesome to prepare in advance without randomly editing. On the other hand, in the first embodiment, the separated background image 24 separated from the original image 22 and the obstacle image 25 added according to the therapeutic irradiation device 11 are randomly edited with random numbers to be uniform. It is possible to easily arrange a sufficient number of distributed data.

Generally, increasing the number of trained images by performing brightness conversion or geometric conversion of the original image is called "data expansion". Data expansion is used to prevent overfitting. This is for the purpose of preventing the generalization performance from being deteriorated due to over-learning of detailed features due to over-learning. However, since these linear transformations can be restored to the original image by a simple transformation, the image increase of up to several tens of times is the limit at most. In other words, the effect of non-linear transformation is indispensable for data expansion of several hundred times or more. In the first embodiment, the superimposed image 26 is a non-linear transformation as a whole. Therefore, it is considered that the generalization performance does not deteriorate even if the data is expanded several hundred times or more.

Further, in the first embodiment, when learning is performed, learning is performed on a CT image during a predetermined learning period. That is, learning is performed on a CT image, that is, a so-called 4DCT, in which an element of time (fourth dimension) is added in addition to an element of space (three dimensions). Therefore, it is possible to accurately track the position and outer shape of the tracking target, which fluctuates with time due to the time lag from imaging to irradiation and respiration. In the first embodiment, when learning with the random overlay, the situation where the tracking target is relatively deviated from the bone or the like is learned, so that the unexpected movement beyond the range of the respiratory movement acquired by 4DCT is learned. It can also handle situations (coughing and sneezing). In particular, in Example 1, when the soft tissue DRR image and the bone structure DRR image are superimposed by random overlay, learning is performed including the situation of an anatomically impossible combination, so that it is anatomically possible. It is easier to respond to unexpected movements than the conventional learning method that learns only by combination.
In addition, by making the CT image with the element of time added, even if the dose at the time of shooting is reduced and the image quality per image is lowered, the accuracy can be sufficiently improved by taking into account the images taken before and after the time. It is also possible to secure it.

Further, in the first embodiment, the tracking accuracy evaluation means C15 evaluates the created classifier. Therefore, it is possible to evaluate the accuracy of the created discriminator before actually performing the treatment of irradiating the radiation, and it is possible to confirm in advance whether the accuracy of the discriminator is sufficient or not before the day of the treatment. Then, if the accuracy is insufficient, it is possible to recreate the classifier. Therefore, treatment can be performed using a discriminator with sufficient accuracy, the accuracy of treatment is improved, and the patient may be exposed to extra radiation as compared with the case of using a discriminator with insufficient accuracy. Can be reduced.

FIG. 7 is an explanatory diagram of an example of how to use the radiotherapy machine of Example 1.
When radiation therapy is performed, a treatment plan is formulated in advance, and before the day of treatment when radiation is irradiated, the affected area is photographed in advance to diagnose the position and condition, and the image is taken before the treatment day. Based on the results, treatment on the day of treatment is also performed. In FIG. 7, before the treatment day, first, a fluoroscopic image of the patient is taken (ST1), and training data is generated based on a preset treatment plan (ST2) (ST3). Then, the CNN of the learning means C9 of the radiotherapy machine 1 is initialized (ST4), and new training (learning) of deep learning is performed based on the training data (ST5). The calculation time for new training is estimated to be about one hour. Then, the trained CNN is output (ST6).
On the day of treatment, a fluoroscopic image of the patient immediately before treatment is taken (ST11). The target is confirmed in the fluoroscopic image immediately before the treatment (ST12), and training data is generated (ST13). The radiotherapy machine 1 reads the CNN trained in ST6 (ST14) and retrains (learns) deep learning using the training data generated in ST13 (ST15). In the retraining, the trained CNN is used, so the calculation time is estimated to be about 10 minutes. Then, the retrained CNN is output (ST16).

When the treatment is started, a fluoroscopic image is taken in real time (ST21), and the position and outer shape of the target target are estimated by deep learning using the real-time fluoroscopic image and the retrained CNN (ST22). ). The calculation time for estimating the position and outer shape is estimated to be about 25 milliseconds per image. Then, therapeutic X-rays are irradiated according to the estimation result of the target target (ST23).
Position and outer shape estimation (ST22) and X-ray irradiation (ST23) are repeated until the treatment is completed (ST24).

When the radiotherapy machine 1 is used as in the example shown in FIG. 7, learning is performed in advance using the images taken before the day before the treatment, and retraining (re-learning) is performed using the images taken on the day of the treatment. ), Fine adjustment (= fine tuning) and readjustment can be performed according to the tumor whose size and shape change daily, and the accuracy of estimating the outer shape of the tumor is improved. In addition, when retraining is performed on the day, the training time can be shortened as compared with the initial training, the patient does not have to wait for a long time until the start of treatment, and the burden on the patient can be reduced.
In particular, in the technique of creating a three-dimensional (3D) image from a two-dimensional (2D) captured image as in the conventional configuration, fine tuning also takes a long time and the burden on the patient increases. Since it is the correlation between the 2D captured image and the image in the 2D treatment direction, the training time can be significantly shortened as compared with the case of creating a 3D image.
It is desirable to perform fine tuning, but it is also possible to perform treatment without fine tuning.

Further, in general medical CT (MDCT), X-rays are irradiated in a fan shape, whereas in CBCT (Cone Beam CT) widely used in dentistry and the like, a cone with a thin tip is used. X-rays are emitted. It is possible to take an image using CBCT, but since the range of X-ray irradiation in CBCT is narrower than that in MDCT, it takes 0.5 seconds in MDCT to take a picture around the patient once. It only takes about 1 minute for CBCT. Therefore, in CBCT, since it takes time to image, there is a problem that the image of organs that move due to breathing etc. is blurred, while bones that hardly move stand out at the time of imaging, so they are aligned with the image taken on the day of treatment. It also has the advantage of being easy to remove. Therefore, for example, it is possible to change such that MDCT is used during pretreatment imaging and treatment, and CBCT is used during fine tuning.

Next, a second embodiment of the present invention will be described. In the second embodiment, the components corresponding to the components of the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted. To do.
This embodiment differs from the first embodiment in the following points, but is configured in the same manner as the first embodiment in other respects.

FIG. 8 is an explanatory diagram of the second embodiment and is a diagram corresponding to FIG. 2 of the first embodiment.
In FIG. 8, the radiation therapy machine 1'of Example 2 is configured to be capable of irradiating the affected area of the patient 2 with a proton beam as an example of therapeutic radiation as a therapeutic radiation irradiator 11'.

(Explanation of the control system (control unit) of the second embodiment)
FIG. 9 is a block diagram of the second embodiment and is a diagram corresponding to FIG. 3 of the first embodiment.
In FIG. 9, in the control unit C'of Example 2, the learning means C9' has the outer shape of the tumor shown in the fluoroscopic image and the outer shape of the tumor in the treatment direction based on the outer shape of the tumor shown in the fluoroscopic image from the imaging direction. In addition to the correlation, the correlation with the distance from the therapeutic irradiation device 11'is also learned. That is, when the tumor appears to have an outer shape in each of the two fluoroscopic images, it is learned how far each position of the outer shape of the tumor is from the therapeutic irradiation device 11'(distance correlation information).

The distance estimation means C21 of the control unit C'of the second embodiment is a radiation irradiator for treating the tumor in the treatment direction based on the outer shape of the tumor shown in the fluoroscopic image taken most recently in which the treatment is performed and the correlation information. Estimate the distance from 11'.
Then, the irradiation means C14'of Example 2 controls the therapeutic radiation irradiator 11', and the region and position of the tumor estimated by the external shape estimation means C13 are included in the irradiation range of the therapeutic proton beam. In this case, the therapeutic proton beam is irradiated to the distance estimated by the distance estimation means C21.

FIG. 10 is an explanatory diagram of an example of the superimposed image and the teacher image of the second embodiment, FIG. 10A is an explanatory diagram of an example of the superimposed image by a DRR image pseudo-transparent from one direction, and FIG. 10B corresponds to FIG. 10A. An explanatory diagram of an example of a teacher image, FIG. 10C is an explanatory diagram of an example of a superimposed image by a pseudo perspective DRR image from two directions, and FIG. 10D is an explanatory diagram of an example of a teacher image corresponding to FIG. 10C.
In Example 2, when estimating the outer shape and distance of the tumor in the treatment direction, as shown in FIGS. 8, 10B, and 10D, in addition to the information in the range corresponding to the outer shape of the tumor, up to the deepest part of the target. Range (corresponding to the Range shown in FIG. 8. The (internal) distance from the body surface to the deepest boundary surface to be tracked is converted into the distance from when the charged particle enters the body to when it advances and stops in the water). The first teacher image 30a including the information 27a of the two-dimensional distribution (Range Map, target range distribution) of the above, and the two-dimensional distribution (Width Map) of the thickness (corresponding to the Width shown in FIG. 8) along the treatment direction of the target. It is preferable to add the second teacher image 30b including the information 27b of the target thickness distribution) because the learning accuracy is improved and the learning time is shortened.

(Action of Example 2)
In the radiotherapy machine 1'of Example 2 having the above configuration, when irradiating a proton beam, the ability (range) for protons to permeate differs between water, muscle, and bone. Therefore, compared to the case where the therapeutic radiation is X-rays, the information on how far the tumor is located (depth from the body surface to the inside of the body) when viewed from the treatment direction and the treatment direction. Information about how thick the tumor is is important. Therefore, as in Example 2, by deriving not only the shape and position of the outer shape but also the distance (range) and thickness to the target from the two captured images using the correlation information, highly accurate treatment can be performed. It can be carried out.

(Change example)
Although the examples of the present invention have been described in detail above, the present invention is not limited to the above-mentioned examples, and various modifications are made within the scope of the gist of the present invention described in the claims. It is possible. Examples of modifications (H01) to (H05) of the present invention are illustrated below.
(H01) In the above embodiment, a configuration in which an X-ray fluoroscopic image using kV-order X-rays is used as a captured image has been illustrated, but the present invention is not limited thereto. For example, an X-ray image using MV-order X-rays (an image in which a therapeutic MV-X-ray is detected after passing through a patient), an ultrasonic examination image (so-called ultrasonic echo image), and an MRI image (Magnetic Resonance Imaging). : Nuclear magnetic resonance imaging), PET image (Positron Emission tomography), photoacoustic imaging (PAI) image, X-ray backward scattering image, etc. can also be used. It is also possible to make changes such as combining an X-ray image (kV-X-ray or MV-X-ray) with an MRI image or the like. As the MRI image, an image of an arbitrary cross section can be output, but only an image of a cross section cut from a specific direction can be acquired. Therefore, the three-dimensional shape of the tumor is unknown only by adjusting the normal direction of the arbitrary cross section to the irradiation direction of the therapeutic radiation. Therefore, in order to obtain a three-dimensional shape, it is necessary to calculate the three-dimensional shape by combining images of a plurality of cross sections. On the other hand, by daringly shifting the normal direction of the MRI image from the irradiation direction of the therapeutic radiation and learning the correlation, the processing can be performed at a higher speed than in the case of calculating the three-dimensional shape. It is possible.
In addition, taking advantage of the deep correlation between the movement of breathing and the movement of the tumor, the body surface shape of the patient, which fluctuates due to breathing, is photographed with an imaging means such as a 3D camera or a distance sensor. It is also possible to estimate the position and outer shape of the tumor using the surface shape image.

(H02) In the above embodiment, a configuration using two captured images has been illustrated, but the present invention is not limited to this. Depending on the required accuracy, it is possible to shoot in one direction (one direction). It is also possible to shoot from three or more directions to further improve the accuracy.
(H03) In the above-described embodiment, a configuration in which tracking is performed without using a marker has been illustrated, but the present invention is not limited to this. It is also possible to use an X-ray image or the like in which a marker is embedded.
(H04) In the above-mentioned example, the case where there is one target tumor or the like has been described, but it can also be applied to the case where there are a plurality of target targets.
(H05) In the present embodiment, since the tracking target portion image 23, the separated background image 24, and the obstacle image 25 are two-dimensional DRR images, a large number of backgrounds randomly edited by the two-dimensional DRR image. An example of generating an image is shown. On the other hand, the tracking target part image 23, the separated background image 24, and the obstacle image 25 are left as the three-dimensional CT image, and a large number of randomly edited background images are generated as the three-dimensional CT image, and the three-dimensional CT is used. After superimposing on the tracking target portion image 23 as an image, a superposed image 26 corresponding to a unidirectional or multidirectional pseudo-transparent two-dimensional image may be created. In this case, since the random editing of the background image is operated as a three-dimensional CT image, the created background images and superimposed images in different directions are linked with all degrees of freedom (coordinates). As a result, the estimation accuracy of the position / shape by learning is improved as compared with the case where the background image is randomly edited with the 2D DRR image (the background image is randomly edited with the 2D DRR image sets in different multiple directions). In that case, since the three-dimensional information is partially lost, the degree of freedom (coordinates) that is not linked between the background images in different directions appears).

2 ... Subject,
4a ... Second irradiation means,
4b ... Third irradiation means,
6a ... Second imaging means (imaging apparatus),
6b ... Third imaging means (imaging apparatus),
9 ... Tracking target (target target for treatment)
11, 11'... First irradiation means,
C9, C9'... Learning means,
C13 ... External shape estimation means,
C21 ... Distance estimation means.

Claims (13)

It has a first irradiation means for irradiating a target in the body of a subject with a first radiation, and a second direction different from the first direction with respect to the first direction for irradiating the first radiation. An imaging means for capturing an image of the area including the target in the subject's body from the direction of
Based on the outer shape of the target reflected in the fluoroscopic image from the second direction, the classifier which is the correlation information is obtained by learning the correlation between the outer shape of the target reflected in the fluoroscopic image and the outer shape of the target in the first direction. Learning means to output and
An outer shape estimating means for estimating the outer shape of the target in the first direction based on the outer shape of the target reflected in the fluoroscopic image from the second direction taken most recently in which the treatment is performed and the discriminator.
A target contour estimation device characterized by being equipped with. The first radiation is an X-ray or a particle beam,
A claim characterized in that the fluoroscopic image is any one or a combination of an X-ray image, a nuclear magnetic resonance image, an ultrasonic inspection image, a positron emission tomography image, a body surface shape image, and a photoacoustic imaging image. Item 2. The target outer shape estimation device according to Item 1. Based on the outer shape of the target shown in the fluoroscopic image, the learning means that also learns the correlation between the outer shape of the target shown in the fluoroscopic image and the distance from the first irradiation means in the first direction.
A distance estimating means for estimating the distance of the target from the first irradiation means in the first direction based on the outer shape of the target reflected in the fluoroscopic image taken most recently in which the treatment is performed and the discriminator. When,
The target external shape estimation device according to claim 1 or 2, wherein the target shape estimation device is provided. An imaging means for capturing an image of a region containing a target in the subject's body from a third direction different from the second direction.
Based on the outer shape of the target reflected in the fluoroscopic image from the second direction and the outer shape of the target reflected in the fluoroscopic image from the third direction, the outer shape of the target reflected in each fluoroscopic image and the outer shape of the target in the first direction. The learning means that learns the correlation with the outer shape of the target and outputs a classifier,
The target outer shape estimation device according to any one of claims 1 to 3, wherein the target shape estimation device is provided. A superimposed image creating means for creating a plurality of superimposed images in which each of a plurality of non-tracked target images not including the image feature is superimposed on the tracked portion image including the target image feature.
Based on the outer shape of the target reflected in the perspective image from the second direction, the correlation between the outer shape of the target reflected in the perspective image and the outer shape of the target in the first direction is learned, and the plurality of superimposed images are displayed. Based on the learning means for creating a classifier by learning at least one of the image feature and the position information of the target.
The target outer shape estimation device according to any one of claims 1 to 4, wherein the target shape estimation device is provided. An input unit for inputting a teacher image for specifying the image feature of the target in advance,
The learning means for creating a classifier by learning at least one of the image feature and the position information of the target based on the plurality of superimposed images and the teacher image.
The target external shape estimation device according to claim 5, wherein the target shape estimation device is provided. Based on the size of the image feature of the target, the resolution of the image, and the preset tracking accuracy, the number of the non-tracking target images is derived, and the non-tracking target image according to the derived number of images is obtained. Non-tracked image editing means to create,
The target contour estimation device according to claim 5 or 6, wherein the target shape estimation device is provided. An image separation means for separating and extracting a tracking target part image including the target image feature and a separate non-tracking target image not including the tracking target image feature from a learning original image including the target image feature. ,
A non-tracking target image editing means for editing and creating the non-tracking target image composed of a plurality of separated non-tracking target images, and a non-tracking target image editing means.
With the superimposed image creating means for creating the superimposed image based on the tracked portion image and the non-tracked target image.
The target outer shape estimation device according to any one of claims 5 to 7, wherein the target shape estimation device is provided. An image separation means for separating an original image including the image feature of the target into a tracking target part image including the image feature of the target and a separation non-tracking target image not including the image feature of the target.
An obstacle image acquisition means for acquiring an image of an obstacle that is not included in the original image and is included in the non-tracking target image.
A non-tracking target image editing means for editing and creating a plurality of the non-tracking target image by editing at least one of the separated non-tracking target image and the obstacle image.
The superimposed image creating means for creating the superimposed image based on the tracked portion image and the non-tracked target image, and
The target outer shape estimation device according to any one of claims 5 to 7, wherein the target shape estimation device is provided. The image separation means for separating and extracting the tracking target portion image and the separation non-tracking target image from the original image for learning including the target image feature based on the image contrast information of the target image feature.
The target contour estimation device according to claim 8 or 9, wherein the target shape estimation device is provided. A tracking accuracy evaluation means for evaluating the tracking accuracy of the target based on the image for evaluation in which at least one of the image feature and the position information of the target is preset and the classifier.
The target outer shape estimation device according to any one of claims 5 to 10, wherein the target shape estimation device is provided. The learning means, which recreates the classifier by increasing the number of non-tracked images when the tracking accuracy by the tracking accuracy evaluation means does not reach a preset accuracy.
11. The target outer shape estimation device according to claim 11. The target outer shape estimation device according to any one of claims 1 to 12,
An irradiation means that irradiates therapeutic radiation from the first direction based on the outer shape of the target estimated by the target outer shape estimation device.
A treatment device characterized by being equipped with.

Images