WO2022070532A1 - 画像処理装置、画像処理装置の作動方法、画像処理装置の作動プログラム - Google Patents

画像処理装置、画像処理装置の作動方法、画像処理装置の作動プログラム Download PDF

Info

Publication number
WO2022070532A1
WO2022070532A1 PCT/JP2021/023596 JP2021023596W WO2022070532A1 WO 2022070532 A1 WO2022070532 A1 WO 2022070532A1 JP 2021023596 W JP2021023596 W JP 2021023596W WO 2022070532 A1 WO2022070532 A1 WO 2022070532A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
resolution
tomographic
interest
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/023596
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
有加里 奥村
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to EP21874836.6A priority Critical patent/EP4224414A4/en
Priority to JP2022553471A priority patent/JP7479494B2/ja
Publication of WO2022070532A1 publication Critical patent/WO2022070532A1/ja
Priority to US18/178,434 priority patent/US20230206397A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/025—Tomosynthesis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/46—Arrangements for interfacing with the operator or the patient
    • A61B6/461—Displaying means of special interest
    • A61B6/463—Displaying means of special interest characterised by displaying multiple images or images and diagnostic data on one display
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/502—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of breast, i.e. mammography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T12/00—Tomographic reconstruction from projections
    • G06T12/20—Inverse problem, i.e. transformations from projection space into object space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T12/00—Tomographic reconstruction from projections
    • G06T12/30—Image post-processing, e.g. metal artefact correction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30068—Mammography; Breast
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/436—Limited angle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/441—AI-based methods, deep learning or artificial neural networks

Definitions

  • the technology of the present disclosure relates to an image processing device, an operation method of the image processing device, and an operation program of the image processing device.
  • a technique for obtaining a plurality of tomographic images on an arbitrary tomographic surface of a subject such as tomosynthesis imaging in which the subject is irradiated with radiation from a plurality of different irradiation angles, is known.
  • the structure of the subject extending in the depth direction in which the tomographic planes are lined up can be separated and visualized for each tomographic image. Therefore, it is possible to depict a structure of interest such as a lesion, which was difficult to depict with a two-dimensional image which is a simple projection image of the subject.
  • US Patent Application Publication No. 2015/201890 describes a technique for generating a tomographic image (hereinafter abbreviated as high-resolution tomographic image) in which the entire area is high-resolution. Is described.
  • the image interpreting doctor when interpreting a tomographic image obtained by tomosynthesis imaging, the image interpreting doctor first interprets a two-dimensional image instead of immediately interpreting the tomographic image. Then, a two-dimensional image is used to roughly estimate the location of the lesion. After that, the tomographic image of the tomographic surface where the lesion with a guess is likely to exist is searched, and the searched tomographic image is interpreted in detail.
  • the reason for this interpretation procedure is that it is inefficient to randomly interpret multiple tomographic images without any guesswork at first.
  • the lesion is expressed in high definition by a high-resolution tomographic image, so that the interpretation is improved.
  • high resolution tomographic images take time to generate. For this reason, when the image interpreting doctor instructs to display the high-resolution tomographic image and starts the generation of the high-resolution tomographic image, it is regarded as a practical problem from the instruction to the actual display of the high-resolution tomographic image. There was a risk of a long time lag.
  • One embodiment according to the technique of the present disclosure is an image processing apparatus capable of shortening the time required for displaying a high-resolution tomographic image without increasing the resolution of the tomographic images of all the tomographic surfaces in advance.
  • An operation method of an image processing device, and an operation program of the image processing device are provided.
  • the image processing apparatus includes a processor and a memory connected to or built in the processor, and the processor represents a plurality of tomographic planes of a subject, respectively, and a plurality of tomographic images having a first resolution. Is acquired, and when an operation instruction regarding image interpretation is received from the user, a part of the tomographic image is set as the target area, and the process of setting the second resolution higher than the first resolution is performed only in the target area. By applying, a high-resolution partial image of the target area is generated and the high-resolution partial image is displayed.
  • the processor accepts an input instruction of a designated area for a two-dimensional image which is a projected image of the subject as an operation instruction, and sets a target area based on the designated area.
  • the processor performs a process of detecting the structure of interest in the designated area, and when the structure of interest is detected, the area including the structure of interest in the tomographic image is set as the target area.
  • the processor performs a process of detecting the structure of interest in the specified area, and if the structure of interest is not detected, the area corresponding to the specified area in the tomographic image of the predetermined set tomographic plane is set as the target area. Is preferable.
  • the subject is preferably the breast and the structure of interest is preferably at least one of a mass, spicula, calcification, and line structure.
  • tomographic images were obtained by tomosynthesis imaging in which the subject was irradiated with radiation from multiple different irradiation angles, and two-dimensional images were simple imaging in which the radiation source was directly facing the radiation detector and the radiation was irradiated. It is preferable to use either a simple two-dimensional image obtained by the above method or a composite two-dimensional image which is a pseudo simple two-dimensional image synthesized from a plurality of tomographic images using a composite image generation technique.
  • the processor performs a process of detecting a structure of interest in the two-dimensional image when the first display instruction of the two-dimensional image which is a projected image of the subject is received, and when the structure of interest is detected, the torpedo It is preferable to set the area including the interest structure detected in the image as the target area.
  • the subject is preferably the breast and the structure of interest is preferably at least one of a mass, spicula, calcification, and line structure.
  • tomographic images were obtained by tomosynthesis imaging in which the subject was irradiated with radiation from multiple different irradiation angles, and two-dimensional images were simple imaging in which the radiation source was directly facing the radiation detector and the radiation was irradiated. It is preferable to use either a simple two-dimensional image obtained by the above method or a composite two-dimensional image which is a pseudo simple two-dimensional image synthesized from a plurality of tomographic images using a composite image generation technique.
  • the processor applies the super-resolution method to the processing in which the first resolution is set to the second resolution.
  • the processor sets a plurality of target areas and generates a plurality of high-resolution partial images, it is preferable to generate the high-resolution partial image each time the second display instruction of the high-resolution partial image is received from the user.
  • the processor When the processor sets a plurality of target areas and generates a plurality of high-resolution partial images, the processor displays each high-resolution partial image of the plurality of target areas before receiving the second display instruction of the high-resolution partial image from the user. It is preferable to generate.
  • the processor displays a high-resolution partial image separately from the tomographic image in which the target area is set.
  • the processor synthesizes and displays a high-resolution partial image with an enlarged image that is simply enlarged according to the second resolution in an area other than the target area of the tomographic image in which the target area is set.
  • the processor displays the high-resolution partial image before the tomographic image in which the target area is not set.
  • the processor When the processor sets a plurality of target areas and generates a plurality of high-resolution partial images, the processor first displays one high-resolution partial image that satisfies a predetermined display condition among the plurality of high-resolution partial images. Is preferable.
  • the processor displays the high-resolution partial image when the second display instruction of the high-resolution partial image is received through the graphical user interface after receiving the operation instruction.
  • the method of operating the image processing device is a case where a plurality of tomographic planes of a subject are represented, a plurality of tomographic images having a first resolution are acquired, and an operation instruction regarding image interpretation is received from a user.
  • a high-resolution partial image of the target area is generated by setting a part of the tomographic image as the target area and performing a process of setting the second resolution higher than the first resolution only to the target area. And to display high resolution partial images.
  • the operation program of the image processing apparatus represents a plurality of tomographic planes of a subject, acquires a plurality of tomographic images having the first resolution, and receives an operation instruction regarding image interpretation from a user.
  • a high-resolution partial image of the target area is generated by setting a part of the tomographic image as the target area and performing a process of setting the second resolution higher than the first resolution only to the target area.
  • the computer perform processing including that and displaying a high-resolution partial image.
  • an image processing device and an image processing device capable of shortening the time required for displaying a high-resolution tomographic image without increasing the resolution of the tomographic images of all the tomographic surfaces in advance. It is possible to provide an operation method of the image processing device and an operation program of the image processing device.
  • FIG. 1 It is a schematic block diagram of the radiation imaging system to which the image processing apparatus of this disclosure is applied. It is a figure which looked at the mammography photographing apparatus from the direction of the arrow A of FIG. It is a figure which shows the hardware configuration of an image processing apparatus. It is a figure which shows the functional structure of an image processing apparatus. It is a figure for demonstrating acquisition of a projection image. It is a figure which shows the tomographic image group. It is a figure which shows the formation of the synthetic 2D image. It is a figure for demonstrating the formation of a synthetic 2D image in detail. It is a figure which shows the screen for designating a tomographic image group, and the screen which includes a synthetic 2D image.
  • FIG. 24 It is a figure which shows the screen which contains the simple magnified tomographic image which combined the high-resolution partial image shown in FIG. 24. It is a figure which shows the screen which provided the annotation addition button and the high resolution button. It is a figure which shows a mode that annotation is added to the place which selected an annotation addition button, moved the cursor and clicked. It is a figure which shows a mode that a designated area is set in the place which selected the high-resolution button, moved the cursor and clicked. It is a figure which shows the process of the interest structure detection part and the target area setting part of 2nd Embodiment. It is a figure which shows the screen of the 2nd Embodiment which includes a synthetic 2D image, and is provided with a high-resolution image display button. It is a figure which shows the processing procedure of the image processing apparatus in 2nd Embodiment.
  • FIG. 1 is a schematic configuration diagram of a radiation imaging system 100 to which the image processing device 4 according to the embodiment of the present disclosure is applied
  • FIG. 2 is a view of a mammography imaging device in the radiation imaging system 100 as viewed from the direction of arrow A in FIG. be.
  • the radiographic imaging system 100 according to the present embodiment performs tomosynthesis imaging of the breast M, which is an example of the "subject" according to the technique of the present disclosure, in order to generate a tomographic image of the breast M.
  • the radiographic imaging system 100 includes a mammography imaging apparatus 1, a console 2, an image storage system 3, and an image processing apparatus 4.
  • the mammography photographing apparatus 1 includes an arm portion 12 connected to a base (not shown) by a rotating shaft 11.
  • An imaging table 13 is attached to one end of the arm portion 12, and a radiation irradiation unit 14 is attached to the other end so as to face the photographing table 13.
  • the arm portion 12 is configured so that the photographing table 13 is fixed and only the end portion to which the radiation irradiation portion 14 is attached can be rotated.
  • a radiation detector 15 such as a flat panel detector is provided inside the photographing table 13.
  • the radiation detector 15 has a radiation detection surface 15A.
  • a charge amplifier that converts the charge signal read from the radiation detector 15 into a voltage signal
  • a correlated double sampling circuit that samples the voltage signal output from the charge amplifier, and a voltage signal.
  • a circuit board or the like provided with an AD (Analog Digital) conversion unit or the like for converting a digital signal into a digital signal is also installed.
  • the radiation source 16 is housed inside the radiation irradiation unit 14.
  • the radiation source 16 emits radiation such as ⁇ -rays and X-rays.
  • the timing of irradiating radiation from the radiation source 16 and the radiation generation conditions in the radiation source 16, that is, the selection of the material of the target and the filter, the tube voltage, the irradiation time, and the like are controlled by the console 2.
  • a compression plate 17 arranged above the imaging table 13 to press and press the breast M, a support portion 18 for supporting the compression plate 17, and a support portion 18 are vertically attached to FIGS. 1 and 2.
  • a moving mechanism 19 for moving in the direction is provided. The distance between the compression plate 17 and the photographing table 13, that is, the compression thickness is input to the console 2.
  • the console 2 displays a shooting order and various information acquired from an unillustrated RIS (Radiology Information System) or the like via a network such as a wireless communication LAN (Local Area Network), and instructions or the like directly given by an engineer or the like. It has a function of controlling the mammography photographing apparatus 1 by using the device. Specifically, the console 2 acquires a plurality of projected images as described later by causing the mammography imaging device 1 to perform tomosynthesis imaging of the breast M, reconstructs the plurality of projected images, and performs a plurality of tomographic images. To generate.
  • the server computer is used as the console 2.
  • the image storage system 3 is a system that stores image data such as a radiation image, a projection image, and a tomographic image taken by the mammography photographing apparatus 1.
  • the image storage system 3 takes out an image in response to a request from the console 2, the image processing device 4, and the like from the stored image, and transmits the image to the requesting device.
  • Specific examples of the image storage system 3 include PACS (Picture Archiving and Communication Systems).
  • the image processing device 4 is a computer such as a workstation, a server computer, and a personal computer, and has a CPU (Central Processing Unit) 21, a non-volatile storage 23, and a memory 26 as a temporary storage area.
  • the image processing device 4 includes a display 24 such as a liquid crystal display, an input device 25 such as a keyboard and a mouse, and a network I / F (Interface Face) 27 connected to a network (not shown).
  • a display 24 such as a liquid crystal display
  • an input device 25 such as a keyboard and a mouse
  • a network I / F Interface Face
  • the CPU 21, storage 23, display 24, input device 25, memory 26, and network I / F 27 are connected to the bus 28.
  • the CPU 21 is an example of a "processor” according to the technique of the present disclosure.
  • the memory 26 is an example of the “memory” according to the technique of the present disclosure.
  • the memory 26 may be built in the CPU 21.
  • the storage 23 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like.
  • the image processing program 22 installed in the image processing device 4 is stored in the storage 23 as a storage medium.
  • the CPU 21 reads the image processing program 22 from the storage 23, expands it into the memory 26, and executes the expanded image processing program 22.
  • the image processing program 22 is an example of the “operation program of the image processing device” according to the technique of the present disclosure.
  • the image processing program 22 is stored in a storage device of a server computer connected to the network or in a network storage in a state of being accessible from the outside, and is downloaded to a computer constituting the image processing device 4 in response to a request. Will be installed. Alternatively, it is recorded and distributed on a recording medium such as a DVD (Digital Versaille Disc) or a CD-ROM (Compact Disc Read Only Memory), and is installed in the computer constituting the image processing device 4 from the recording medium.
  • a recording medium such as a DVD (Digital Versaille Disc) or a CD-ROM (Compact Disc Read Only Memory)
  • FIG. 4 is a diagram showing a functional configuration of the image processing apparatus 4 according to the first embodiment.
  • the image processing device 4 includes an image acquisition unit 30, a composition unit 31, an instruction reception unit 32, an interest structure detection unit 33, a target area setting unit 34, a high resolution unit 35, and a display control unit 36.
  • the image processing device 4 includes an image acquisition unit 30, a composition unit 31, an instruction reception unit 32, an interest structure detection unit 33, a target area setting unit 34, and a high resolution unit 35. And functions as a display control unit 36.
  • the image acquisition unit 30 acquires a tomographic image from the console 2 or the image storage system 3 via the network I / F27. Further, the image acquisition unit 30 may acquire a projected image from the console 2 or the image storage system 3 via the network I / F27.
  • the console 2 controls the mammography imaging device 1 to cause the mammography imaging device 1 to perform tomosynthesis imaging.
  • the mammography imaging apparatus 1 moves the radiation source 16 to the respective radiation source positions of S1, S2, ..., Sn by rotating the arm portion 12 (see FIG. 1) around the rotation axis 11.
  • the radiation source 16 moves to each radiation source position in this way, the irradiation angle of the radiation to the breast M changes.
  • radiation is applied to the breast M, which is the subject, under predetermined imaging conditions for tomosynthesis imaging.
  • the radiation source position Sc is the radiation source position where the optical axis X0 of the radiation emitted from the radiation source 16 is orthogonal to the detection surface 15A of the radiation detector 15. That is, the radiation source position Sc is a position for simple radiography in which the radiation source 16 faces the radiation detector 15 and irradiates the radiation.
  • a simple two-dimensional image Gc0 may be acquired by irradiating the breast M with a higher dose of radiation than other radiation source positions at this radiation source position Sc.
  • the simple two-dimensional image Gc0 is an example of a "two-dimensional image" according to the technique of the present disclosure.
  • the console 2 generates a tomographic image that emphasizes the desired tomographic plane of the breast M by reconstructing a plurality of projected images Gi acquired by tomosynthesis imaging.
  • the console 2 uses a well-known back projection method such as a simple back projection method or a filter back projection method from a plurality of projected images Gi to a plurality of tomographic planes of the breast M as shown in FIG.
  • the plurality of tomographic images Dj constitute a tomographic image group SD which is three-dimensional volume data in the set three-dimensional space.
  • the pixel value of the tomographic image Dj has a larger value as the brightness is higher (that is, white) and a smaller value as the brightness is lower (that is, black).
  • the tomographic image group SD a plurality of tomographic images Dj are arranged along the depth direction of the tomographic plane in the breast M.
  • the coordinate positions of each pixel in each tomographic plane correspond to each other.
  • pixels having the same coordinate position in the tomographic plane are referred to as corresponding pixels.
  • the tomographic image Dj has the first resolution.
  • the first resolution is the resolution of the projected image Gi output by the radiation detector 15, and the in-fault plane in the three-dimensional space set when the tomographic image group SD is reconstructed from the projected image Gi by the back projection method or the like. It is determined according to the number of coordinate positions.
  • the console 2 directly transfers the generated tomographic image group SD to the image processing device 4, or transfers it to the image storage system 3.
  • the compositing unit 31 synthesizes a plurality of tomographic images Dj of the tomographic image group SD to generate a composite two-dimensional image CG1.
  • FIG. 7 is a diagram for explaining a method of generating a composite two-dimensional image CG1. As shown in FIG. 7, the compositing unit 31 synthesizes the corresponding pixels of the plurality of tomographic images Dj in the depth direction in which the tomographic planes of the tomographic images Dj are lined up (that is, the depth direction of the breast M). Generates a two-dimensional image CG1.
  • the synthetic two-dimensional image CG1 is an example of a "two-dimensional image" according to the technique of the present disclosure, like the simple two-dimensional image Gc0 described above.
  • the corresponding pixels of the tomographic image Dj of all the tomographic planes are used, and the average value of these pixels and the like are calculated. You may. It is also possible to use the corresponding pixels of some tomographic images Dj instead of all tomographic images Dj and use the average value of the pixel values of some of the pixels. For example, only the pixels of the three tomographic images D1, D2, and D3 of the three tomographic planes selected from all the tomographic images Dj may be used, and the average value thereof may be used as the pixel value.
  • the tomographic plane used for calculating the pixel value may be changed for each pixel of the composite two-dimensional image CG1. For example, for one pixel, only the pixels of the three tomographic images D1, D2, and D3 of the three fault planes are used, the average value of these is used as the pixel value, and for another pixel, the pixels of the two fault planes are used. Only the pixels of the two tomographic images D2 and D3 are used, and the average value or the like thereof is used as the pixel value.
  • the synthesis unit 31 selects a part of the tomographic image Dj in which the tomographic structure 40 exists from among the plurality of tomographic images Dj. Select. Then, the compositing unit 31 synthesizes using only the pixels of the selected tomographic image Dj.
  • the structures of interest 40A, 40B, and 40C are present only in the tomographic images D2 to D4 of the three tomographic planes.
  • the synthesis unit 31 uses only the pixels in which the interest structures 40A to 40C corresponding to the interest structure 40 exist among the three tomographic images D2 to D4, and is interested in each pixel of the composite two-dimensional image CG1. Pixels at the coordinate positions where the structure 40 exists are combined. Similarly, the portion of the breast M in which the structure of interest 40 does not exist is also synthesized using the pixels of the plurality of tomographic images Dj. It should be noted that the missing portion other than the breast M is synthesized by using, for example, all the pixels of the tomographic image Dj.
  • the compositing unit 31 corresponds to the generated synthetic two-dimensional image CG1 with the tomographic surface information DPI for each pixel indicating which tomographic image Dj was used for each pixel of the breast M of the synthetic two-dimensional image CG1. Attach and record.
  • the instruction receiving unit 32 receives various instructions input through the input device 25 by a user such as an image interpreting doctor. For example, the instruction receiving unit 32 receives an operation instruction regarding interpretation of the composite two-dimensional image CG1 and the tomographic image Dj from the user.
  • the operation instruction regarding image interpretation is an input instruction of the designated area SR (see FIG. 10) for the composite two-dimensional image CG1.
  • the instruction receiving unit 32 outputs the image information of the designated area SR to the interest structure detecting unit 33.
  • the display control unit 36 displays a screen 45 for designating the tomographic image group SD received from the console 2 or the image storage system 3 on the display 24.
  • the screen 45 is provided with an input box 46 for the file path of the tomographic image group SD, a reference button 47 for displaying an explorer, and a display button 48.
  • the image acquisition unit 30 requests the distribution of the tomographic image group SD in which the file path is input to the input box 46.
  • the image acquisition unit 30 acquires the tomographic image group SD distributed from the console 2 or the image storage system 3 in response to the distribution request.
  • the synthesis unit 31 generates a composite two-dimensional image CG1 from the tomographic image group SD.
  • the display control unit 36 switches the display from the screen 45 to the screen 49 including the composite two-dimensional image CG1. That is, the operation of the display button 48 is an example of the "first display instruction of the two-dimensional image" according to the technique of the present disclosure.
  • the user moves the mouse cursor 50 to the portion of interest of the breast M reflected in the composite two-dimensional image CG1 of the screen 49 and clicks.
  • the designated area SR including the clicked point is set on the composite two-dimensional image CG1. That is, the operation of moving the mouse cursor 50 to the portion of interest of the breast M and clicking the breast M is an example of the "input instruction of the designated area" according to the technique of the present disclosure.
  • the operation of moving the mouse cursor 50 to the portion of interest of the breast M and clicking the breast M is also an example of the "second display instruction of the high-resolution partial image" according to the technique of the present disclosure.
  • the simple two-dimensional image Gc0 may be displayed instead of the composite two-dimensional image CG1 and the input instruction of the designated area SR for the simple two-dimensional image Gc0 may be accepted.
  • the designated area SR is a square centered on the clicked point, and has a size of, for example, 500 pixels ⁇ 500 pixels.
  • the shape of the designated area SR is not limited to a rectangle, and may be any shape such as a circle.
  • the size of the designated area SR may be changed according to the size of the interest structure 40 detected by the interest structure detection unit 33.
  • the size of the designated area SR may be changed according to the type of the interest structure 40 detected by the interest structure detection unit 33. For example, if the structure of interest 40 is a mass 51 (see FIG. 11), the size of the designated area SR is 500 pixels ⁇ 500 pixels, and if the structure of interest 40 is calcification 53 (see FIG. 11), it is designated.
  • the size of the area SR is 50 pixels ⁇ 50 pixels.
  • the region surrounded by the contour of the interest structure 40 detected by the interest structure detection unit 33 may be designated as the designated region SR.
  • an area surrounded by the user freehand and having an arbitrary shape and size may be designated as a designated area SR.
  • the interest structure detection unit 33 performs a process of detecting the interest structure 40 for the designated area SR. Specifically, as shown in FIG. 11, the interest structure detection unit 33 detects the tumor 51, the spicula 52, the calcification 53, and the linear structure 54 contained in the breast M as the interest structure 40.
  • the linear structure is a mammary gland such as a leaflet or a duct.
  • the interest structure detection unit 33 outputs the information of the interest structure 40 such as the coordinates of the pixels of the detected interest structure 40 to the target area setting unit 34.
  • the interest structure detection unit 33 outputs to the target area setting unit 34 that the interest structure 40 has not been detected.
  • the structure detection unit 33 may detect all of the tumor 51, the spicula 52, the calcification 53, and the line structure 54, but it detects at least one of them. May be good.
  • the interest structure detection unit 33 detects the interest structure 40 from the designated region SR by using a known computer-aided diagnosis (CAD; Computer-Aided Diagnosis) algorithm.
  • CAD computer-aided diagnosis
  • the probability (likelihood) indicating that the pixel in the designated area SR is the interest structure 40 is derived, and the pixel whose probability is equal to or higher than the predetermined threshold value is detected as the interest structure 40.
  • the CAD algorithm is prepared for each type of interest structure 40. In the present embodiment, a CAD algorithm for detecting the mass 51, a CAD algorithm for detecting the spicula 52, a CAD algorithm for detecting the calcification 53, and an algorithm for detecting the linear structure 54 are prepared.
  • the technique for detecting the structure of interest 40 is not limited to the one using CAD.
  • the interest structure 40 is obtained from the designated area SR by filtering processing by a filter for detecting the interest structure 40, a detection model in which machine learning is performed by deep learning or the like to detect the interest structure 40, or the like. It may be something to detect.
  • the target area setting unit 34 sets a part of the tomographic image Dj as the target area OR.
  • the target region OR is a region to be subjected to a second resolution higher than the first resolution, that is, a high resolution treatment, and is a region including one or more pixels in each tomographic image Dj.
  • FIG. 12 shows a case where the interest structure detection unit 33 detects the interest structure 40 from the designated area SR.
  • the target area setting unit 34 sets the area including the structure of interest 40 in the tomographic image Dj as the target area OR. More specifically, the target area setting unit 34 receives the information of the interest structure 40 from the interest structure detection unit 33. Then, the tomographic plane information DPI of the pixel of the interest structure 40 represented by the information of the interest structure 40 is read out.
  • the target area setting unit 34 sets the same area as the designated area SR of the tomographic image Dj of the tomographic surface represented by the read tomographic surface information DPI as the target area OR.
  • FIG. 13 shows a case where the interest structure detection unit 33 does not detect the interest structure 40 from the designated area SR.
  • the target area setting unit 34 sets the area corresponding to the designated area SR in the tomographic image Dj of the preset tomographic surface as the target area OR. More specifically, the target area setting unit 34 receives from the interest structure detection unit 33 that the interest structure 40 has not been detected. The target area setting unit 34 sets the same area as the designated area SR of the tomographic image Dj of the set tomographic surface as the target area OR.
  • the set fault plane is not limited to the example central fault plane and one fault plane above and below it. It may be a fault plane at the center and a fault plane above and below the d (d is an integer of 2 or more). Further, the fault plane of the uppermost stage and the fault plane of the d + 1 plane below the fault plane may be used, or the fault plane of the lowermost stage and the fault plane of the d-1 plane above the fault plane may be used. Furthermore, all fault planes may be set as set fault planes.
  • the interest structure detection unit 33 does not detect the interest structure 40 from the designated area SR. For example, there is no interest structure 40 in the area where the user thinks that the interest structure 40 exists and clicks with the cursor 50. Alternatively, the interest structure 40 is not recognized, but the user clicks for confirmation.
  • the high resolution unit 35 performs a process of increasing the resolution of the target area OR from the first resolution to the second resolution.
  • the second resolution is higher than the first resolution, and the number of pixels is, for example, four times.
  • the high-resolution unit 35 does not perform high-resolution processing on the area of the tomographic image Dj other than the target area OR. In this way, the high-resolution unit 35 performs the high-resolution processing only on the target region OR to generate a high-resolution partial image HRP of the target region OR.
  • the high-resolution partial image HRP is an image in which the number of pixels is larger than that of the image of the original target region OR and the details such as the structure of interest 40 are expressed in detail.
  • the high-resolution unit 35 outputs the high-resolution partial image HRP to the display control unit 36.
  • FIG. 14 illustrates a case where the target region OR of the tomographic image D2 shown in FIG. 12 is subjected to high resolution processing.
  • the target region OR of the tomographic images D3 and D4 is also subjected to the processing of increasing the resolution.
  • high resolution processing is performed on each target region OR of the tomographic images DC, DC-1, and DC + 1.
  • the super-resolution method is applied as the process of increasing the resolution.
  • the super-resolution is a process in which the resolution exceeds the resolution set at the time of reconstructing the tomographic image Dj based on the projected image Gi.
  • a super-resolution method for example, the method described in Japanese Patent Application Laid-Open No. 2020-0257886 can be mentioned.
  • the super-resolution method described in JP-A-2020-205786 is a process using a trained model machine-learned so that an input image becomes a super-resolution image.
  • the trained model outputs a super-resolution image by adding a new pixel between the pixels of the input image and interpolating the pixel value of the added new pixel.
  • Such a trained model is constructed using, for example, a convolutional neural network, a recurrent neural network, a support vector machine, or the like.
  • the super-resolution method is not limited to the method described in the above-mentioned Japanese Patent Application Laid-Open No. 2020-0257886.
  • any high-order interpolation method such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation can be used.
  • it is also possible to use a method of extracting a small region (called a patch) that repeatedly appears from an image and super-resolution the original image using the pixel values of the extracted small region. can.
  • a high-resolution partial image HRP may be generated by using the tomographic images Dj adjacent to the top and bottom.
  • the target area OR set in the tomographic image D2 and the target area OR set in the tomographic image D4 are also used for high resolution.
  • the image of the target region OR in the tomographic images D2 to D4 is regarded as a three-dimensional image, and k-neighborhood interpolation is performed using k pixels located closest to the pixel to be interpolated in the target region OR of the tomographic image D3. .. Any value such as 6 or 26 can be used as the value of k.
  • FIG. 15 illustrates a case where 4 ⁇ 4 pixels are 8 ⁇ 8 pixels.
  • MGP is an image in which the pixel value of the original pixel is simply addressed to the pixel value of the increased pixel.
  • HRP does not simply address the pixel value of the original pixel to the pixel value of the increased pixel, but interpolates the pixel value of the increased pixel with the pixel value of the surrounding pixels.
  • the pixel 61 of the target area OR corresponds to the area 62 of 2 ⁇ 2 pixels in the simple enlarged image MGP and the high resolution partial image HRP, but the pixel of the area 62 of the simple enlarged image MGP has the same pixel value as the pixel 61. Is.
  • the pixels in the region 62 of the high-resolution partial image HRP may have the same pixel value as the pixel 61 or may have a different pixel value.
  • the method for increasing the resolution is used when reconstructing the tomographic image group SD as shown in FIG.
  • a method of using the projected image Gi a well-known back projection method such as a simple back projection method is used, and an image of the target region OR selected in the tomographic image Dj is used as a high-resolution partial image HRP from the projected image Gi. It is a method of reconstruction.
  • the high resolution unit 35 further arranges the coordinate positions Ps1, Ps2, between the coordinate positions P100, P101, P102, ... ... Is added, and the pixel values of the corresponding coordinate positions in the projected images G1 to G4 are back-projected to the added coordinate positions Ps1, Ps2, .... As a result, the pixel values are also calculated for the coordinate positions Ps1, Ps2, ... Added in the fault plane Tj.
  • the high-resolution unit 35 generates a high-resolution partial image HRP having a second resolution corresponding to the target region OR.
  • the display control unit 36 determines the display order of the high-resolution partial image HRP and the tomographic image Dj according to the display conditions 66A or 66B.
  • FIG. 17 shows a case where the interest structure detection unit 33 detects the interest structure 40 from the designated area SR, as in the case of FIG. 12.
  • the display control unit 36 determines the display order according to the display condition 66A.
  • the display condition 66A is the content of "maximum area of interest structure". Therefore, the display control unit 36 displays the high-resolution partial image HRP having the largest area of the captured interest structure 40, that is, the largest number of pixels of the captured interest structure 40, among the plurality of high-resolution partial image HRPs.
  • the order is decided to be the first place.
  • the display control unit 36 determines the display order of the high-resolution partial image HRP other than the high-resolution partial image HRP having the largest number of pixels of the captured interest structure 40 in descending order of the area of the captured interest structure 40. Further, the display control unit 36 does not set the target area OR, and the display order of the other tomographic image Dj that did not generate the high-resolution partial image HRP is inferior to that of the high-resolution partial image HRP. The tomographic image Dj of the upper tomographic plane is displayed in the higher order.
  • the display control unit 36 determines the display order of the high-resolution partial image HRP (D3) to be first.
  • the subsequent display order of the high-resolution partial image HRP is, for example, the order in which the area of the structure 40 of interest is large.
  • the display order of the other tomographic images Dj that did not generate the high-resolution partial image HRP is the order after the high-resolution partial image HRP, and the order before the tomographic image Dj of the upper tomographic surface is set. Will be done.
  • the display condition 66A may be such that the display order of the high-resolution partial image HRP in which the three-dimensional center of the structure of interest 40 exists is ranked first.
  • the display condition 66A sets the display order of the high-resolution partial image HRP generated from the tomographic image Dj of the uppermost or lowermost tomographic plane among the plurality of tomographic images Dj in which the same interest structure 40 exists. , May be the content.
  • FIG. 18 shows a case where the interest structure detection unit 33 does not detect the interest structure 40 from the designated area SR, as in the case of FIG.
  • the display control unit 36 determines the display order according to the display condition 66B.
  • the display condition 66B is the content of "reference fault plane among set fault planes". Therefore, the display control unit 36 determines the display order of the high-resolution partial image HRP generated from the tomographic image Dj of the reference tomographic plane to be the first among the plurality of high-resolution partial image HRPs.
  • the display control unit 36 displays the high-resolution partial image HRP other than the high-resolution partial image HRP generated from the tomographic image Dj of the reference fault plane in the display order as the high-resolution partial image HRP generated from the tomographic image Dj of the upper fault plane. Up. Further, as in the case of FIG. 17, the display control unit 36 does not set the target area OR, and the other tomographic image Dj that does not generate the high-resolution partial image HRP is higher than the high-resolution partial image HRP. The display order is subordinated, and the display order is increased as the tomographic image Dj of the upper tomographic plane is higher.
  • the display control unit 36 determines the display order of the high-resolution partial image HRP (DC) generated from the tomographic image DC of the central tomographic plane, which is the reference tomographic plane, in the first place.
  • HRP the high-resolution partial image
  • the higher the tomographic plane the earlier the order is set.
  • the display order of the other tomographic images Dj that did not generate the high-resolution partial image HRP is the order after the high-resolution partial image HRP, and the order before the tomographic image Dj of the upper tomographic surface is set. Will be done.
  • the reference fault plane may be the uppermost fault plane or the lowest fault plane.
  • the display control unit 36 displays the screen 71 shown in FIG. 19 on the display 24 when the high-resolution partial image HRP whose display order is determined to be the first is input from the high-resolution unit 35.
  • the composite two-dimensional image CG1, the tomographic image Dj, and the high-resolution partial image HRP are displayed side by side on the screen 71.
  • the target area OR is displayed in the tomographic image Dj that generates the high-resolution partial image HRP by designating the target area OR.
  • a feed button 72A and a return button 73A for sequentially displaying the tomographic images Dj of a plurality of tomographic planes are provided.
  • a feed button 72B and a return button 73B for sequentially displaying a plurality of high-resolution partial image HRPs one by one are provided below the high-resolution partial image HRP.
  • the instruction receiving unit 32 receives a display switching instruction of the tomographic image Dj by operating the feed button 72A and the return button 73A, and a display switching instruction of the high-resolution partial image HRP by operating the feed button 72B and the return button 73B.
  • the display switching instruction of the high-resolution partial image HRP by operating the forward button 72B and the return button 73B is an example of the “second display instruction of the high-resolution partial image” according to the technique of the present disclosure.
  • FIG. 19 shows an example in which the tomographic image D3 and the high-resolution partial image HRP (D3) generated from the tomographic image D3 are displayed.
  • the tomographic image Dj may not be displayed, and only the composite two-dimensional image CG1 and the high-resolution partial image HRP may be displayed.
  • the target area OR is displayed on the composite two-dimensional image CG1.
  • the high-resolution unit 35 receives a display switching instruction of the high-resolution partial image HRP by operating the feed button 72B and the return button 73B at the instruction receiving unit 32, and the high-resolution partial image HRP. To generate.
  • FIG. 20 shows the case where the feed button 72B is operated in the examples shown in FIGS. 12 and 17.
  • the display control unit 36 first displays the high-resolution partial image HRP (D3) on the screen 71.
  • the high-resolution section 35 generates the high-resolution partial image HRP (D4)
  • the display control section 36 generates the high-resolution partial image HRP (D4).
  • the image HRP (D4) is displayed on the screen 71.
  • the high-resolution section 35 When the feed button 72B is operated while the high-resolution partial image HRP (D4) is displayed, the high-resolution section 35 generates the high-resolution partial image HRP (D2), and the display control section 36 generates the high-resolution partial image HRP (D2).
  • the image HRP (D2) is displayed on the screen 71.
  • FIG. 21 shows a case where the return button 73B is operated while the high-resolution partial image HRP (D2) is displayed.
  • the high-resolution unit 35 generates the high-resolution partial image HRP (D4)
  • the display control unit 36 displays the high-resolution partial image HRP (D4) on the screen 71.
  • the return button 73B is operated while the high-resolution partial image HRP (D4) is displayed
  • the high-resolution section 35 generates the high-resolution partial image HRP (D3)
  • the display control section 36 generates the high-resolution partial image HRP (D3).
  • the image HRP (D3) is displayed on the screen 71.
  • the CPU 21 of the image processing device 4 has an image acquisition unit 30, a composition unit 31, an instruction reception unit 32, and an interest structure detection. It functions as a unit 33, a target area setting unit 34, a high resolution unit 35, and a display control unit 36.
  • the screen 45 shown in FIG. 9 is displayed on the display 24.
  • the instruction receiving unit 32 receives the first display instruction of the composite two-dimensional image CG1 (step). YES with ST100).
  • the distribution request of the tomographic image group SD in which the file path is input to the input box 46 is transmitted from the image acquisition unit 30 to the console 2 or the image storage system 3. Then, the tomographic image group SD distributed from the console 2 or the image storage system 3 in response to the distribution request is acquired by the image acquisition unit 30 (step ST110). The tomographic image group SD is output from the image acquisition unit 30 to the composition unit 31.
  • the composite unit 31 generates a composite two-dimensional image CG1 from the tomographic image group SD (step ST120).
  • the image storage system 3 is provided with a composition unit 31, and the composition two-dimensional image CG1 delivered from the image storage system 3 may be acquired by the image acquisition unit 30.
  • the composite two-dimensional image CG1 is output from the composite unit 31 to the display control unit 36.
  • the screen 49 including the composite two-dimensional image CG1 is displayed on the display 24 under the control of the display control unit 36 (step ST130). The user interprets the composite two-dimensional image CG1 through the screen 49.
  • the input instruction of the designated area SR is received by the instruction receiving unit 32 (step). YES in ST140). As a result, the designated area SR is set on the composite two-dimensional image CG1. The information of the designated area SR is output from the instruction receiving unit 32 to the target area setting unit 34.
  • the target area OR is set in the tomographic image Dj according to the designated area SR by the target area setting unit 34 (step ST150).
  • the image of the target area OR is output from the target area setting unit 34 to the high resolution unit 35.
  • the image of the target area OR is made high resolution by the high resolution unit 35, and a high resolution partial image HRP of the target area OR is generated (step ST160).
  • the high-resolution partial image HRP is output from the high-resolution unit 35 to the display control unit 36.
  • the screen 71 shown in FIG. 19 including the composite two-dimensional image CG1, the tomographic image Dj, and the high-resolution partial image HRP is displayed on the display 24 (step ST170).
  • the user interprets the tomographic image Dj and the high-resolution partial image HRP in detail through the screen 71.
  • the CPU 21 of the image processing device 4 functions as an image acquisition unit 30, an instruction reception unit 32, a target area setting unit 34, a high resolution unit 35, and a display control unit 36.
  • the image acquisition unit 30 represents each of the plurality of tomographic planes of the breast M, and acquires a plurality of tomographic images Dj having the first resolution.
  • the instruction receiving unit 32 receives an input instruction of the designated area SR as an operation instruction regarding image interpretation from the user.
  • the target area setting unit 34 sets a part of the tomographic image Dj, which corresponds to the designated area SR, as the target area OR. do.
  • the high-resolution unit 35 generates a high-resolution partial image HRP of the target region OR by performing a process of setting the second resolution higher than the first resolution only on the target region OR.
  • the display control unit 36 displays a high-resolution partial image HRP.
  • the area where the resolution is increased is limited to the target area OR. Therefore, it is possible to shorten the time required for displaying the high-resolution tomographic image (in this case, the high-resolution partial image HRP) as compared with the case where the entire region of the tomographic image Dj is made high-resolution. Further, since it is not necessary to increase the resolution of the tomographic images Dj of all the tomographic planes in advance, there is no problem that the storage capacity of the storage 23 is compressed.
  • the instruction receiving unit 32 receives an input instruction of the designated area SR for the synthetic two-dimensional image CG1 which is an example of the two-dimensional image which is the projected image of the breast M as the operation instruction regarding the image interpretation.
  • the target area setting unit 34 sets the target area OR based on the designated area SR. Therefore, it is possible to pinpoint and increase the resolution of the part of interest to the user.
  • the interest structure detection unit 33 performs a process of detecting the interest structure 40 with respect to the designated area SR.
  • the target area setting unit 34 sets the region including the interest structure 40 in the tomographic image Dj as the target area OR. Therefore, the structure of interest 40 can be made high resolution and read in detail.
  • the region corresponding to the designated region SR in the tomographic image Dj of the preset tomographic plane is set as the target region OR. Set as. Therefore, although the structure of interest 40 is not recognized, it is possible to deal with the case where the user clicks on the breast M portion of the synthetic two-dimensional image CG1 in order to take a look for confirmation. If the interest structure 40 is not detected by the interest structure detection unit 33, "the interest structure is" without setting the target area OR, generating the high-resolution partial image HRP, and displaying the high-resolution partial image HRP. A warning message display screen such as "Cannot be found. Please specify another part.” May be displayed on the screen 49.
  • the structure of interest 40 is at least one of a mass 51, a spicula 52, a calcification 53, and a line structure 54.
  • a few percent of the mass 51 may be malignant.
  • Spicula 52 is a characteristic finding in hard cancer and invasive lobular carcinoma.
  • Calcification 53 can become cancerous.
  • the line structure 54 is prone to lesions such as mass 51, spicula 52, and calcification 53. Therefore, if at least one of the tumor 51, the spicula 52, the calcification 53, and the line structure 54 is the structure of interest 40, more efficient interpretation can be performed.
  • the high-resolution unit 35 applies the super-resolution method to the process of setting the first resolution to the second resolution. Therefore, it is possible to perform image interpretation with an image in which details are expressed in very high definition.
  • the high-resolution partial image from the user is generated.
  • a high-resolution partial image HRP is generated each time the second display instruction of the HRP (operation of the forward button 72B and the return button 73B) is received.
  • the high resolution partial image HRP can be generated in a relatively short time. Therefore, there is no possibility that a long time lag that is regarded as a problem in practical use will occur from the second display instruction of the high-resolution partial image HRP to the actual display of the high-resolution partial image HRP.
  • the display control unit 36 displays the high-resolution partial image HRP separately from the tomographic image Dj in which the target area OR is set. Therefore, the target region OR can be observed in detail with the high-resolution partial image HRP while observing the entire breast M with the tomographic image Dj.
  • the display control unit 36 displays the high-resolution partial image HRP before the tomographic image Dj in which the target area OR is not set. Therefore, the high-resolution partial image HRP having a higher priority can be read first.
  • the display control unit 36 has a plurality of display control units 36.
  • the high-resolution partial image HRPs of the above one high-resolution partial image HRP that satisfies the predetermined display conditions 66A or 66B is displayed first. Therefore, the most important high-resolution partial image HRP can be read first.
  • a plurality of radiation sources 16 may be arranged at each source position Si, and radiation may be sequentially irradiated from the plurality of radiation sources 16 to capture a projected image Gi.
  • each of the plurality of target areas OR is not received every time the second display instruction of the high-resolution partial image HRP is received by the user, but before the second display instruction of the high-resolution partial image HRP is received.
  • High resolution partial image HRP may be generated.
  • FIG. 23 shows the case where the feed button 72B is operated in the examples shown in FIGS. 12 and 17.
  • the high-resolution unit 35 generates high-resolution partial images HRP (D2), HRP (D3), and HRP (D4) in advance.
  • the display control unit 36 first displays the high-resolution partial image HRP (D3) on the screen 71.
  • the display control unit 36 displays the high-resolution partial image HRP (D4) on the screen 71.
  • the display control unit 36 displays the high-resolution partial image HRP (D2) on the screen 71.
  • FIG. 23 shows a case where it is applied to the examples shown in FIGS. 12 and 17, but it is applied to the examples shown in FIGS. 13 and 18 in advance to obtain high-resolution partial images HRP (DC) and HRP. (DC-1) and HRP (DC + 1) may be generated.
  • the display mode shown in FIGS. 24 and 25 may be adopted.
  • the display control unit 36 simply enlarges the area other than the target area OR of the tomographic image Dj in which the target area OR is set according to the second resolution, and makes the tomographic image Dj a simple enlarged tomographic image. Let it be Dj_MGP. Then, the display control unit 36 synthesizes a high-resolution partial image HRP with the simple enlarged tomographic image Dj_MGP.
  • FIG. 24 illustrates a state in which a high-resolution partial image HRP including the interest structure 40B is synthesized with the simple enlarged tomographic image D3_MGP of the tomographic image D3 in which the target region OR is set in the interest structure 40B.
  • the display control unit 36 displays the screen 81 shown in FIG. 25 on the display 24 instead of the screen 71 shown in FIG.
  • the composite two-dimensional image CG1 and the simple enlarged tomographic image Dj_MGP in which the high-resolution partial image HRP is synthesized are displayed side by side.
  • FIG. 25 shows an example in which the simple magnified tomographic image D3_MGP is displayed.
  • the high-resolution partial image HRP can be observed on the tomographic image Dj. Similar to the screen 71 shown in FIG. 19, the high-resolution partial image HRP may also be displayed side by side on the screen 81.
  • the designated area SR is designated by moving the cursor 50 to the portion of interest by the user and clicking, and then various processes are performed to generate a high-resolution partial image HRP.
  • the high-resolution partial image HRP is displayed by the screen 71 shown in FIG. 19, but is not limited to this.
  • the embodiments shown in FIGS. 26 to 28 may be adopted.
  • the screen 86 displayed on the display 24 instead of the screen 49 shown in FIG. 10 is provided with an annotation button 87 and a high resolution button 88.
  • annotation button 87 When it is desired to add annotation 89 to the part of interest of the breast M shown in the synthetic two-dimensional image CG1, the user moves the mouse cursor 50 to the part of interest and clicks the annotation as shown in FIG. 27. Select the grant button 87. As a result, the annotation 89 can be added to the place where the cursor 50 is placed.
  • the user wants to see the high-resolution partial image HRP of the part of interest
  • the user puts the mouse cursor 50 on the part of interest and clicks on the part, and then presses the high-resolution button 88. select.
  • the designated area SR is set at the position where the cursor 50 is placed, and then various processes are performed to display the high-resolution partial image HRP.
  • the high resolution button 88 is an example of a "graphical user interface" according to the technique of the present disclosure.
  • the operation of the high-resolution button 88 is an example of the "second display instruction of the high-resolution partial image" according to the technique of the present disclosure.
  • processing other than increasing the resolution can be performed on the portion of interest.
  • the process other than increasing the resolution is not limited to the addition of the annotation 89, and may be, for example, a process of applying CAD that estimates the type of lesion to a portion clicked by moving the cursor 50.
  • the order of clicking the mouse and selecting the high resolution button 88 may be reversed. That is, after selecting the high-resolution button 88, the mouse cursor 50 may be placed on the portion of interest and clicked.
  • the target area OR is set based on the designated area SR set by the user's operation, but the present invention is not limited to this.
  • the second embodiment shown in FIGS. 29 to 31 may be adopted.
  • the interest structure detection unit 33 performs a process of detecting the interest structure 40 with respect to the composite two-dimensional image CG1 when the instruction receiving unit 32 receives the first display instruction of the composite two-dimensional image CG1.
  • the first display instruction of the composite two-dimensional image CG1 is the operation of the display button 48.
  • the interest structure 40 detected by the interest structure detection unit 33 is a tumor 51, a spicula 52, a calcification 53, and a line structure 54, as in the first embodiment.
  • a process of detecting the structure of interest 40 may be performed on the simple two-dimensional image Gc0.
  • the interest structure detection unit 33 sets the detection area DR including the detected interest structure 40.
  • the detection region DR is a rectangle that surrounds the detected interest structure 40 and whose center coincides with the center of the interest structure 40.
  • the detection region DR has a size that is one size larger than that of the structure of interest 40, for example, about 20% to 30%.
  • the interest structure detection unit 33 outputs the image information of the detection area DR to the target area setting unit 34.
  • FIG. 29 illustrates a case where the interest structures 40_1 and 40_1 are detected, the detection region DR_1 is set for the interest structure 40_1, and the detection region DR_1 is set for the interest structure 40_1.
  • the detection area DR may have a predetermined size as in the designated area SR.
  • the detection region DR may be a region surrounded by the contour of the interest structure 40 detected by the interest structure detection unit 33.
  • the target area setting unit 34 sets the area including the detected interest structure 40 in the tomographic image Dj, that is, the detection area DR as the target area OR. More specifically, the target area setting unit 34 identifies the tomographic plane in which the structure of interest 40 exists by referring to the tomographic plane information DPI, and detects the tomographic image Dj of the specified tomographic plane, as in the first embodiment. The same area as the area DR is set as the target area OR. In FIG.
  • the target regions OR_1 of the interest structures 40_1A, 40_1B, 40_1C, and 40_1D corresponding to the interest structure 40_1 are set for the detection region DR_1 of the interest structure 40_1, and the target regions OR_1 of the interest structure 40_1 are set with respect to the detection region DR_1 of the interest structure 40_1.
  • the case where the target region OR_2 of the interest structures 40_2A, 40_2B, and 40_2C corresponding to the interest structure 40_2 is set is illustrated. It should be noted that a region surrounded by the contour of the structure of interest 40, which is not the same region as the detection region DR, may be set as the target region OR.
  • a high-resolution image display button 92 is provided on the screen 91 including the composite two-dimensional image CG1.
  • the screen 71 including the high-resolution partial image HRP shown in FIG. 19 or the simple enlarged tomographic image Dj_MGP in which the high-resolution partial image HRP shown in FIG. 25 is synthesized is displayed.
  • the including screen 81 is displayed.
  • the high-resolution image display button 92 is an example of the "graphical user interface" according to the technique of the present disclosure.
  • the operation of the high-resolution image display button 92 is an example of the "second display instruction of the high-resolution partial image” according to the technique of the present disclosure.
  • the high-resolution partial image HRP to be displayed first is, for example, the high-resolution partial image HRP having the largest area of the structure 40 of interest, as in the case of FIG.
  • FIG. 31 is a flowchart for explaining the operation of the second embodiment. Similar to the first embodiment, the instruction receiving unit 32 receives the first display instruction of the composite two-dimensional image CG1 (YES in step ST100), and the image acquisition unit 30 acquires the tomographic image group SD (step ST110). .. Then, the composite 2D image CG1 is generated by the composite unit 31 (step ST120), and the screen 91 including the composite 2D image CG1 is displayed on the display 24 under the control of the display control unit 36 (step ST130).
  • the interest structure detection unit 33 performs a process of detecting the structure 40 of interest for the composite 2D image CG1.
  • the target region setting unit 34 sets the target region OR corresponding to the detection region DR including the structure of interest 40 in the tomographic image Dj (step ST210).
  • the high-resolution unit 35 increases the resolution of the image of the target area OR, and generates a high-resolution partial image HRP of the target area OR (step ST160).
  • the second display instruction of the high-resolution partial image HRP is received by the instruction receiving unit 32 (YES in step ST220). Then, under the control of the display control unit 36, the high-resolution partial image HRP is displayed on the display 24 (step ST170).
  • the interest structure detection unit 33 when the first display instruction of the synthetic two-dimensional image CG1 is received, the interest structure detection unit 33 performs a process of detecting the interest structure 40 with respect to the synthetic two-dimensional image CG1. .. Then, the target area setting unit 34 sets the area including the detected interest structure 40 in the tomographic image Dj as the target area OR. Therefore, the high-resolution partial image HRP can be displayed without performing the operation of moving the mouse cursor 50 to the portion of interest and clicking as in the first embodiment. Further, if the detection accuracy of the interest structure detection unit 33 is high, the interest structure 40 that may be overlooked by the user's eyes can be completely detected, and a high-resolution partial image HRP can be generated and displayed.
  • the high-resolution partial image HRP may be displayed on the screen 91 as soon as the high-resolution partial image HRP is generated without providing the high-resolution image display button 92. Further, when the high-resolution image display button 92 is selected, a process of detecting the structure of interest 40 may be performed.
  • the user may be able to select between a mode in which the user performs an operation of moving the mouse cursor 50 to a portion of interest in the first embodiment and clicking the position, and a mode in which the user performs the operation of clicking.
  • the tomographic plane information DPI may be generated by the following method. That is, the simple two-dimensional image Gc0 and the tomographic image Dj are divided into a plurality of regions (for example, a region of 2 pixels ⁇ 2 pixels). Then, the correlation between the region of the simple two-dimensional image Gc0 and the region of each tomographic image Dj is obtained, and the tomographic plane of the tomographic image Dj having a region having a relatively large correlation is used as a pixel of the region of the simple two-dimensional image Gc0. Record as the corresponding fault plane.
  • the tomographic plane information DPI may be generated by applying the above-mentioned method for obtaining the correlation.
  • the operation of the forward button 72A and the return button 73A is exemplified, and the operation for displaying a plurality of high-resolution partial image HRPs one by one in order.
  • the operation of the forward button 72B and the return button 73B has been exemplified, but the present invention is not limited thereto.
  • the tomographic image Dj of a plurality of tomographic planes and / or a plurality of high-resolution partial images HRP may be displayed in order.
  • the tomographic image Dj obtained by tomosynthesis imaging is exemplified, but the present invention is not limited to this.
  • CT Computer Planar Tomography
  • PET Positron Emission Tomography
  • SPECT Single Photon Emission Tomography
  • MRI Magnetic Image imaging
  • MRI Magnetic Image
  • various processes such as an image acquisition unit 30, a composition unit 31, an instruction reception unit 32, an interest structure detection unit 33, a target area setting unit 34, a high resolution unit 35, and a display control unit 36 are performed.
  • various processors Processors shown below can be used.
  • the CPU 21 which is a general-purpose processor that executes software (image processing program 22) and functions as various processing units, FPGA (Field Programmable Gate Array) and the like are manufactured.
  • a processor having a circuit configuration specially designed to execute a specific process such as a programmable logic device (PLC), an ASIC (Application Specific Integrated Circuit), which is a processor whose circuit configuration can be changed later. Dedicated electric circuit etc. are included.
  • PLC programmable logic device
  • ASIC Application Specific Integrated Circuit
  • One processing unit may be composed of one of these various processors, or may be a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs and / or a CPU). It may be configured in combination with FPGA). Further, a plurality of processing units may be configured by one processor.
  • one processor is configured by a combination of one or more CPUs and software, as represented by a computer such as a client and a server.
  • the processor functions as a plurality of processing units.
  • SoC System On Chip
  • SoC system On Chip
  • the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.
  • an electric circuit in which circuit elements such as semiconductor elements are combined can be used.
  • a and / or B is synonymous with "at least one of A and B". That is, “A and / or B” means that it may be A alone, B alone, or a combination of A and B. Further, in the present specification, when three or more matters are connected and expressed by "and / or", the same concept as “A and / or B" is applied.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Veterinary Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Human Computer Interaction (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Algebra (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
PCT/JP2021/023596 2020-09-29 2021-06-22 画像処理装置、画像処理装置の作動方法、画像処理装置の作動プログラム Ceased WO2022070532A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21874836.6A EP4224414A4 (en) 2020-09-29 2021-06-22 Image processing device, method for operating image processing device, and program for operating image processing device
JP2022553471A JP7479494B2 (ja) 2020-09-29 2021-06-22 画像処理装置、画像処理装置の作動方法、画像処理装置の作動プログラム
US18/178,434 US20230206397A1 (en) 2020-09-29 2023-03-03 Image processing device, method for operating image processing device, and program for operating image processing device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020163981 2020-09-29
JP2020-163981 2020-09-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/178,434 Continuation US20230206397A1 (en) 2020-09-29 2023-03-03 Image processing device, method for operating image processing device, and program for operating image processing device

Publications (1)

Publication Number Publication Date
WO2022070532A1 true WO2022070532A1 (ja) 2022-04-07

Family

ID=80949848

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/023596 Ceased WO2022070532A1 (ja) 2020-09-29 2021-06-22 画像処理装置、画像処理装置の作動方法、画像処理装置の作動プログラム

Country Status (4)

Country Link
US (1) US20230206397A1 (https=)
EP (1) EP4224414A4 (https=)
JP (1) JP7479494B2 (https=)
WO (1) WO2022070532A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025047368A1 (ja) * 2023-08-29 2025-03-06 ソニーグループ株式会社 情報処理装置、情報処理方法、および、コンピュータ読み取り可能な非一時的記憶媒体

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12450793B2 (en) * 2022-09-20 2025-10-21 United Imaging Intelligence (Beijing) Co., Ltd. Systems and methods for processing breast slice images through an artificial neural network to predict abnormalities in breasts

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006087921A (ja) * 2004-09-21 2006-04-06 General Electric Co <Ge> 対象領域情報を用いて連続的に多重解像度３次元画像を再構成する方法およびシステム
JP2009276163A (ja) * 2008-05-14 2009-11-26 Shimadzu Corp Ｘ線断層像撮影装置
JP2010011964A (ja) * 2008-07-02 2010-01-21 Toshiba Corp 医用画像処理装置および医用画像処理プログラム
JP2015006324A (ja) * 2013-05-27 2015-01-15 株式会社東芝 Ｘ線ｃｔ装置及び画像診断装置
US20150201890A1 (en) 2012-07-09 2015-07-23 The Trustees Of The University Of Pennsylvania Super-resolution tomosynthesis imaging systems and methods
JP2020025786A (ja) 2018-08-14 2020-02-20 富士フイルム株式会社 画像処理装置、方法及びプログラム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8634622B2 (en) * 2008-10-16 2014-01-21 Icad, Inc. Computer-aided detection of regions of interest in tomographic breast imagery
WO2014145452A1 (en) * 2013-03-15 2014-09-18 Real Time Tomography, Llc Enhancements for displaying and viewing tomosynthesis images

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006087921A (ja) * 2004-09-21 2006-04-06 General Electric Co <Ge> 対象領域情報を用いて連続的に多重解像度３次元画像を再構成する方法およびシステム
JP2009276163A (ja) * 2008-05-14 2009-11-26 Shimadzu Corp Ｘ線断層像撮影装置
JP2010011964A (ja) * 2008-07-02 2010-01-21 Toshiba Corp 医用画像処理装置および医用画像処理プログラム
US20150201890A1 (en) 2012-07-09 2015-07-23 The Trustees Of The University Of Pennsylvania Super-resolution tomosynthesis imaging systems and methods
JP2015006324A (ja) * 2013-05-27 2015-01-15 株式会社東芝 Ｘ線ｃｔ装置及び画像診断装置
JP2020025786A (ja) 2018-08-14 2020-02-20 富士フイルム株式会社 画像処理装置、方法及びプログラム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DANIEL GLASNER ET AL.: "Super-Resolution from a Single Image", ICCV, vol. 29, 2 October 2009 (2009-10-02)
See also references of EP4224414A4

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025047368A1 (ja) * 2023-08-29 2025-03-06 ソニーグループ株式会社 情報処理装置、情報処理方法、および、コンピュータ読み取り可能な非一時的記憶媒体

Also Published As

Publication number Publication date
JP7479494B2 (ja) 2024-05-08
JPWO2022070532A1 (https=) 2022-04-07
US20230206397A1 (en) 2023-06-29
EP4224414A1 (en) 2023-08-09
EP4224414A4 (en) 2024-03-13

Similar Documents

Publication Publication Date Title
JP7326070B2 (ja) 画像表示装置、方法およびプログラム、画像管理装置、方法およびプログラム
JP7203705B2 (ja) 画像処理装置、方法およびプログラム、並びに画像表示装置、方法およびプログラム
JP7384990B2 (ja) 画像管理装置、方法およびプログラム
US20230206397A1 (en) Image processing device, method for operating image processing device, and program for operating image processing device
JP5002344B2 (ja) 医用画像診断装置及び医用画像表示装置
US12488423B2 (en) Image processing device, method for operating image processing device, and program for operating image processing device
JP7209599B2 (ja) 画像処理装置、方法およびプログラム
US12272055B2 (en) Image processing device, image processing method, and image processing program
US12536649B2 (en) Image processing device, image processing method, and image processing program for facilitating image diagnosis using mammography apparatus to generate composite two-dimensional tomographic image using selected tomographic image slices corresponding to detected spicula regions
JP7270781B2 (ja) 画像設定装置、画像設定装置の作動方法および画像設定プログラム
US12419590B2 (en) Image processing device, image processing method, and image processing program
US12112479B2 (en) Image processing device, image processing method, and image processing program
US12307664B2 (en) Image processing device, image processing method, and image processing program
US20230146430A1 (en) Image processing device, image processing method, and image processing program
EP4524891A1 (en) Image processing apparatus, radiography system, and program
JP2025051943A (ja) 画像処理装置、放射線画像撮影システム及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21874836

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022553471

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021874836

Country of ref document: EP

Effective date: 20230502