Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
  • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/08—Auxiliary means for directing the radiation beam to a particular spot, e.g. using light beams
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/46—Arrangements for interfacing with the operator or the patient
  • A61B6/461—Displaying means of special interest
  • A61B6/463—Displaying means of special interest characterised by displaying multiple images or images and diagnostic data on one display
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/46—Arrangements for interfacing with the operator or the patient
  • A61B6/467—Arrangements for interfacing with the operator or the patient characterised by special input means
  • A61B6/469—Arrangements for interfacing with the operator or the patient characterised by special input means for selecting a region of interest [ROI]
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/48—Diagnostic techniques
  • A61B6/488—Diagnostic techniques involving pre-scan acquisition
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/54—Control of apparatus or devices for radiation diagnosis
  • A61B6/545—Control of apparatus or devices for radiation diagnosis involving automatic set-up of acquisition parameters
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/58—Testing, adjusting or calibrating thereof
  • A61B6/587—Alignment of source unit to detector unit
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/58—Testing, adjusting or calibrating thereof
  • A61B6/588—Setting distance between source unit and detector unit
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
  • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
  • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/44—Constructional features of apparatus for radiation diagnosis
  • A61B6/4429—Constructional features of apparatus for radiation diagnosis related to the mounting of source units and detector units
  • A61B6/4464—Constructional features of apparatus for radiation diagnosis related to the mounting of source units and detector units the source unit or the detector unit being mounted to ceiling
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
  • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
  • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
  • A61B6/5247—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20081—Training; Learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30196—Human being; Person

Definitions

• This disclosure relates to an information processing device, an information processing method, and an information processing program.
• JP 2014-117368 A discloses guidance that enables imaging to be reproduced at present under the same imaging conditions and with the same positioning as in past imaging, based on optical images and imaging conditions of a subject at a past time.
• the present disclosure provides an information processing device, an information processing method, and an information processing program that can support high-quality radiography.
• a first aspect of the present disclosure is an information processing device that includes at least one processor, which acquires at least one optical image obtained by optically photographing a subject, extracts feature points of the subject based on the optical image, identifies a target imaging area in the optical image that will be a target when the subject is radiographed from approximately the same direction as the imaging direction of the optical imaging based on the feature points, generates a superimposed image in which the target imaging area is superimposed on the optical image, and controls the display of the superimposed image.
• the processor may extract multiple feature points of the subject based on the optical image, and identify the target imaging area based on the relative positional relationship of the multiple feature points.
• the processor may identify at least one predetermined reference feature point from among the multiple feature points, and identify the target shooting area based on the reference feature point.
• the processor may obtain imaging site information indicating the imaging site of the radiograph, and identify reference feature points according to the imaging site information.
• the processor may generate a superimposed image using a trained model that has been trained in advance to receive at least one of an optical image and feature points as input and to receive a target shooting area or a superimposed image as output.
• the processor may issue a warning when a feature point other than a predetermined feature point among the multiple feature points is included in the target shooting area or is located within a predetermined range from the target shooting area.
• the processor may determine whether the subject is positioned in a predetermined manner based on the positional relationship of the multiple feature points, and may issue a warning if it is determined that the subject is not positioned in the predetermined manner.
• the processor may obtain at least one optical image for determination obtained by optically photographing the subject from a direction different from the photographing direction of the optical image, extract a plurality of characteristic points for determination of the subject based on the optical image for determination, determine whether the subject is positioned in a predetermined manner based on the positional relationship of the plurality of characteristic points for determination, and issue a warning when it is determined that the subject is not positioned in the predetermined manner.
• the optical image may be at least one of a visible light image and a distance image representing the distance to the subject.
• a second aspect of the present disclosure is an information processing method, which includes a process of acquiring at least one optical image obtained by optically photographing a subject, extracting feature points of the subject based on the optical image, identifying a target imaging area in the optical image that will be a target when the subject is radiographed from approximately the same direction as the imaging direction of the optical imaging based on the feature points, generating a superimposed image in which the target imaging area is superimposed on the optical image, and controlling the display of the superimposed image on a display.
• a third aspect of the present disclosure is an information processing program in which a computer executes a process of acquiring at least one optical image obtained by optically photographing a subject, extracting feature points of the subject based on the optical image, identifying a target imaging area in the optical image that will be a target when the subject is radiographed from approximately the same direction as the imaging direction of the optical imaging based on the feature points, generating a superimposed image in which the target imaging area is superimposed on the optical image, and controlling the display of the superimposed image on a display.
• the information processing device, information processing method, and information processing program disclosed herein can support high-quality radiography.
• FIG. 1 is a diagram illustrating an example of a schematic configuration of an imaging system.
• FIG. 2 is a schematic diagram showing an example of a usage mode of the imaging device.
• FIG. 2 is a block diagram showing an example of a hardware configuration of a console.
• FIG. 2 is a block diagram showing an example of a functional configuration of a console.
• FIG. 2 is a diagram showing an example of an optical image.
• FIG. 11 is a diagram illustrating an example of feature points.
• FIG. 4 is a diagram showing an example of a target imaging area.
• FIG. 13 is a diagram showing an example of an optical image for determination. 13 is an example of a screen displayed on a display. 11 is a flowchart illustrating an example of information processing.
• FIG. 1 is a diagram showing a schematic configuration of the imaging system 1.
• the imaging system 1 includes an imaging device 10 and a console 50.
• the imaging device 10 and the console 50, and the console 50 and an external RIS (Radiology Information System) is connected to each other.
• the ion System 6 is configured to be connectable via a wired or wireless network.
• the console 50 acquires imaging orders and the like from the RIS 6, and controls the imaging device 10 in accordance with the acquired imaging orders and user instructions.
• the imaging device 10 acquires a radiation image of the subject H in accordance with the control of the console 50.
• the console 50 is an example of an information processing device of the present disclosure.
• FIG. 2 is a diagram showing the schematic configuration of the imaging device 10.
• the imaging device 10 includes a radiation irradiation unit 12, a radiation detector 20, a first optical camera 26, and a second optical camera 28.
• FIG. 2 shows, as an example, a state in which the chest of subject H is radiographed as the imaging site.
• the radiation irradiation unit 12 includes a radiation source 13 that irradiates radiation R, such as X-rays.
• the radiation irradiation unit 12 also includes a collimator (not shown) and is configured to be able to change the irradiation field (the range shown by the two-dot chain line in FIG. 2) of the radiation R irradiated from the radiation source 13.
• the type of radiation source 13 is not particularly limited, and for example, a hot cathode type, cold cathode type, or other radiation source can be used as appropriate.
• the radiation irradiation unit 12 may be a so-called ceiling-traveling type irradiation unit that is supported by a support suspended from the ceiling of the imaging room.
• a ceiling-traveling type irradiation unit has a support that can expand and contract in the vertical direction (Z direction) attached via wheels to a rail that runs around the ceiling, and can move horizontally (X direction and Y direction) within the imaging room.
• the horizontal movement and vertical expansion and contraction of the support causes the radiation irradiation unit 12 to translate horizontally and vertically.
• the radiation irradiation unit 12 may be rotatable around a rotation axis that extends horizontally, or may be rotatable around a rotation axis that extends vertically.
• the radiation irradiation unit 12 may be a portable irradiation unit.
• the portable irradiation unit may be used, for example, for simple radiation testing in medical facilities, radiation testing during home medical care, outdoor radiation testing, and on-site medical care in disaster areas or medically underserved areas.
• the radiation irradiation unit 12 may be a stationary irradiation unit installed in an imaging room.
• the radiation detector 20 detects the radiation R that has passed through the subject H on the detection surface 20A, generates a radiation image based on the detected radiation R, and outputs image data representing the generated radiation image.
• the radiation detector 20 may be, for example, a portable electronic cassette that can be placed on any pedestal or held by the subject H. In other words, the radiation detector 20 may be movable to any position in the horizontal direction (X direction and Y direction) and vertical direction (Z direction) relative to the radiation irradiation unit 12. Also, for example, the radiation detector 20 may be a stationary type that is placed inside an imaging table installed in an imaging room.
• the type of radiation detector 20 is not particularly limited. For example, it may be an indirect conversion type radiation detector that converts radiation R into light and then converts the converted light into an electric charge, or it may be a direct conversion type radiation detector that directly converts radiation R into an electric charge.
• the first optical camera 26 and the second optical camera 28 are optical digital cameras that capture images based on visible light and are configured to include, for example, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor or a CCD (Charge Coupled Device) type image sensor.
• CMOS Complementary Metal Oxide Semiconductor
• CCD Charge Coupled Device
• the first optical camera 26 captures an image of an area (the area shown by the dashed line in FIG. 2) that is larger than the irradiation field of the radiation R (the area shown by the dashed line in FIG. 2) to generate an optical image 30.
• the angle of view ⁇ of the first optical camera 26 is stored in advance in the storage unit 52.
• the shooting direction of the optical shooting by the first optical camera 26 and the shooting direction of the radiation shooting using the radiation irradiation unit 12 and the radiation detector 20 are substantially the same direction.
• substantially the same direction may include a deviation that allows alignment with the radiation image by performing image correction (geometric transformation) such as affine transformation and projective transformation on the optical image 30.
• the position of the first optical camera 26 is not particularly limited, and may be attached, for example, to a surface approximately flush with the radiation R irradiation opening of the radiation irradiation unit 12 as shown in FIG. 2, or may be attached to a wall surface of the imaging room. However, since it is preferable to be able to optically image the entire subject H in order to identify the joint points described below, it is preferable that the first optical camera 26 is attached to a surface approximately flush with the radiation R irradiation opening and below the radiation R irradiation opening. In addition, it is preferable that the optical axis Ao of the first optical camera 26 is approximately parallel to the irradiation axis Ar of the radiation R irradiated from the radiation source 13.
• the positional relationship between the radiation source 13 and the first optical camera 26 is determined in advance. As shown in FIG. 2, the positional relationship is represented, for example, by a distance dz in the Z direction and a distance dx in the X direction (not shown) between the irradiation axis Ar of the radiation R irradiated from the radiation source 13 and the optical axis Ao of the first optical camera 26, and a distance dy in the Y direction between the radiation source 13 and the first optical camera 26.
• the distances dx, dy, and dz that represent these positional relationships, and the angle of view ⁇ of the first optical camera 26 are stored in advance in the storage unit 52.
• the second optical camera 28 optically captures the subject H from a direction different from the direction in which the optical image 30 is captured by the first optical camera 26, and generates an optical image for determination 34 for determining whether the subject H is positioned as determined in advance (details will be described later).
• FIG. 2 an example is shown in which the first optical camera 26 captures the subject H from behind, while the second optical camera 28 captures the subject H from above.
• the imaging device 10 may also include a control device (not shown) that controls the overall operation of the imaging device 10 in response to instructions from the console 50 and the user. Specifically, the control device acquires image data representing a radiation image generated by the radiation detector 20, and outputs the image data to the console 50. The control device also acquires an optical image 30 of the subject H captured by the first optical camera 26 and an optical image for determination 34 of the subject H captured by the second optical camera 28, and outputs the image data to the console 50.
• a control device (not shown) that controls the overall operation of the imaging device 10 in response to instructions from the console 50 and the user. Specifically, the control device acquires image data representing a radiation image generated by the radiation detector 20, and outputs the image data to the console 50. The control device also acquires an optical image 30 of the subject H captured by the first optical camera 26 and an optical image for determination 34 of the subject H captured by the second optical camera 28, and outputs the image data to the console 50.
• the control device may include, for example, a central processing unit (CPU), a read-only memory (ROM), The control device includes a RAM (Random Access Memory), a storage medium, an I/F (Interface) section, an operation section, etc. (not shown).
• the control device transmits and receives various information to and from a console 50 via the I/F section.
• radiography it is desirable to obtain high-quality radiographic images in which the imaging site of subject H is properly imaged. To achieve this, it is important to align the radiation irradiation unit 12 (radiation source 13), radiation detector 20, and subject H. This alignment is particularly important when at least one of the radiation irradiation unit 12 and the radiation detector 20 is movable, as described above. In addition, subject H is required to be positioned in a posture that is predetermined by guidelines, etc.
• the console 50 utilizes the optical image 30 obtained by the first optical camera 26 and the determination optical image 34 obtained by the second optical camera 28 to assist in high-quality radiation imaging.
• the console 50 includes a CPU (Central Processing Unit) 51, a non-volatile storage unit 52, and a memory 53 as a temporary storage area.
• the console 50 also includes a display 54 such as a liquid crystal display, an operation unit 55 such as a touch panel, a keyboard, and a mouse, and an I/F (Interface) unit 56.
• the I/F unit 56 is a unit for controlling the imaging device 10, the RI, and the like. S6 and other external devices, etc.
• the CPU 51, the storage unit 52, the memory 53, the display 54, the operation unit 55, and the I/F unit 56 are connected via a bus 58 such as a system bus and a control bus so as to be able to transmit and receive various information to and from each other.
• a bus 58 such as a system bus and a control bus so as to be able to transmit and receive various information to and from each other.
• the storage unit 52 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory.
• An information processing program 57 for the console 50 is stored in the storage unit 52.
• the CPU 51 reads the information processing program 57 from the storage unit 52, expands it in the memory 53, and executes the expanded information processing program 57.
• the CPU 51 is an example of a processor of the present disclosure.
• a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, etc. can be appropriately applied as the console 50.
• the console 50 includes an acquisition unit 60, an extraction unit 61, an identification unit 62, a generation unit 63, a determination unit 64, and a control unit 65.
• the CPU 51 executes the information processing program 57, the CPU 51 functions as each of the functional units of the acquisition unit 60, the extraction unit 61, the identification unit 62, the generation unit 63, the determination unit 64, and the control unit 65.
• the target imaging region is a region including a target part of radiation imaging, and for example, the target imaging region is near the chest when imaging the chest, near the knee when imaging the knee joint, and near the head when imaging the head.
• the acquisition unit 60 acquires at least one optical image 30 obtained by optically photographing the subject H with the first optical camera 26.
• FIG. 5 shows an example of the optical image 30 captured by the first optical camera 26 in FIG. 2. In the optical image 30 in FIG. 5, the subject H is photographed from the rear side.
• the extraction unit 61 extracts feature points of the subject H based on the optical image 30 acquired by the acquisition unit 60.
• the multiple feature points P1L-P6L and P1R-P6R extracted from the optical image 30 in FIG. 5 are indicated by black dots.
• the feature points P1L-P6L and P1R-P6R correspond to the joint points of the subject H, namely, the ears, shoulders, elbows, wrists, hips, and knees.
• feature points P when the multiple feature points P1L-P6L and P1R-P6R are not differentiated, they will be simply referred to as "feature points P", and the feature points corresponding to each joint point will be referred to as "feature points of (joint point name)".
• feature points P feature points of (joint point name)
• known posture estimation techniques can be appropriately applied as a method for extracting the feature points P (joint points).
• the identification unit 62 identifies a target imaging area 90 in the optical image 30 that is a target when radiography of the subject H is performed from approximately the same direction as the imaging direction of the optical imaging, based on the feature points P extracted by the extraction unit 61.
• the target imaging area 90 identified from the feature points P in the optical image 30 in FIG. 7 is shown by a solid-line rectangle.
• the extraction unit 61 extracts multiple feature points P of the subject H based on the optical image 30, and the identification unit 62 identifies the target imaging area 90 based on the relative positional relationship of the multiple feature points P.
• the identification unit 62 first identifies at least one predetermined reference feature point from the multiple feature points P1L to P6L and P1R to P6R.
• the identification unit 62 identifies the shoulder feature points P2L and P2R and the waist feature points P5L and P5R as the reference feature points. It can be determined which feature points P are the shoulder feature points P2L and P2R and the waist feature points P5L and P5R based on the relative positional relationship of the multiple feature points P.
• Which feature point P is to be used as the reference feature point may be determined in advance, may be arbitrarily set by the user, or may correspond to the body part being imaged.
• eye feature points and ear feature points are suitable reference feature points when imaging the head
• waist feature points, knee feature points, and ankle feature points are suitable reference feature points when imaging the knee joint. Therefore, the identification unit 62 may obtain imaging body part information indicating the imaging body part of the radiography included in the imaging order obtained from the RIS 6 or the like, and identify the reference feature point corresponding to the obtained imaging body part information.
• the type of reference feature point corresponding to the imaging body part information may be stored in advance in the storage unit 52, for example.
• the identification unit 62 identifies the target imaging region 90 based on the identified reference feature points.
• the identification unit 62 calculates a distance d in the craniocaudal direction (Z direction) between the midpoint of the line segment (shown by a dotted line) connecting the shoulder feature points P2L and P2R and the midpoint of the line segment (shown by a dotted line) connecting the waist feature points P5L and P5R.
• the identification unit 62 also calculates a distance Zd by multiplying the distance d by a predetermined coefficient, and identifies a point Q (shown by a star) that is the distance Zd away in the head direction from the midpoint of the line segment connecting the waist feature points P5L and P5R.
• the coefficient used to calculate the distance Zd may be arbitrarily determined by the user, for example, anatomically and/or statistically. Also, for example, the coefficient may be derived by a machine learning model that is previously trained by unsupervised learning using a combination of the learning optical image 30 and the target imaging region 90 as training data. In the case of chest imaging, the point Q corresponds to the vertebral column.
• the identification unit 62 specifies a rectangular region having the identified point Q as the center of the upper side and a size corresponding to the detection surface 20A of the radiation detector 20 as the target imaging region 90.
• the size of the target imaging region 90 i.e., the size corresponding to the detection surface 20A of the radiation detector 20
• SID Source to Image Receptor Distance
• the value of SID is For example, a SID that is pre-stored in the storage unit 52 or the like may be used on the premise that the image capturing device 10 is used in a state in which a predetermined appropriate SID is secured.
• the SID value may be an actual measurement value measured by a distance measuring sensor such as a LIDAR (Laser Imaging Detection and Ranging or Light Detection and Ranging), a TOF (Time Of Flight) camera, or a stereo camera.
• LIDAR and TOF cameras irradiate light such as infrared light and visible light, and measure distance based on the time it takes to receive the reflected light or the phase change between the emitted light and the received light.
• LIDAR measures the distance to the object by arranging multiple laser light emitters in the vertical direction and scanning (rotating) each emitter horizontally.
• TOF cameras measure the distance to the object by irradiating diffused light.
• Stereo cameras measure the distance to the object using the principle of triangulation based on multiple images obtained by photographing the object from different directions.
• the generating unit 63 generates a superimposed image 32 by superimposing the target shooting area 90 identified by the identifying unit 62 on the optical image 30. Furthermore, as indicated by a dashed line in Fig. 7 , the generating unit 63 may further superimpose an irradiation field 92 of radiation R emitted from the radiation irradiation unit 12 on the superimposed image 32.
• the irradiation field 92 of radiation R is found by a geometric calculation using, for example, the value of the SID, the positional relationship (distances dx, dy, and dz) between the radiation source 13 and the first optical camera 26 stored in the storage unit 52, the angle of view ⁇ of the first optical camera 26, and the like (see Fig. 2 ).
• the control unit 65 controls the display of the superimposed image 32 generated by the generation unit 63 on the display 54.
• FIG. 9 shows an example of a screen D1 displayed on the display 54 by the control unit 65.
• the screen D1 includes a superimposed image 32 in which a target imaging area 90 and an irradiation field 92 of radiation R are superimposed on the optical image 30.
• the target imaging area 90 and the irradiation field 92 are misaligned, and if radiation imaging is performed in this state, a proper radiation image cannot be obtained.
• the user checks the screen D1 and moves and aligns at least one of the radiation irradiation unit 12, the radiation detector 20, and the subject H so that the target imaging area 90 and the irradiation field 92 overlap.
• the control unit 65 may also compare the coordinates of the target shooting area 90 and the coordinates of the irradiation field 92 in the superimposed image 32, and if the difference in coordinates is equal to or greater than a predetermined threshold (i.e., if the target shooting area 90 and the irradiation field 92 are significantly misaligned), perform control to issue a warning.
• Screen D1 in FIG. 9 includes a warning message indicating that the target shooting area 90 and the irradiation field 92 are misaligned.
• control unit 65 may, for example, control the display 54 to display a notification, thereby urging appropriate alignment and positioning.
• the determination unit 64 determines whether the subject H is positioned as determined in advance based on the positional relationship of the multiple feature points P extracted by the extraction unit 61. For example, in the example of FIG. 6, if the length of the line segment connecting the shoulder feature points P2L and P2R is less than a predetermined threshold, the subject H may not face forward with respect to the radiation detector 20, but may face in an inclined direction. Also, for example, if the line segment connecting the shoulder feature points P2L and P2R is inclined at an angle equal to or greater than a predetermined threshold, the subject H may not face forward with respect to the radiation detector 20, but may face in an inclined direction. Therefore, the determination unit 64 determines whether the multiple feature points P extracted by the extraction unit 61 maintain a predetermined positional relationship, thereby determining whether the subject H is positioned as determined in advance.
• the suitability of the positioning may also be judged based on at least one optical image for judgment 34 obtained by optically photographing the subject H from a direction different from the photographing direction of the optical image 30 by the second optical camera 28.
• the acquisition unit 60 acquires the optical image for judgment 34 obtained by the second optical camera 28.
• FIG. 8 shows an example of the optical image for judgment 34 photographed by the second optical camera 28 in FIG. 2. In the optical image for judgment 34 in FIG. 8, the subject H is photographed from above the head.
• the extraction unit 61 extracts a plurality of determination feature points of the subject H based on the determination optical image 34 acquired by the acquisition unit 60.
• a plurality of determination feature points J1L to J3L and J1R to J3R extracted from the determination optical image 34 are indicated by black dots.
• the determination feature points J1L to J3L and J1R to J3R correspond to the joint points of the subject H, namely, the ear, shoulder, and elbow, respectively.
• determination feature points J when the plurality of determination feature points J1L to J3L and J1R to J3R are not distinguished from each other, they are simply referred to as “determination feature points J,” and the determination feature points corresponding to each joint point are referred to as “determination feature points of (joint point name).” Note that, as a method for extracting the determination feature points J (joint points), a known posture estimation technique or the like can be appropriately applied.
• the determination unit 64 determines whether the subject H is positioned as determined in advance based on the positional relationship of the multiple determination feature points J extracted by the extraction unit 61. For example, in chest imaging, guidelines and the like stipulate that the subject should take a posture in which the shoulders and elbows are as close as possible to the detection surface 20A of the radiation detector 20. Therefore, if the line segment connecting the shoulder determination feature point J2L (J2R) and the elbow determination feature point J3L (J3R) is inclined at an angle equal to or greater than a predetermined threshold, there is a possibility that the subject H has not brought his/her elbow sufficiently close to the detection surface 20A. Therefore, the determination unit 64 determines whether the multiple determination feature points J extracted by the extraction unit 61 maintain a predetermined positional relationship, thereby determining whether the subject H is positioned as determined in advance.
• the control unit 65 performs control to issue a warning when the judgment unit 64 judges that the subject H is not in a predetermined position based on at least one of the characteristic point P and the judgment characteristic point J.
• the screen D1 in FIG. 9 includes a warning message warning the subject H to bring his/her elbow closer to the radiation detector 20, as there is a possibility that the subject H has not brought his/her elbow close enough to the detection surface 20A.
• the determination unit 64 may also determine whether feature points P other than the predetermined feature points are included in the target imaging area 90 or are located within a predetermined range from the target imaging area 90 (i.e., close to the target imaging area 90). For example, if an unnecessary part overlaps with the part desired to be radiographed, an appropriate radiographic image may not be obtained.
• the determination unit 64 determines whether feature points other than the shoulder feature points P2L and P2R and the waist feature points P5L and P5R are included in or close to the target imaging area 90. As a result, it is determined that the wrist feature points P4L and P4R are located within a predetermined range from the target imaging area 90 (i.e., close to it).
• the control unit 65 performs control to issue a warning when the determination unit determines that a feature point other than a predetermined feature point among the multiple feature points P is included in the target shooting area 90 or is located within a predetermined range from the target shooting area 90.
• Screen D1 in FIG. 9 includes a warning message indicating that the right hand (wrist feature point P4R) and left hand (wrist feature point P4L) may be entering the target shooting area 90.
• the determination unit 64 may determine whether the target shooting area 90 includes an unnecessary part by using a known image recognition technology. For example, when the target shooting area 90 in the optical image 30 should include only the color of the examination gown, but skin color is detected, it may be determined that the target shooting area 90 includes an unnecessary part.
• the CPU 51 executes the information processing program 57, thereby executing the information processing shown in FIG. 10.
• the information processing is executed, for example, when a command to start execution is given by the user via the operation unit 55.
• step S10 the acquisition unit 60 acquires at least one optical image 30 obtained by optically photographing the subject H using the first optical camera 26.
• step S12 the extraction unit 61 extracts feature points P of the subject H based on the optical image 30 acquired in step S10.
• step S14 the identification unit 62 identifies a target imaging region 90 in the optical image 30, based on the feature points P extracted in step S12, that will be the target when radiographing the subject H from approximately the same direction as the imaging direction of the optical imaging.
• the console 50 includes at least one processor, which acquires at least one optical image 30 obtained by optically photographing the subject H, extracts feature points P of the subject H based on the optical image 30, identifies a target imaging area 90 in the optical image 30 that will be a target when radiographing the subject H from approximately the same direction as the imaging direction of the optical imaging based on the feature points P, generates a superimposed image 32 in which the target imaging area 90 is superimposed on the optical image 30, and controls the display of the superimposed image 32 on the display 54.
• the console 50 can identify the target imaging region 90 used to align the radiation emitting unit 12 (radiation source 13), the radiation detector 20, and the subject H. Therefore, the radiation emitting unit 12 (radiation source 13), the radiation detector 20, and the subject H can be appropriately aligned, which can support high-quality radiation imaging.
• the optical image 30 obtained by the first optical camera 26 and the optical image for determination 34 obtained by the second optical camera 28 are described as visible light images, but the present invention is not limited to this.
• at least one of the optical image 30 obtained by the first optical camera 26 and the optical image for determination 34 obtained by the second optical camera 28 may be a distance image representing the distance to the subject H.
• the first optical camera 26 and the second optical camera 28 may be a LIDAR (Laser Imaging Detection and Ranging, or Light A digital still camera, a time-of-flight (TOF) camera, a stereo camera, or the like can be appropriately applied.
• LIDAR Laser Imaging Detection and Ranging
• TOF time-of-flight
• the distance image can also be used to extract the feature points P and the judgment feature points J. Therefore, it is possible to use these to identify the target shooting area 90 and to judge whether the positioning is appropriate.
• first optical camera 26 is also possible to combine the first optical camera 26 as a digital camera that obtains visible light images and the second optical camera 28 as a three-dimensional camera that obtains distance images. It is also possible to use both a digital camera that obtains visible light images and a three-dimensional camera that obtains distance images as the first optical camera 26, and to use both the visible light images and the distance images to identify the target shooting area and to determine whether positioning is appropriate.
• the specification unit 62 specifies the target shooting area 90 by a calculation formula using the relative positional relationship of the feature points P and a predetermined coefficient, and the generation unit 63 generates the superimposed image 32.
• the specification unit 62 may specify the target shooting area 90 using a trained model that is trained in advance to input at least one of the optical image 30 and the feature points P and output the target shooting area 90.
• the generation unit 63 may generate the superimposed image 32 by superimposing the target shooting area 90 specified using the trained model on the optical image 30.
• the optical image to be input may be a visible light image, a distance image, or both.
• the identification unit 62 may identify the target shooting area 90 and generate the superimposed image 32 in an integrated manner.
• the generation unit 63 may generate the superimposed image 32 using a trained model that has been trained in advance to receive at least one of the optical image 30 and the feature point P as input and output the superimposed image 32.
• the input optical image may be a visible light image, a distance image, or both.
• the following various processors can be used as the hardware structure of the processing units that execute various processes, such as the acquisition unit 60, extraction unit 61, identification unit 62, generation unit 63, determination unit 64, and control unit 65.
• the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array); This includes dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed specifically to execute specific processes.
• ASICs Application Specific Integrated Circuits
• a single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.
• Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by client and server computers, and this processor functions as multiple processing units. Secondly, a form in which a processor is used to realize the functions of the entire system, including multiple processing units, with a single IC (Integrated Circuit) chip, as typified by system on chip (SoC). In this way, the various processing units are configured as a hardware structure using one or more of the various processors listed above.
• SoC system on chip
• the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor elements.
• the information processing program 57 in the console 50 is pre-stored in the storage unit 52, but this is not limited to the above.
• the information processing program 57 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory.
• the information processing program 57 may also be downloaded from an external device via a network.
• the technology disclosed herein extends to storage media that non-temporarily store programs, in addition to programs.
• the technology of the present disclosure can also be appropriately combined with the above-mentioned embodiment examples and examples.
• the above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure.
• the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure.
• At least one processor is Obtaining at least one optical image obtained by optically photographing the subject; Extracting feature points of the object based on the optical image; specifying a target imaging region in the optical image based on the feature points, the target imaging region being a target when the subject is radiographed from a direction substantially the same as an imaging direction of the optical imaging; generating a superimposed image by superimposing the target shooting area on the optical image; An information processing device that controls displaying the superimposed image on a display.
• the processor is Extracting a plurality of feature points of the object based on the optical image; The information processing device according to claim 1, further comprising: identifying the target shooting area based on a relative positional relationship of the plurality of feature points.
• the processor is Identifying at least one predetermined reference feature point from among the plurality of feature points; The information processing device according to claim 2, further comprising: identifying the target shooting area based on the reference feature points.
• the processor is Obtaining imaging site information indicating an imaging site of the radiation imaging; The information processing device according to claim 2 or 3, further comprising: identifying the reference feature points according to the body part information.
• the processor is The information processing device according to any one of supplementary items 2 to 4, wherein a warning is issued when a feature point other than a predetermined feature point among the plurality of feature points is included in the target shooting area or is located within a predetermined range from the target shooting area.
• the processor is determining whether the subject is positioned in a predetermined manner based on the positional relationship of the plurality of feature points; 6.
• the information processing device according to any one of supplementary items 2 to 5, further comprising: a warning when it is determined that the subject is not positioned in a predetermined manner.
• the processor is The information processing device according to any one of supplementary items 1 to 6, wherein the superimposed image is generated using a trained model that has been trained in advance to use at least one of the optical image and the feature points as input and the target shooting area or the superimposed image as output.
• the processor is acquiring at least one optical image for determination obtained by optically photographing the subject from a direction different from the photographing direction of the optical image; Extracting a plurality of determination feature points of the subject based on the determination optical image; determining whether the subject is positioned in a predetermined manner based on a positional relationship between the plurality of determination feature points; 8.
• the information processing device further comprising: a warning when it is determined that the subject is not positioned in a predetermined manner.
• a warning when it is determined that the subject is not positioned in a predetermined manner.
• [Additional Item 10] Obtaining at least one optical image obtained by optically photographing the subject; Extracting feature points of the object based on the optical image; specifying a target imaging region in the optical image based on the feature points, the target imaging region being a target when the subject is radiographed from a direction substantially the same as an imaging direction of the optical imaging; generating a superimposed image by superimposing the target shooting area on the optical image; An information processing method including a process of controlling display of the superimposed image on a display.

Landscapes

• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Medical Informatics (AREA)
• Biomedical Technology (AREA)
• Physics & Mathematics (AREA)
• Public Health (AREA)
• General Health & Medical Sciences (AREA)
• Molecular Biology (AREA)
• Heart & Thoracic Surgery (AREA)
• Veterinary Medicine (AREA)
• Animal Behavior & Ethology (AREA)
• Surgery (AREA)
• Radiology & Medical Imaging (AREA)
• Pathology (AREA)
• Biophysics (AREA)
• High Energy & Nuclear Physics (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Optics & Photonics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Epidemiology (AREA)
• Primary Health Care (AREA)
• General Business, Economics & Management (AREA)
• Business, Economics & Management (AREA)
• Human Computer Interaction (AREA)
• Apparatus For Radiation Diagnosis (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=92500889

Family Applications (1)

Country Status (5)

Citations (7)

• 2024
  • 2024-02-20 CN CN202480012086.1A patent/CN120676906A/zh active Pending
  • 2024-02-20 JP JP2025502745A patent/JPWO2024177072A1/ja active Pending
  • 2024-02-20 EP EP24760370.7A patent/EP4670639A1/en active Pending
  • 2024-02-20 WO PCT/JP2024/006071 patent/WO2024177072A1/ja not_active Ceased
• 2025
  • 2025-08-18 US US19/303,250 patent/US20250384582A1/en active Pending

Patent Citations (7)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events