CN114305458A - Method, device, system and storage medium for generating medical combined image - Google Patents

Method, device, system and storage medium for generating medical combined image Download PDF

Info

Publication number
CN114305458A
CN114305458A CN202011071745.9A CN202011071745A CN114305458A CN 114305458 A CN114305458 A CN 114305458A CN 202011071745 A CN202011071745 A CN 202011071745A CN 114305458 A CN114305458 A CN 114305458A
Authority
CN
China
Prior art keywords
image
medical
imaging
dimensional
dimensional image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011071745.9A
Other languages
Chinese (zh)
Inventor
黄紫潇
马伏波
游章科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Shanghai Medical Equipment Ltd
Original Assignee
Siemens Shanghai Medical Equipment Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Shanghai Medical Equipment Ltd filed Critical Siemens Shanghai Medical Equipment Ltd
Priority to CN202011071745.9A priority Critical patent/CN114305458A/en
Priority to PCT/CN2021/112291 priority patent/WO2022073382A1/en
Publication of CN114305458A publication Critical patent/CN114305458A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/0035Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/742Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/7425Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/40ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The embodiment of the invention discloses a method, a device, a system and a storage medium for generating a medical combined image. The method comprises the following steps: acquiring a medical image generated by performing medical imaging on an imaging subject with a medical imaging system; acquiring a three-dimensional image generated by shooting the imaging object by using a camera assembly; generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image; a medically combined image is generated comprising the medical image and the two-dimensional image. The medical combined image includes both a medical image generated by imaging and a two-dimensional image converted from a captured three-dimensional image. The medical combined image enriches the information content, and is beneficial for a reader to master a plurality of information of the imaging object in a multi-dimensional way. The medical combined image further comprises label information, has a distinctive prompting effect and does not interfere with the presentation of the imaging object.

Description

Method, device, system and storage medium for generating medical combined image
Technical Field
The present invention relates to the field of medical imaging technologies, and in particular, to a method, an apparatus, a system, and a storage medium for generating a medical combined image.
Background
Medical imaging refers to the technique and process of obtaining images of internal tissues of a human body or a part of the human body in a non-invasive manner for medical treatment or medical research. It contains the following two relatively independent directions of study: medical imaging systems (medical imaging systems) and medical image processing (medical image processing). The former refers to the process of image formation, including the study of problems such as imaging mechanism, imaging equipment, imaging system analysis, etc.; the latter refers to further processing of the acquired image, including: restoring the original image which is not clear; highlighting certain feature information in the image; pattern classification of the image, and so on.
Common medical imaging techniques include radiography, Computed Tomography (CT), and Magnetic Resonance Imaging (MRI), among others. Many medical imaging techniques both expand the scope of examination and improve the level of diagnosis.
In the prior art, medical imaging systems provide medical images that contain only image information generated based on medical imaging. However, such medical images contain less information. It is desirable to be able to acquire medical images containing more information.
Disclosure of Invention
The embodiment of the invention provides a method, a device, a system and a storage medium for generating a medical combined image.
The technical scheme of the embodiment of the invention comprises the following steps:
a method of generating a medical composite image, comprising:
acquiring a medical image generated by performing medical imaging on an imaging subject with a medical imaging system;
acquiring a three-dimensional image generated by shooting the imaging object by using a camera assembly;
generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image;
a medically combined image is generated comprising the medical image and the two-dimensional image.
Therefore, the medical combined image of the embodiment of the invention not only comprises the medical image generated by medical imaging, but also comprises the two-dimensional image which is obtained by converting the imaging object in the shot three-dimensional image into the preset angle, thereby enriching the information content and being beneficial for a reader to master a plurality of information of the imaging object in a multi-dimension way.
In one embodiment, the method further comprises:
determining label information;
the tag information is attached in a region not containing the imaging subject included in the two-dimensional image of the medically combined image.
Therefore, the medical combined image further comprises label information, so that the information content is further enriched. Further, since the label information is added to the two-dimensional image, which is usually in a color format, a distinctive cue effect is also provided. In addition, tag information is attached to a region not containing an imaging subject, and does not interfere with the presentation of the imaging subject.
In one embodiment, wherein the tag information comprises at least one of:
identity information of the imaging subject; imaging information of the imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; the predetermined angle; parameter information of a region of interest of the imaging subject.
As can be seen, in the embodiment of the present invention, the label information included in the medical combined image has a variety of rich contents. In particular, by including a predetermined angle in the label information, it is convenient for a reader to quickly understand the two-dimensional image.
In one embodiment, the generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image comprises:
displaying the three-dimensional image;
rotating the imaging subject by the predetermined angle in the three-dimensional image;
and projecting the rotated three-dimensional image into the two-dimensional image.
Accordingly, by rotating the imaging subject in the three-dimensional image, a two-dimensional image in which the imaging subject is transformed to a predetermined angle can be conveniently generated.
In one embodiment, the generating a medically combined image comprising the medical image and the two-dimensional image comprises at least one of:
stitching the medical image and the two-dimensional image as the medical combined image without overlapping or partially overlapping;
adjusting the size of the two-dimensional image, and overlaying the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image;
cutting out an interested area from the two-dimensional image; and overlaying the cut-out region of interest into a region which does not contain the imaging object in the medical image.
Therefore, the embodiment of the invention can generate medical combined images in various expression forms, and provides rich options for film readers.
An apparatus for generating a medical composite image, comprising:
a first acquisition module for acquiring a medical image generated by performing medical imaging on an imaging subject with a medical imaging system;
the second acquisition module is used for acquiring a three-dimensional image generated by shooting the imaging object by using the camera shooting assembly;
a first generating module for generating a two-dimensional image of the imaging object transformed to a predetermined angle based on the three-dimensional image;
a second generation module for generating a medically combined image comprising the medical image and the two-dimensional image.
Therefore, the medical combined image of the embodiment of the invention not only comprises the medical image generated by medical imaging, but also comprises the two-dimensional image which is obtained by converting the imaging object in the shot three-dimensional image into the preset angle, thereby enriching the information content and being beneficial for a reader to master a plurality of information of the imaging object in a multi-dimension way.
In one embodiment, further comprising: the tag determining module is used for determining tag information;
wherein the second generating module is further configured to append the label information in a region not containing the imaging subject included in the two-dimensional image of the medically combined image.
Therefore, the medical combined image further comprises label information, so that the information content is further enriched. Further, since the label information is added to the two-dimensional image, which is usually in a color format, a distinctive cue effect is also provided. In addition, tag information is attached to a region not containing an imaging subject, and does not interfere with the presentation of the imaging subject.
In one embodiment, wherein the tag information comprises at least one of:
identity information of the imaging subject; imaging information of the imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; the predetermined angle; parameter information of a region of interest of the imaging subject.
As can be seen, in the embodiment of the present invention, the label information included in the medical combined image has a variety of rich contents. In particular, by including a predetermined angle in the label information, it is convenient for a reader to quickly understand the two-dimensional image.
In one embodiment, the first generating module is configured to present the three-dimensional image; rotating the imaging subject by the predetermined angle in the three-dimensional image; and projecting the rotated three-dimensional image into the two-dimensional image.
Accordingly, by rotating the imaging subject in the three-dimensional image, a two-dimensional image in which the imaging subject is transformed to a predetermined angle can be conveniently generated.
In one embodiment, the second generating module is configured to perform at least one of:
stitching the medical image and the two-dimensional image as the medical combined image without overlapping or partially overlapping;
adjusting the size of the two-dimensional image, and overlaying the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image;
cutting out an interested area from the two-dimensional image; and overlaying the cut-out region of interest into a region which does not contain the imaging object in the medical image.
Therefore, the embodiment of the invention can generate medical combined images in various expression forms, and provides rich options for film readers.
A system for generating a medically combined image, comprising:
a medical imaging system for performing medical imaging on an imaging subject to generate a medical image;
a photographing component for photographing the imaging object to generate a three-dimensional image;
a processor for acquiring the medical image and the three-dimensional image; generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image; a medically combined image is generated comprising the medical image and the two-dimensional image.
Therefore, the embodiment of the invention also provides a generation system of the medical combined image, enriches the information content, and is beneficial for a reader to master a plurality of information of the imaging object in a multi-dimensional way.
In one embodiment, the medical imaging system comprises at least one of: a projection X-ray imaging system; an X-ray computed tomography system; a radionuclide imaging system; an ultrasound imaging system; a magnetic resonance imaging system; the shooting assembly comprises: at least one three-dimensional camera; or, at least two-dimensional cameras and a second processor, wherein each two-dimensional camera is respectively arranged at a predetermined position; the second processor is used for synthesizing at least two-dimensional images shot by the at least two-dimensional cameras into a three-dimensional image of an imaging object, wherein the depth of field adopted in the synthesis is the depth of field of any one two-dimensional image of the at least two-dimensional images; or, at least one two-dimensional camera, at least one depth of field sensor and a second processor, the at least one three-dimensional camera and the at least one depth of field sensor being mounted at the same location; the second processor is used for generating a three-dimensional image of the imaging object by utilizing at least one depth of field provided by the at least one depth of field sensor and at least one two-dimensional picture provided by the at least one two-dimensional camera.
Therefore, the embodiment of the invention is suitable for various medical imaging systems, and the shooting assembly can be deployed in various ways.
An apparatus for generating a medical composite image, comprising a processor and a memory;
the memory has stored therein an application program executable by the processor for causing the processor to execute the method of generating a medical composite image as any one of the above.
Therefore, the embodiment of the invention also provides a medical combined image generation device with a processor-memory architecture, which is beneficial for a reader to grasp a plurality of information of an imaging object in a multi-dimension way.
A computer readable storage medium having stored therein computer readable instructions for performing the method of generating a medical composite image as any one of the above.
Therefore, the embodiment of the invention also provides a computer readable storage medium containing computer readable instructions, which is beneficial for a reader to grasp a plurality of information of the imaging object in a multi-dimension way.
Drawings
Fig. 1 is a flowchart of a method of generating a medical composite image according to an embodiment of the present invention.
Fig. 2 is a first exemplary schematic diagram of generating a medically combined image comprising an X-ray image according to an embodiment of the present invention.
Fig. 3 is a second exemplary schematic diagram of generating a medically combined image according to an embodiment of the present invention.
Fig. 4 is a first exemplary schematic diagram of a medically combined image according to an embodiment of the present invention.
Fig. 5 is a second exemplary schematic diagram of a medically combined image according to an embodiment of the present invention.
Fig. 6 is a block diagram of a medical composite image generation apparatus according to an embodiment of the present invention.
Fig. 7 is a block diagram of a system for generating a medical composite image according to an embodiment of the present invention.
Fig. 8 is a block diagram of a medical composite image generation apparatus having a memory-processor architecture according to an embodiment of the present invention.
Wherein the reference numbers are as follows:
Figure BDA0002715121430000041
Figure BDA0002715121430000051
Detailed Description
In order to make the technical scheme and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
For simplicity and clarity of description, the invention will be described below by describing several representative embodiments. Numerous details of the embodiments are set forth to provide an understanding of the principles of the invention. It will be apparent, however, that the invention may be practiced without these specific details. Some embodiments are not described in detail, but rather are merely provided as frameworks, in order to avoid unnecessarily obscuring aspects of the invention. Hereinafter, "including" means "including but not limited to", "according to … …" means "at least according to … …, but not limited to … … only". In view of the language convention of chinese, the following description, when it does not specifically state the number of a component, means that the component may be one or more, or may be understood as at least one.
In view of the defect that the medical image in the prior art cannot provide more information, the embodiment of the invention provides a technical scheme for generating a novel medical combined image, wherein the medical combined image comprises a medical image generated by performing medical imaging on an imaging object and a two-dimensional image converted from a three-dimensional image of an imaging object shot by a shooting component, so that the information content in the image can be enriched, and a reader can conveniently grasp a plurality of pieces of information of the imaging object in a multi-dimension manner.
Fig. 1 is a flowchart of a method of generating a medical composite image according to an embodiment of the present invention. Preferably, the method of fig. 1 may be performed by a controller. Wherein the controller may be implemented as or integrated into a control host of the medical imaging system, and may also be implemented as a control unit separate from the control host.
As shown in fig. 1, the method includes:
step 101: a medical image generated by performing medical imaging of an imaging subject with a medical imaging system is acquired.
A medical imaging system performs medical imaging on an imaging subject and may generate a medical image. For example, a medical imaging system may include: a projection X-ray imaging system; an X-ray computed tomography system; a radionuclide imaging system; an ultrasound imaging system; magnetic resonance imaging systems, and the like. Accordingly, the medical image comprises: an X-ray image; a radionuclide image; an ultrasound image; magnetic resonance images, etc.
Step 102: a three-dimensional image generated by photographing an imaging object with a camera assembly is acquired.
In one embodiment, the capture assembly includes at least one three-dimensional camera. The three-dimensional camera photographs an imaging subject using a three-dimensional imaging technique to generate a three-dimensional image of the imaging subject. The three-dimensional camera may be arranged in an examination room in which the imaging subject is located, at any location suitable for photographing the imaging subject, such as on a ceiling, on a floor, or on various components in a medical imaging system, etc.
In one embodiment, the photographing assembly includes at least two-dimensional cameras, wherein each two-dimensional camera is respectively disposed at a predetermined position. In practice, a person skilled in the art can select an appropriate position as a predetermined position to arrange the two-dimensional cameras as necessary. The capture assembly may further include a processor. The processor synthesizes the two-dimensional images captured by the respective two-dimensional cameras into a three-dimensional image of the imaging object, wherein the depth of field adopted by the processor in the synthesis may be the depth of field of any two-dimensional image. Alternatively, each two-dimensional camera may send the respective captured two-dimensional image to a processor outside the capturing component, so that the processor outside the capturing component synthesizes the two-dimensional images captured by the respective two-dimensional cameras into a three-dimensional image of the imaging object, wherein the depth of field adopted by the processor outside the capturing component in the synthesizing process may also be the depth of field of any two-dimensional image. In particular, the processor outside the shooting assembly can be implemented as a control host in the medical imaging system, and can also be implemented as an independent control unit separated from the medical imaging system. Each two-dimensional camera may be arranged in an examination room in which the imaging subject is located, at any location suitable for photographing the imaging subject, such as on a ceiling, on a floor, or on various components in a medical imaging system, etc.
In one embodiment, the photographing assembly may include: at least one two-dimensional camera and at least one depth of field sensor. At least one two-dimensional camera and at least one depth of field sensor are mounted at the same location. The capture assembly may further include a processor. The processor utilizes the depth of field provided by the depth of field sensor in conjunction with the two-dimensional picture provided by the two-dimensional camera to generate a three-dimensional image of the imaged object. Optionally, the two-dimensional camera sends the shot two-dimensional image of the imaging object to a processor outside the shooting assembly, and the depth sensor sends the collected depth of field to the processor outside the shooting assembly, so that the processor outside the shooting assembly generates a three-dimensional image of the imaging object by using the depth of field together with the two-dimensional picture. Preferably, the processor outside the shooting assembly can be implemented as a control host in the medical imaging system, and can also be implemented as an independent control unit separated from the medical imaging system. The two-dimensional camera may be arranged in an examination room in which the imaging subject is located, at any location suitable for photographing the imaging subject, such as on a ceiling, on a floor, or on various components in a medical imaging system, etc.
The above exemplary description describes a typical example in which the image pickup assembly picks up an imaging object to generate a three-dimensional image, and those skilled in the art will appreciate that this description is merely exemplary and is not intended to limit the scope of the embodiments of the present invention.
Preferably, the three-dimensional image may be acquired from the camera assembly in a wired communication manner or a wireless communication manner.
Here, there is no particular requirement on the execution order of step 101 and step 102. For example, step 102 may be performed before step 101, step 102 may be performed after step 101, or step 101 and step 102 may be performed simultaneously.
In step 103: a two-dimensional image of the imaging subject transformed to a predetermined angle is generated based on the three-dimensional image.
Here, with the three-dimensional image acquired in step 102, a two-dimensional image in which the imaging subject is transformed to a predetermined angle is acquired. Wherein the angle is preferably settable. For example, the user may edit the angle via a human-machine interface.
In one embodiment, generating a two-dimensional image of the imaging subject transformed to the predetermined angle based on the three-dimensional image comprises: displaying a three-dimensional image; rotating (for example, rotating along an arbitrary axis in a three-dimensional coordinate system of the three-dimensional image) the imaging object by a predetermined angle in the three-dimensional image; the rotated three-dimensional image is projected onto a predetermined projection plane, thereby forming a two-dimensional image. Preferably, the projected two-dimensional image is a color image. Preferably, the projection surface is configurable. For example, the user may set the projection surface via the interactive interface.
The above exemplary descriptions have been given of typical examples of generating a two-dimensional map of an imaging object transformed to a predetermined angle based on a three-dimensional image, and those skilled in the art will appreciate that such descriptions are merely exemplary and are not intended to limit the scope of embodiments of the present invention.
Step 104: a medically combined image is generated that includes the medical image and the two-dimensional image.
Here, generating the medical combined image including the medical image and the two-dimensional image includes at least one of:
(1) and splicing the medical image and the two-dimensional image into a medical combined image without overlapping.
In such an embodiment, the complete medical image and the complete two-dimensional image together constitute a medical combined image.
(2) And splicing the medical image and the two-dimensional image into a medical combined image in a partially overlapped mode.
In such an embodiment, the complete or partial medical image together with the complete or partial two-dimensional image form a medical combined image.
(3) And adjusting the size of the two-dimensional image, and superposing the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image.
In such an embodiment, the two-dimensional image is resized such that the resized two-dimensional image fits into a region of the medical image that does not contain the imaging subject, and therefore the two-dimensional image does not occlude the imaging subject in the medical image.
(4) Cutting out an interested area from the two-dimensional image; and superposing the cut-out region of interest into a region which does not contain the imaging object in the medical image.
In such an embodiment, the cropped region of interest may be highlighted without the region of interest obscuring the imaging subject in the medical image.
In one embodiment, the method further comprises: determining label information; tag information is attached in a region not containing an imaging subject included in a two-dimensional image of the medically combined image.
Therefore, the medical combined image further comprises label information, so that the information content is further enriched. Further, since the label information is added to the two-dimensional image, which is usually in a color format, a distinctive cue effect is also provided. In addition, tag information is attached to a region not containing an imaging subject, and does not interfere with the presentation of the imaging subject. Preferably, the tag information includes: identity information of the imaging subject; imaging information of an imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; a predetermined angle; parameter information of a region of interest of the imaging subject, etc. As can be seen, in the embodiment of the present invention, the label information included in the medical combined image has a variety of rich contents. In particular, by including a predetermined angle in the label information, it is convenient for a reader to quickly understand the two-dimensional image.
The medically combined images of embodiments of the present invention may be packaged in Digital Imaging and Communications in Medicine (DICOM) files that facilitate the exchange of medical image information. For example, a DICOM file may be sent to a picture management and communication system (PACS) for various process analyses.
The following will illustrate embodiments of the present invention by taking X-ray imaging as an example.
X-ray imaging systems typically include: an X-ray generation assembly, a flat panel detector, a chest frame (BWS) assembly and/or a Table assembly and a workstation. The subject stands near the chest frame assembly or lies on the examination table assembly and may be subjected to X-ray radiography of various parts of the skull, chest, abdomen, joints, etc.
Fig. 2 is a first exemplary schematic diagram of generating a medically combined image comprising an X-ray image according to an embodiment of the present invention.
In fig. 2, the X-ray generation assembly, including the X-ray tube 71 and the beam splitter 72, is connected via a support with a telescopic socket 78, the telescopic socket 78 being connected to the ceiling 70. Also, two-dimensional cameras, a two-dimensional camera 73 and a two-dimensional camera 74, are fixed at different positions on the ceiling 70, wherein the shooting directions of the two- dimensional cameras 73 and 74 are both directed toward the imaging subject 75 on the couch assembly 76.
The X-ray generating assembly emits X-rays that are transmitted through the imaging subject 75 and forms X-ray imaging data of the subject on the flat panel detector 90. The flat panel detector 90 sends X-ray imaging data of the imaging subject 75 to the control host via the access point. The control host can be a control host arranged in a local control room, and can also be a remote control host, such as a control host in a cloud. The control host can be a control host arranged in a local control room, and can also be a remote control host, such as a control host in a cloud. The X-ray imaging data is a raw image of the imaging subject 75. In general, such an original image cannot obtain an ideal visual effect, and a variety of image processing methods are required to be performed on the original image to improve the visual effect. For example, the control host performs image processing (e.g., adjusting image contrast, global equalization, and local detail) on the X-ray imaging data (raw image) provided by the flat panel detector 90 to generate an X-ray image suitable for presentation to a user.
The two-dimensional camera 73 and the two-dimensional camera 74 respectively photograph the imaging subject 75 to acquire respective photographed images. The two- dimensional cameras 73 and 74 transmit the images captured by the cameras to the control host via wired or wireless communication with the control host. The control host reconstructs the images taken by the two-dimensional camera 73 and the two-dimensional camera 74 into a three-dimensional image of the imaging object 75 based on the polarization three-dimensional imaging mode. And displaying the three-dimensional image on a display screen connected with the control host, and determining the rotation angle set by the user. And rotating the imaging object in the three-dimensional image on the display screen by the rotation angle, and then projecting to generate a two-dimensional image.
The control host generates a medically combined image comprising the medical image and the two-dimensional image. The control host may generate the medically combined image based on a variety of ways. For example, in one embodiment, the control host stitches the medical image and the two-dimensional image into a medical combined image without or with partial registration. For example, in one embodiment, the control host adjusts the size of the two-dimensional image, and then superimposes the two-dimensional image after the size adjustment onto a region of the medical image that does not include the imaging object. For example, in one embodiment, the control host cuts out a region of interest from the two-dimensional image, and superimposes the cut region of interest onto a region of the medical image that does not include the imaging subject.
In fig. 2, the two-dimensional camera 73 and the two-dimensional camera 74 are respectively arranged at different positions on the ceiling 70. In fact, either one of the two-dimensional camera 73 and the two-dimensional camera 74 may be disposed at any position suitable for photographing the imaging subject 75 in the examination room. Such as. Either the two-dimensional camera 73 or the two-dimensional camera 74 may be disposed at a mounting location 81 on the housing of the beam splitter 72 or at a mounting location 80 on the housing of the telescoping tube sleeve 78.
Fig. 3 is a second exemplary schematic diagram of generating a medically combined image according to an embodiment of the present invention.
In fig. 3, an X-ray generating assembly including an X-ray tube 71 and a beam splitter 72 is connected to a column 83 via a support. The imaging subject 75 stands adjacent to a chest frame assembly 78. Further, a two-dimensional camera 73 is fixed to the ceiling 70, and a two-dimensional camera 74 is fixed to the upright 83, wherein the shooting directions of both the two-dimensional camera 73 and the two-dimensional camera 74 are directed toward the imaging subject 75 near the chest stand assembly 78.
The X-ray generating assembly emits X-rays that are transmitted through the imaging subject 75 and forms medical image data of the subject on the flat panel detector 90. The flat panel detector 90 sends X-ray imaging data of the imaging subject 75 to the control host via the access point. The control host can be a control host arranged in a local control room, and can also be a remote control host, such as a control host in a cloud. The control host can be a control host arranged in a local control room, and can also be a remote control host, such as a control host in a cloud. The X-ray imaging data is a raw image of the imaging subject 75. In general, such an original image cannot obtain an ideal visual effect, and a variety of image processing methods are required to be performed on the original image to improve the visual effect. For example, the control host performs image processing (e.g., adjusting image contrast, global equalization, and local detail) on the X-ray imaging data (raw image) provided by the flat panel detector 90 to generate an X-ray image suitable for presentation to a user.
The two-dimensional camera 73 and the two-dimensional camera 74 each photograph an imaging subject 75, and each photographed image is transmitted to the control host. The control host reconstructs the images taken by the two-dimensional camera 73 and the two-dimensional camera 74 into a three-dimensional image of the imaging object 75 based on the polarization three-dimensional imaging mode. And displaying the three-dimensional image on a display screen connected with the control host, and determining the rotation angle set by the user. And rotating the imaging object in the three-dimensional image on the display screen by the rotation angle, and then projecting to generate a two-dimensional image.
The control host generates a medically combined image comprising the medical image and the two-dimensional image.
In fig. 3, the two-dimensional camera 73 and the two-dimensional camera 74 are arranged at the ceiling 70 and on the pillar 83, respectively. In fact, either one of the two-dimensional camera 73 and the two-dimensional camera 74 may be disposed at any position suitable for photographing the imaging subject 75 in the examination room. Such as. Either one of the two-dimensional camera 73 and the two-dimensional camera 74 may also be disposed at a mounting location 82 on the housing of the beam splitter 72 or at a mounting location 84 on the floor 77.
In fig. 2 and 3, an example of synthesizing a three-dimensional image in a control host of a medical imaging system is described. In practice, the processor may also be disposed on the two-dimensional camera 73 or the two-dimensional camera 74. The processor synthesizes a three-dimensional image based on the two-dimensional images photographed by the two-dimensional camera 73 and the two-dimensional camera 74, and transmits the synthesized three-dimensional image to a control host of the medical imaging system. In such an embodiment, there is no need to deploy three-dimensional image synthesis functionality on the control host of the medical imaging system.
A typical example of a medically combined image is described below.
Fig. 4 is a first exemplary schematic diagram of a medically combined image according to an embodiment of the present invention.
In fig. 4, the medically combined image 40 contains an X-ray image 41 of the left hand of which the imaging subject is a left hand and a two-dimensional image 42 of the left hand, wherein the two-dimensional image 42 has a smaller rotation angle. The X-ray image 41 and the two-dimensional image 42 are stitched as a medically combined image 40 without being coincident. Also, in a region (upper right corner as shown in the figure) in which the two-dimensional image 42 does not contain the imaging subject, there is also a tag information presentation area 43. The tag information may be presented in the tag information presentation area 43, such as: the name of the patient; the sex of the patient; the age of the patient; date, time of photograph taking; shooting a part; examining the organ program name; the name of the hospital; the name of the technician; a photograph taking angle or a photograph rotation angle; a photo pixel size; measured dimensions of an imaging subject (e.g., a hand); thickness of the imaging subject (e.g., hand); and so on. The tag information presentation area 43 may also be located in other areas of the two-dimensional image 42 that do not contain imaging objects, such as the upper left corner, the lower left corner, or the lower right corner of the two-dimensional image 42, and so forth. In a variant embodiment, two or more tag information presentation areas may be provided in the two-dimensional image 42. In other words, those skilled in the art can select and set the number, position, size, shape, etc. of the tag information display areas according to actual needs, and the present invention is not limited in this respect.
Fig. 5 is a second exemplary schematic diagram of a medically combined image according to an embodiment of the present invention.
In fig. 5, the medically combined image 50 contains an X-ray image 51 of which the imaging subject is the left hand and a two-dimensional image 52 of the left hand. Wherein the two-dimensional image 52 has a smaller rotation angle. The two-dimensional image 52 is superimposed into a region of the medical image 50 that does not contain the imaging subject. Further, in a region of the two-dimensional image 52 not including the imaging object, there is also a tag information presentation region 53. The tag information may be presented in the tag information presentation area 53, such as: the name of the patient; the sex of the patient; the age of the patient; date, time of photograph taking; shooting a part; examining the organ program name; the name of the hospital; the name of the technician; a photograph taking angle or a photograph rotation angle; a photo pixel size; measured dimensions of an imaging subject (e.g., a hand); thickness of the imaging subject (e.g., hand); and so on.
The generation manner of the medical combined image and the specific content of the medical combined image are exemplarily described above by taking the X-ray imaging system as an example. Those skilled in the art will appreciate that this description is by way of example only, and is not intended to limit the scope of embodiments of the invention.
Based on the above description, the embodiment of the invention also provides a device for generating the medical combined image. Fig. 6 is a block diagram of a medical composite image generation apparatus according to an embodiment of the present invention.
As shown in fig. 6, the generation apparatus 600 of the medical combined image includes:
a first acquisition module 601 for acquiring a medical image generated by performing medical imaging on an imaging object with a medical imaging system;
a second obtaining module 602, configured to obtain a three-dimensional image generated by capturing an image of the imaging object with a camera assembly;
a first generating module 603 for generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image;
a second generating module 604 for generating a medically combined image comprising the medical image and the two-dimensional image.
In one embodiment, further comprising: a tag determination module 605 for determining tag information; wherein the second generating module 604 is further configured to append the label information in a region not containing the imaging subject included in the two-dimensional image of the medically combined image.
In one embodiment, wherein the tag information comprises at least one of: identity information of the imaging subject; imaging information of the imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; the predetermined angle; parameter information of a region of interest of the imaging subject.
In one embodiment, the first generating module 601 is configured to display the three-dimensional image; rotating the imaging subject by the predetermined angle in the three-dimensional image; and projecting the rotated three-dimensional image into the two-dimensional image.
In one embodiment, the second generating module 604 is configured to perform at least one of the following: stitching the medical image and the two-dimensional image as the medical combined image without overlapping or partially overlapping; adjusting the size of the two-dimensional image, and overlaying the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image; cutting out an interested area from the two-dimensional image; and superposing the cut-out region of interest to a region which does not contain the imaging object in the medical image, and the like.
Based on the above description, the embodiment of the invention also provides a system for generating the medical combined image. Fig. 7 is a block diagram of a system for generating a medical composite image according to an embodiment of the present invention.
As shown in fig. 7, the generation system 700 of the medical combined image includes:
a medical imaging system 701 for performing medical imaging on an imaging subject to generate a medical image;
a capture component 702 for capturing the imaging subject to generate a three-dimensional image;
a first processor 703 for acquiring the medical image and the three-dimensional image; generating a two-dimensional image of a region of interest in the imaging subject at a predetermined angle based on the three-dimensional image; a medically combined image is generated comprising the medical image and the two-dimensional image.
In one embodiment, the medical imaging system 701 includes at least one of: a projection X-ray imaging system; an X-ray computed tomography system; a radionuclide imaging system; an ultrasound imaging system; a magnetic resonance imaging system.
In one embodiment, the capture component 702 includes: at least one three-dimensional camera.
In one embodiment, the capture component 702 includes: at least two-dimensional cameras and a second processor, wherein each two-dimensional camera is respectively arranged at a predetermined position; the second processor synthesizes at least two-dimensional images captured by at least two-dimensional cameras into a three-dimensional image of an imaging object, wherein a depth of field employed in the synthesis is a depth of field of any one of the at least two-dimensional images. Preferably, the second processor is integrated with either of the two-dimensional cameras.
In one embodiment, the capture component 702 includes: at least one two-dimensional camera, at least one depth of field sensor, and a second processor, wherein the at least one three-dimensional camera and the at least one depth of field sensor are mounted at the same location; the second processor generates a three-dimensional image of the imaged object using at least one depth of field provided by the at least one depth of field sensor in conjunction with at least one two-dimensional photograph provided by the at least one two-dimensional camera. Preferably, the second processor is integrated with the at least one two-dimensional camera, or with the depth of field sensor.
Fig. 8 is a block diagram of a medical composite image generation apparatus having a memory-processor architecture according to an embodiment of the present invention.
As shown in fig. 8, the apparatus 800 for generating a medically combined image comprises a processor 801, a memory 802 and a computer program stored on the memory 802 and operable on the processor 801, which computer program, when executed by the processor 801, implements the method for generating a medically combined image as described in any one of the above. The memory 802 may be embodied as various storage media such as an Electrically Erasable Programmable Read Only Memory (EEPROM), a Flash memory (Flash memory), and a Programmable Read Only Memory (PROM). The processor 801 may be implemented to include one or more central processors or one or more field programmable gate arrays that integrate one or more central processor cores. In particular, the central processor or central processor core may be implemented as a CPU or MCU or DSP, etc.
It should be noted that not all steps and modules in the above flows and structures are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The division of each module is only for convenience of describing adopted functional division, and in actual implementation, one module may be divided into multiple modules, and the functions of multiple modules may also be implemented by the same module, and these modules may be located in the same device or in different devices.
The hardware modules in the various embodiments may be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (e.g., a special purpose processor such as an FPGA or ASIC) for performing specific operations. A hardware module may also include programmable logic devices or circuits (e.g., including a general-purpose processor or other programmable processor) that are temporarily configured by software to perform certain operations. The implementation of the hardware module in a mechanical manner, or in a dedicated permanent circuit, or in a temporarily configured circuit (e.g., configured by software), may be determined based on cost and time considerations.
The present invention also provides a machine-readable storage medium storing instructions for causing a machine to perform a method as described herein. Specifically, a system or an apparatus equipped with a storage medium on which a software program code that realizes the functions of any of the embodiments described above is stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program code stored in the storage medium. Further, part or all of the actual operations may be performed by an operating system or the like operating on the computer by instructions based on the program code. The functions of any of the above-described embodiments may also be implemented by writing the program code read out from the storage medium to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causing a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on the instructions of the program code. Examples of the storage medium for supplying the program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD + RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or the cloud by a communication network.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (14)

1. A method (100) of generating a medical composite image, comprising:
acquiring a medical image (101) generated by performing medical imaging on an imaging subject with a medical imaging system;
acquiring a three-dimensional image (102) generated by shooting the imaging object by using a camera assembly;
generating a two-dimensional image (103) of the imaging subject transformed to a predetermined angle based on the three-dimensional image;
a medically combined image (104) comprising the medical image and the two-dimensional image is generated.
2. A method (100) of generating a medical composite image according to claim 1, the method further comprising:
determining label information;
the tag information is attached in a region not containing the imaging subject included in the two-dimensional image of the medically combined image.
3. A method (100) of generating a medical composite image according to claim 2, wherein the label information comprises at least one of:
identity information of the imaging subject; imaging information of the imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; the predetermined angle; parameter information of a region of interest of the imaging subject.
4. A method (100) for generating a medical combined image according to claim 1, wherein the generating of the two-dimensional image (103) of the imaging subject transformed to the predetermined angle based on the three-dimensional image comprises:
displaying the three-dimensional image;
rotating the imaging subject by the predetermined angle in the three-dimensional image;
and projecting the rotated three-dimensional image into the two-dimensional image.
5. A method (100) for generating a medical combined image according to claim 1, wherein the generating of the medical combined image (104) comprising the medical image and the two-dimensional image comprises at least one of:
stitching the medical image and the two-dimensional image as the medical combined image without overlapping or partially overlapping;
adjusting the size of the two-dimensional image, and overlaying the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image;
cutting out an interested area from the two-dimensional image; and overlaying the cut-out region of interest into a region which does not contain the imaging object in the medical image.
6. A generation apparatus (600) of a medical composite image, comprising:
a first acquisition module (601) for acquiring a medical image generated by performing medical imaging of an imaging subject with a medical imaging system;
a second acquisition module (602) for acquiring a three-dimensional image generated by shooting the imaging object with the camera assembly;
a first generating module (603) for generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image;
a second generation module (604) for generating a medically combined image comprising the medical image and the two-dimensional image.
7. A generation apparatus (600) of a medical combined image according to claim 6, further comprising:
a tag determination module (605) for determining tag information;
wherein the second generating module (604) is further configured to append the label information in a region not containing the imaging subject contained in the two-dimensional image of the medically combined image.
8. A generating device (600) of a medically combined image according to claim 7, wherein the label information comprises at least one of:
identity information of the imaging subject; imaging information of the imaging subject; shooting information of the medical image; shooting information of the three-dimensional image; the predetermined angle; parameter information of a region of interest of the imaging subject.
9. A generation apparatus (600) of a medically combined image according to claim 6,
the first generation module (601) is used for displaying the three-dimensional image; rotating the imaging subject by the predetermined angle in the three-dimensional image; and projecting the rotated three-dimensional image into the two-dimensional image.
10. A generation apparatus (600) of a medically combined image according to claim 6,
the second generating module (604) is configured to perform at least one of:
stitching the medical image and the two-dimensional image as the medical combined image without overlapping or partially overlapping;
adjusting the size of the two-dimensional image, and overlaying the two-dimensional image after size adjustment to a region which does not contain the imaging object in the medical image;
cutting out an interested area from the two-dimensional image; and overlaying the cut-out region of interest into a region which does not contain the imaging object in the medical image.
11. A system (700) for generating a medical composite image, comprising:
a medical imaging system (701) for performing medical imaging on an imaging subject to generate a medical image;
a capture component (702) for capturing the imaging subject to generate a three-dimensional image;
a first processor (703) for acquiring the medical image and the three-dimensional image; generating a two-dimensional image of the imaging subject transformed to a predetermined angle based on the three-dimensional image; a medically combined image is generated comprising the medical image and the two-dimensional image.
12. A generation system (700) of a medically combined image according to claim 11,
the medical imaging system comprises at least one of: a projection X-ray imaging system; an X-ray computed tomography system; a radionuclide imaging system; an ultrasound imaging system; a magnetic resonance imaging system;
the capture component (702) comprises:
at least one three-dimensional camera; or
At least two-dimensional cameras and a second processor, wherein each two-dimensional camera is respectively arranged at a predetermined position; the second processor is used for synthesizing at least two-dimensional images shot by the at least two-dimensional cameras into a three-dimensional image of an imaging object, wherein the depth of field adopted in the synthesis is the depth of field of any one two-dimensional image of the at least two-dimensional images; or
The system comprises at least one two-dimensional camera, at least one depth of field sensor and a second processor, wherein the at least one three-dimensional camera and the at least one depth of field sensor are arranged at the same position; the second processor is used for generating a three-dimensional image of the imaging object by utilizing at least one depth of field provided by the at least one depth of field sensor and at least one two-dimensional picture provided by the at least one two-dimensional camera.
13. A generation apparatus (800) of a medically combined image, comprising a processor (801) and a memory (802);
the memory (802) has stored therein an application program executable by the processor (801) for causing the processor (801) to perform the method (100) of generating a medically combined image as defined in any one of claims 1 to 5.
14. A computer-readable storage medium, characterized in that computer-readable instructions are stored therein for performing the method (100) of generating a medically combined image as defined in any one of claims 1 to 5.
CN202011071745.9A 2020-10-09 2020-10-09 Method, device, system and storage medium for generating medical combined image Pending CN114305458A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202011071745.9A CN114305458A (en) 2020-10-09 2020-10-09 Method, device, system and storage medium for generating medical combined image
PCT/CN2021/112291 WO2022073382A1 (en) 2020-10-09 2021-08-12 Method, apparatus, and system for generating medical combination image, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011071745.9A CN114305458A (en) 2020-10-09 2020-10-09 Method, device, system and storage medium for generating medical combined image

Publications (1)

Publication Number Publication Date
CN114305458A true CN114305458A (en) 2022-04-12

Family

ID=77520463

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011071745.9A Pending CN114305458A (en) 2020-10-09 2020-10-09 Method, device, system and storage medium for generating medical combined image

Country Status (2)

Country Link
CN (1) CN114305458A (en)
WO (1) WO2022073382A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1710615A (en) * 2004-06-17 2005-12-21 奥林巴斯株式会社 Image processing method and image processing apparatus
CN104799874A (en) * 2014-01-28 2015-07-29 上海西门子医疗器械有限公司 Medical image projection method, device and system and medical equipment
CN105796052A (en) * 2015-01-16 2016-07-27 西门子公司 Joint visualization of 3D reconstructed photograph and internal medical scan
US20200219255A1 (en) * 2019-01-03 2020-07-09 Camerad Technologies Systems and methods for radiographic and photographic imaging of patients
CN111479509A (en) * 2017-10-16 2020-07-31 皇家飞利浦有限公司 Determination of object distribution using a camera

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1710615A (en) * 2004-06-17 2005-12-21 奥林巴斯株式会社 Image processing method and image processing apparatus
CN104799874A (en) * 2014-01-28 2015-07-29 上海西门子医疗器械有限公司 Medical image projection method, device and system and medical equipment
CN105796052A (en) * 2015-01-16 2016-07-27 西门子公司 Joint visualization of 3D reconstructed photograph and internal medical scan
CN111479509A (en) * 2017-10-16 2020-07-31 皇家飞利浦有限公司 Determination of object distribution using a camera
US20200219255A1 (en) * 2019-01-03 2020-07-09 Camerad Technologies Systems and methods for radiographic and photographic imaging of patients

Also Published As

Publication number Publication date
WO2022073382A1 (en) 2022-04-14

Similar Documents

Publication Publication Date Title
CN106687046B (en) Guidance system for positioning a patient for medical imaging
CN109381208B (en) Radiographic system and method for operating the same
US9990744B2 (en) Image registration device, image registration method, and image registration program
KR101666943B1 (en) The method and apparatus otaining a image related to region of interest of a target object
JP6205078B2 (en) Vertebral level imaging system
CN107106099B (en) Imaging system for imaging an elongated region of interest of an object
CN106572829A (en) Control of the positioning of a scanning region of a medical imaging system
US11006915B2 (en) Mammography examinations
JP2002052018A (en) Image display device, image display method and storage medium
US10271801B2 (en) Radiation imaging system, image processing device, and image processing program
US10433795B2 (en) Radiation imaging system, image processing device, radiation imaging method, and image processing program
JP6345468B2 (en) Medical diagnostic imaging equipment
US10078906B2 (en) Device and method for image registration, and non-transitory recording medium
CN108475443A (en) The synthesis of blood vessel structure indicates
CN111803110B (en) X-ray fluoroscopic apparatus
US10102638B2 (en) Device and method for image registration, and a nontransitory recording medium
US10631948B2 (en) Image alignment device, method, and program
CN114305458A (en) Method, device, system and storage medium for generating medical combined image
JP6824641B2 (en) X-ray CT device
JP2007135828A (en) Diagnostic imaging support system and image display method
US10049480B2 (en) Image alignment device, method, and program
Lapeer et al. PC-based volume rendering for medical visualisation and augmented reality based surgical navigation
JP2005012248A (en) Method and apparatus of assisting image reading
JP6956514B2 (en) X-ray CT device and medical information management device
JP2016538065A (en) Positioning partial volumes of anatomy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination