JP6061991B2 - Information processing apparatus, information processing method, information processing system, program, and storage medium - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for processing an image of a subject taken by CT or MRI in the medical field, for example, and in particular, as a key (KEY) image attached to a CT or MRI diagnosis report. The present invention relates to a reporting system and a reporting method using a dimensional image.

  With the use of spiral CT, multi-slice CT, or the development of high-speed MRI, more than 300 images can be generated instantaneously in one shot.

  In addition, when diagnosing these images, a doctor (diagnostic doctor) such as a radiologist uses a high-speed computer environment to make a diagnosis by switching and displaying a plurality of cross-sectional images at high speed. .

  Then, a key image or a schema (a sketch indicating the relationship between an organ and a lesion) referred to by a requesting doctor (such as a physician) is attached to the diagnosis report from the diagnosis doctor.

  Although it is possible to attach all 300 images or more as key images to the diagnostic report, in this case, a large load may be applied to the network or computer, and high-speed display may not be performed.

  Therefore, it is conceivable to select important images, but it is painful for the client doctor to select key images from 300 or more images, and as a simple method, simple thinning is performed at regular intervals of about one third. It has also been done. However, with such simple thinning, there is a possibility that images important for diagnosis may be thinned out easily, which may hinder the understanding of the client doctor.

  The present invention has been made in view of the above problems, and an object thereof is to determine a parameter when rendering an image based on diagnostic information, and to create an image or a group of images including a lesion site from the parameter. And providing an image processing method.

  Another object is to provide a reporting system and a reporting method capable of outputting an image selected by the image processing or an image rendered by the image processing as a diagnostic report by attaching it to diagnostic information.

To solve the problems described above and achieve the object, the information processing apparatus of the present invention includes an image input unit for inputting a plurality of medical images captured, input by a user of the diagnostic information for the previous SL Medical Imaging An information analysis unit that analyzes the diagnostic information, and a key image that is attached to the diagnostic information and output in the diagnostic report is generated from the plurality of medical images based on the result of the analysis a key image generation unit for, Ru comprising a.

  As described above, according to the present invention, it is possible to determine a parameter for rendering an image based on diagnostic information and create an image or a group of images including a lesion site from the parameter.

  In addition, it is possible to construct a reporting system and a reporting method that can output an image selected by the image processing or an image rendered by the image processing attached to diagnostic information and output as a diagnostic report.

1 is a diagram schematically showing functional blocks that constitute an image processing apparatus according to a first embodiment of the present invention. FIG. 6 is a flowchart illustrating image processing according to the first embodiment. The figure which shows schematically the functional block which comprises the image processing apparatus of 2nd Embodiment which concerns on this invention. The flowchart which shows the image processing of 2nd Embodiment.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 schematically shows functional blocks constituting the image processing apparatus according to the first embodiment of the present invention. An image search unit 4 includes all of the captured image data input from the image input unit 1 based on diagnostic information. A lesion area to be output is retrieved from the image output unit 6 and a captured image or a group of captured images corresponding to the lesion area is output from the image output unit 6 as a key image.

  Here, the image input unit 1 may be considered as a medical image photographing device such as CT or MRI, or an image input interface that is input from these devices via a network and a storage device associated therewith. There is also.

  The diagnosis information input unit 2 receives a diagnosis result using the image data taken by a doctor (radiologist or the like) as diagnosis information (hereinafter also referred to as a diagnosis report). This diagnostic report is input in the form of text information or voice information, for example. The diagnosis result is output to a display device (not shown). In addition to the diagnosis by the doctor, diagnosis results using a computer, direct diagnosis results using a photographing film, and the like can be applied.

  The diagnostic information input from the diagnostic information input unit 2 is searched for keywords by the diagnostic information analysis unit 3. In this keyword search, anatomical terms and pathological terms are extracted in association with each other.

  The anatomical terms include right frontal lobe, left lung, liver, and the like, and pathological terms include tumors, emboli, nodules, and stages I, II, III, etc. relating to the degree of disease progression.

  Moreover, a keyword can be input in the diagnostic information input unit 2, and in this case, the keyword input in the diagnostic information analysis unit 3 is not necessary.

  The parameter determination unit 5 calculates parameters used in the image search unit 4 based on the input keyword.

  Examples of the parameters include a CT value serving as an anatomical threshold, a range of MRI values, a CT value designating a lesion, a range of MRI values, and a slice position. These initial values are selected from a table empirically determined in consideration of the degree of disease progression, but vary from individual to individual.

  In order to correct (optimize) this variation, the selected parameter is applied to the input image to determine the appropriateness of the parameter.

  This determination can be made based on how the area or connectivity of the searched lesion area changes within the cross section or between cross sections when the parameter is changed. For example, air is inhaled during lung extraction. It may be bad and the CT value of the lung may be measured higher.

  For such an image, when the normal CT value range is applied, the area of the lung field becomes extremely small compared to the cross-sectional area of the chest, so that it becomes an appropriate ratio (about 50-60%). The CT value is corrected.

  In general, the area of the lung field gradually decreases in the lower part of the lung, but in rare cases, gas in the intestine may be detected as the lung field.

  Such a gas can be eliminated by the region restriction in the CT value range restriction based on the connectivity condition of the lung field from the middle lobe.

  By performing the above determination, the region of the target organ is limited from the image.

  When the boundary of the target organ is determined, the lesion area is then extracted inside the boundary.

  As a means of extracting a lesion area from an image, for example, it has been studied as a diagnostic support technology (Toshiaki Okumura, Tomoko Miwa, Hiroshi Oku, Nobuaki Masudo, Junji Yamamoto, Mitsumi Matsumoto, Norio Kanno, Takeshi Iinuma , Toru Matsumoto: Diagnosis support system for CT for lung cancer screening (LSCT). Computer diagnosis support technology described in Computer Aided Imaging Diagnosis Journal, Vol.2, No.3, 1998) is applied.

  In the present embodiment, a candidate that matches the final diagnosis by the doctor is selected from lesion candidates detected by the computer diagnosis support technology.

  The parameter determination unit 5 determines the slice position including the lesion detected last. Since there are usually a plurality of slices (cross sections) including lesions, all of these cross sections are searched by the image search unit 4 as key images.

  Further, the image search unit 4 may thin out a cross section having a small change in the area or shape of the lesion.

  The key image retrieved by the image retrieval unit 4 is output to the image output unit 6, and is output from the image output unit 6 to a display device, a network computer, or the like. The information output to the image output unit 6 may be a key image alone or may be output with diagnostic information attached to the key image.

  FIG. 2 is a flowchart illustrating image processing according to the first embodiment.

  As shown in FIG. 2, in step S1, an image input unit is used as a medical imaging apparatus such as CT or MRI, or as an image input interface and a storage device associated therewith input from those apparatuses via a network. The photographed image data is input to 1.

  In step S2, a diagnosis result using photographed image data by a doctor is input to the diagnosis information input unit 2 as diagnosis information.

  In step S <b> 3, the keyword input to the diagnostic information input unit 2 or the diagnostic information analysis unit 3 is searched from the keyword table 7.

  In step S <b> 4, the parameter determination unit 5 selects an initial value parameter used in the image search unit 4 from the initial value table 8 based on the input keyword.

  In steps S5 and S6, the image search unit 4 applies the selected parameter to the input image to determine the appropriateness of the parameter based on the area and connectivity of the lesion area (optimization). Lesion area is extracted using computer diagnostic support technology.

  Thereafter, a candidate that matches the final diagnosis by the doctor is selected from the lesion candidates detected by the computer diagnosis support technology.

[Second Embodiment]
FIG. 3 schematically shows functional blocks constituting the image processing apparatus according to the second embodiment of the present invention. The difference from the first embodiment is that the rendering unit 11 and the voxel data creation unit are replaced with the image search unit 4. 12, and the parameter determination unit 5 uses parameters such as opacity and color (density value in the case of a grayscale image) at the time of three-dimensional rendering executed by the rendering unit 11 based on diagnosis information, as in the first embodiment. And the created rendered image is used as a key image.

  Since the other elements are the same as those in the first embodiment, the same numbers are assigned and the description is omitted.

  When a plurality of CT or MRI images are input to the image input unit 1, voxel data generation unit 12 generates voxel data. Voxel data is obtained by joining two-dimensional images in the Z-axis direction in consideration of pixel continuity.

  A three-dimensional rendering image can be obtained by assigning opacity and color to each pixel of the pixel data and applying a volume rendering method, for example.

  In the present embodiment, the volume rendering method is applied, but shading is performed on the surface boundary detected by the threshold selected by the parameter determination unit 5 by a gray level / gradient method in which a three-dimensional density gradient is reflected in shading. A three-dimensional image may be formed by a surface rendering method.

  In addition, the diagnostic information input to the diagnostic information input unit 2 is subjected to keyword search by referring to the keyword table 7 in the diagnostic information analysis unit 3 as in the first embodiment.

  In the parameter determination unit 5, the opacity, color information corresponding to each CT value, and the line of sight when generating a three-dimensional image from the initial value table 8 corresponding to the anatomical part searched by the keyword, the pathological information, and the disease stage. A direction is selected. Basically, volume rendering is executed by the rendering unit 11 using these predetermined setting values.

  On the other hand, it is desirable to perform segmentation in order to clearly express the boundary between organs and lesions. That is, the opacity and color information corresponding to each predetermined CT value may not be able to smoothly reproduce the boundaries of nodules and tumors existing in the lungs, liver and their organs.

  Therefore, by observing the connection in the cross section or between the cross sections with respect to the binary image in which the threshold value of the CT value is changed, the CT value range is converged and binarized in the converged CT value range. By removing the isolated region that occurs at the time of labeling, organs and lesions can be extracted more smoothly.

  A smoother three-dimensional image can be obtained by newly assigning opacity and color information to the pixels constituting the organs and lesions in the segmented image as described above.

  The rendering unit 11 has functions of a shading process and a ray casting process. In the shading process, first, gradient values of normal vectors are calculated for segmented organs and lesion pixels (voxels). The Then, the product (shading value) of the color information and the gradient value of the normal vector (gradient) is obtained. Here, the gradient value is calculated from 2 points on each axis of the target point and 6 points in the vicinity.

  In the ray casting process, a rendering image is created by integrating the voxel values in the line-of-sight direction based on the opacity, shading value, and incident light intensity assigned to the voxels. An optical attenuation calculation is also performed at this point. The gaze direction used at this time is also an important determination parameter.

  Although the direction of observation may be determined diagnostically, when it is behind the human body such as a tumor, it is clearly easier to recognize from the back side.

  That is, the parameter determination unit 5 also selects the direction in which the distance from the organ surface or body surface to a lesion such as a tumor is short in the line-of-sight direction. When there are a plurality of directions of the same distance, a three-dimensional image from a plurality of directions is generated and output.

  The three-dimensional image generated by the rendering unit 11 is output to the image output unit 6, and is output from the image output unit 6 to a display device, a network computer, or the like. Note that the information output to the image output unit 6 may be a single rendered image or may be output with diagnostic information attached to the rendered image.

  FIG. 4 is a flowchart illustrating image processing according to the second embodiment.

  As shown in FIG. 4, in step S1, an image input unit is used as a medical imaging apparatus such as CT or MRI, or as an image input interface and a storage device associated therewith that are input from those apparatuses via a network. The photographed image data is input to 1.

  In step T1, voxel data creation unit 12 creates voxel data from the captured image data.

  In step S2, a diagnosis result using photographed image data by a doctor is input to the diagnosis information input unit 2 as diagnosis information.

  In step S <b> 3, the keyword input to the diagnostic information input unit 2 or the diagnostic information analysis unit 3 is searched from the keyword table 7.

  In step T2, the parameter determination unit 5 selects from the initial value table 8 initial value parameters such as opacity, color, and line-of-sight direction for the three-dimensional rendering executed by the rendering unit 11 based on the input keyword. To do.

  In step T3, the rendering unit 11 applies the selected parameters to the input image, and executes segmentation based on the area of the lesion area, its change, and connectivity in order to clearly represent the organ and lesion boundary. (Adjust).

  In step T4, a gradient value (gradient) of the normal vector is calculated for the segmented organ and lesion voxel in the rendering unit 11, and the product of the color information and the gradient value (gradient) of the normal vector (shading value). ) Is required (shading).

  In step T5, a rendering image is created by integrating the voxel values in the line-of-sight direction based on the opacity assigned to the voxels, the shading value, and the intensity of incident light (ray casting).

  In the second embodiment, the example using the volume rendering method has been described. However, this three-dimensional image generation technique has been extensively studied in addition to medical image applications, and there are many reference books. For example, the description of “Future Image Information Series 5, Computer Graphics” Takeshi Aoiin, other authors, Chapter 7 of Shosodo may be referred to.

  According to the first and second embodiments, a small amount of small amounts of small amounts of captured images are taken from a large amount of medical information on the basis of diagnosis information (organs including lesions, forms of lesions, and progress of disease) of doctors. It is possible to select or generate a key image with an appropriate capacity.

  In addition, a slice image that accurately expresses disease information from a large number of slice images can be designated from the diagnostic information.

  In addition, a three-dimensional image that accurately represents a lesion site can be generated using the same technique, and a small-capacity key image can be obtained.

  In determining the line-of-sight direction when creating a key image, it is possible to select a direction in which the distance from the organ or body surface to the lesion is small, and to set the lesion site so that it can be easily observed.

  In addition, it is possible to construct a reporting system and a reporting method that can output an image selected by the image processing or an image rendered by the image processing attached to diagnostic information and output as a diagnostic report.

[Other embodiments]
The present invention supplies a software program (the code of each step in the flowcharts of FIGS. 2 and 4) for realizing the functions of the above-described embodiments directly or remotely to a system or apparatus, and the computer of the system or apparatus This includes a case where the program code is also achieved by reading and executing the supplied program code. In that case, the form does not have to be a program as long as it has the function of the program.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the claims of the present invention include the computer program itself for realizing the functional processing of the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R).

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Diagnostic information input part, 3 ... Diagnostic information analysis part, 4 ... Image search part, 5 ... Parameter determination part, 6 ... Image output part, 11 ... Rendering part, 12 ... Voxel data creation part

Claims (32)

An image input unit for inputting a plurality of medical images taken;
And a reception unit that receives the input by the user of the diagnostic information for the previous Symbol medical image,
An information analysis unit for analyzing the diagnostic information;
A key image generating unit that generates a key image attached to and output from the diagnostic information in a diagnostic report, based on the analysis result, from the plurality of medical images;
Information processing apparatus according to claim Rukoto equipped with.   The key image generation unit generates the key image by selecting a medical image including a lesion from the plurality of medical images based on a result of the analysis. The information processing apparatus described.   The key image generation unit generates the key image by thinning out medical images in which the size of change in the area or shape of the lesion is smaller than a threshold in the selected plurality of medical images. The information processing apparatus according to claim 2. Wherein A medical image two-dimensional image, the pixel data producing formation unit pixel data to generate a pixel by connecting with continuity in the direction perpendicular to the cross section of the two-dimensional image,
A rendering unit that renders the medical image;
A parameter determining unit that determines a parameter for executing the rendering process based on a result of the analysis;
The information analysis unit analyzes the diagnosis information and acquires a keyword related to the diagnosis information as a result of the analysis.
The parameter determination unit, the information processing apparatus according to claim 1, characterized in that determining the parameters when executing the rendering process from the pixel data and the keyword. The parameters include opacity, color information corresponding to each CT value of the medical image, and a line-of-sight direction when generating a three-dimensional image, and these initial values are selected by the parameter determination unit from a table stored in advance. information processing apparatus according to claim 4, characterized in that it is. Information processing apparatus according to claim 5, further comprising a optimization unit for optimizing a parameter selected from the table by the parameter determining unit to the medical image. The rendering unit performs each process of shading and ray casting with respect to the pixel data, in the shading process, the optimized color information parameters for the application to that were generated pixel data and shading calculated values, in the ray casting process, opacity assigned to the pixel data, shading value, and to claim 6, characterized in that generate the rendered image in the viewing direction based on the intensity of the incident light information processing apparatus according.   The information analysis unit uses a keyword extracted from the diagnostic information as a result of the analysis,
  The information processing apparatus according to claim 1, wherein the key image generation unit generates the key image using a parameter determined based on the keyword.   The information processing apparatus according to claim 8, wherein the information analysis unit uses an anatomical term and a pathological term as a result of the analysis from the diagnosis information.   The key image generation unit determines a region of an organ targeted for diagnosis from the medical image, extracts a lesion region inside a boundary of the determined region, and a medical image including the extracted lesion The information processing apparatus according to claim 1, wherein the information processing apparatus generates the key image as the key image.   The information processing apparatus according to claim 1, further comprising an output unit that outputs the diagnostic information with the key image attached thereto.   11. The information processing according to claim 1, further comprising a display control unit configured to display a diagnostic report in which the key image is attached to the diagnostic information by outputting the diagnostic report to a display device. apparatus.   An image input unit for inputting a plurality of medical images taken;
  A reception unit that receives input by the user of diagnostic information related to the medical image and keywords related to the diagnostic information;
  A key image generating unit that generates a key image attached to and output from the diagnostic information in a diagnostic report from the plurality of medical images based on the keywords;
An information processing apparatus comprising: An image input process for inputting a plurality of medical images taken;
A reception step of receiving the input by the user of the diagnostic information for the previous Symbol medical image,
An information analysis step of analyzing the diagnostic information;
A key image generating step for generating a key image attached to and output from the diagnostic information in a diagnostic report from the plurality of medical images based on the result of the analysis;
Information processing method comprising: a.   The key image generation step generates the key image by selecting a medical image including a lesion from the plurality of medical images based on a result of the analysis. The information processing method described.   In the key image generation step, the key image is generated by thinning out medical images in which the magnitude of change in the area or shape of the lesion is smaller than a threshold in the selected plurality of medical images. The information processing method according to claim 15. The medical image is a two-dimensional image, the pixel data producing formation step of the pixel data to generate a pixel by connecting with continuity in the direction perpendicular to the cross section of the two-dimensional image,
A rendering process for rendering the medical image;
A parameter determining step of determining a parameter for executing the rendering process based on the result of the analysis,
The information analysis step analyzes the diagnostic information and acquires a keyword related to the diagnostic information as a result of the analysis.
Wherein the parameter determining step, information processing method according to claim 14, characterized in that determining the parameters when executing the rendering process from the pixel data and the keyword. The parameters include opacity corresponding to each CT value of the medical image, color information, and line-of-sight direction when generating a three-dimensional image, and these initial values are selected from the table stored in advance by the parameter determination step. information processing method according to claim 17, characterized in that it is. Information processing method according to claim 18, further comprising a optimization process for optimizing the parameters selected from the table by the parameter determining step with respect to the medical image. The extent rendering Engineering executes each process of shading and ray casting with respect to the pixel data, in the shading processing, color information for the optimized parameters applied to that were generated pixel data calculated shading value, the ray a casting process, the opacity assigned to the pixel data, according to claim 19, wherein the shading value, and to generate the rendered image in the viewing direction based on the intensity of the incident light information processing method according to.   In the information analysis step, a keyword extracted from the diagnostic information is used as a result of the analysis,
  The information processing method according to any one of claims 14 to 20, wherein the key image generation step generates the key image using a parameter determined based on the keyword.   The information processing method according to claim 21, wherein the information analysis step uses an anatomical term and a pathological term as a result of the analysis from the diagnosis information.   The key image generating step determines a region of an organ targeted for diagnosis from the medical image, extracts a lesion region inside a boundary of the determined region, and includes a medical image including the extracted lesion 23. The information processing method according to claim 14, wherein the key image is generated as the key image.   The information processing method according to any one of claims 14 to 23, further comprising an output step of outputting the diagnostic information with the key image attached thereto.   The information processing according to any one of claims 14 to 23, further comprising a display control step of displaying a diagnostic report in which the key image is attached to the diagnostic information by outputting the diagnostic report to a display device. Method.   An image input process for inputting a plurality of medical images taken;
  An accepting step of accepting input by the user of diagnostic information related to the medical image and keywords related to the diagnostic information;
  A key image generating step of generating a key image attached to the diagnostic information and output in the diagnostic report from the plurality of medical images based on the keyword;
An information processing method comprising: Program for executing the information processing method according to the computer in any one of claims 14 to 26. A computer-readable storage medium storing the program according to claim 27 .   An image input unit for inputting a plurality of medical images taken;
  A reception unit that receives input by a user of diagnostic information related to the medical image;
  An information analysis unit for analyzing the diagnostic information;
  A key image generating unit that generates a key image attached to and output from the diagnostic information in a diagnostic report, based on the analysis result, from the plurality of medical images;
An information processing system comprising:   The medical image is a two-dimensional image, and a pixel data generation unit that generates pixel data in which the pixels of the two-dimensional image are connected in a direction orthogonal to a cross section with continuity;
  A rendering unit that renders the medical image;
  A parameter determining unit that determines a parameter for executing the rendering process based on a result of the analysis;
  The information analysis unit analyzes the diagnosis information and acquires a keyword related to the diagnosis information as a result of the analysis.
  30. The information processing system according to claim 29, wherein the parameter determination unit determines a parameter for executing the rendering process from the keyword and the pixel data.   31. The information processing system according to claim 29 or 30, further comprising an output unit that outputs the diagnostic information with the key image attached thereto.   The information processing system according to claim 29 or 30, further comprising a display control unit configured to display a diagnostic report in which the key image is attached to the diagnostic information by outputting the diagnostic report to a display device.

Images