JP6387166B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to a technique for handling a tomographic image.

  In the medical field, a doctor makes a diagnosis using medical images (three-dimensional image data representing three-dimensional information inside a subject) captured with a plurality of modalities and medical images captured at different dates and times. . In order to use a plurality of types of medical images for diagnosis, it is important to associate (identify) a site (a site of interest, a site of interest) such as a lesion site of interest in each medical image. Therefore, the doctor responds to the lesion by using the similarities such as the shape of the lesion and the appearance of the periphery thereof while looking at the image of the lesion of interest pointed out on one medical image. An operation of searching for a region (corresponding region, corresponding lesion) from the other medical image is performed.

  In the department of mammary gland, lesions etc. are pointed out on the MRI image of the breast imaged in the prone position, and then the corresponding lesion in the ultrasonic tomographic image is searched (identified) for diagnosis by ultrasonic examination in the supine position. May be performed. However, since the subject breast is soft and there is a large difference in body position, there is a problem that the position and appearance of the lesioned part greatly change. Therefore, it is not easy to search for a corresponding lesion, and some kind of computer-assisted load reduction is required.

  Patent Document 1 discloses a technique for estimating breast deformation from the prone position to the supine position. By using this deformation estimation result, the position of the lesion in the supine position can be estimated, and the estimated position can be presented as support information for ultrasonic probe operation. Patent Document 2 discloses a technique for automatically extracting a nipple position from a simple X-ray image of a breast and drawing a region in the image included in a certain range therefrom in the image.

JP 2011-123682 A JP 2010-227215 A

  When the method described in Patent Document 1 is used, calculation of deformation simulation such as a finite element method is necessary, and thus there is a problem that an operator (physician) has to wait until the calculation is completed. In addition, there are some pieces of measurement information to be input by the operator in order to perform the deformation simulation, but there is a problem that the input effort may not be allowed. Furthermore, there remains a problem that it is not easy to accurately estimate the deformation of the breast. Further, in the method described in Patent Document 2, it is possible to know a region within a certain range from the feature point of the subject. However, since this is only a cross-section including the feature point in the image, the cross-section can be selected with any cross-section. I cannot know the area.

  The present invention has been made in view of such problems, and is intended to derive and present support information for searching for a lesion in an image by simple calculation without using much measurement information. Provide technology.

According to one aspect of the present invention, acquisition means for acquiring a two-dimensional image of a subject;
Reference point position acquisition means for acquiring information representing the three-dimensional position of the reference point of the subject in space;
Identifying means for identifying a location in the two-dimensional image corresponding to a portion separated by a predetermined three-dimensional distance from the three-dimensional position of the reference point;
Generating means for generating a synthesized image obtained by synthesizing information indicating the location on the two-dimensional image ,
The specifying means specifies a circle obtained by cutting a sphere centered on the three-dimensional position of the reference point and having a radius of the predetermined three-dimensional distance in a cross section corresponding to the two-dimensional image, Identify the arc included in the two-dimensional image as the location,
The generating means generates the composite image in which the arc is superimposed on the two-dimensional image .

  According to the configuration of the present invention, it is possible to derive and present support information for searching for a lesioned part or the like in an image only by simple calculation without using much measurement information.

The block diagram which shows the function structural example of an information processing system. The block diagram which shows the hardware structural example of a computer. The flowchart of the process which a computer performs. The figure which shows an example of a synthesized image. The block diagram which shows the function structural example of an information processing system. The flowchart of the process which a computer performs. The figure which shows an example of a synthesized image. The figure which shows an example of a synthesized image. The figure which shows an example of a synthesized image. The figure which shows an example of a synthesized image. FIG. 6 is a diagram for explaining information held in a data server 560. The figure explaining statistical information. The block diagram which shows the function structural example of an information processing system. The flowchart of the process which a computer performs. It is a figure explaining the example of the synthesized image in 3rd Embodiment.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
The information processing system according to the present embodiment searches for (identifies) a corresponding part on an ultrasonic tomographic image by ultrasonic examination after an attention part such as a lesion is pointed out on an image acquired by a modality such as MRI. In this case, the search support information is presented to the operator. This system tends to maintain a three-dimensional distance from the nipple to the site of interest, such as a lesion, regardless of the body position, such as prone or supine, when the breast is examined. Based on the knowledge that the corresponding part exists, a place where the corresponding part is likely to exist is presented on the ultrasonic tomographic image. In addition, by displaying information on the ultrasonic tomographic image so that the distance from the nipple to each point on the image can be grasped, it is easy to search for a corresponding part on the ultrasonic tomographic image. The information processing system according to this embodiment will be described below.

  First, a functional configuration example of the information processing system according to the present embodiment will be described with reference to the block diagram of FIG. As illustrated in FIG. 1, the information processing system according to the present embodiment includes a data server 160, an information processing apparatus 100, a medical image collection apparatus 170, and a display unit 180.

  First, the data server 160 will be described. The data server 160 holds information such as a three-dimensional distance from the reference point (papillae) of the subject to be observed to the site of interest (distance between the sites of interest of the reference point) and the like to the information processing apparatus 100. Output. The distance between the reference point attention sites held by the data server 160 is a numerical value calculated in advance from another three-dimensional medical image such as an MRI image obtained by imaging the same subject. This value is not shown, for example, by detecting the position of the reference point (papillary position) from the MRI image by image processing and calculating the three-dimensional distance from the position of the target region pointed out by the operator on the image. It can be acquired by executing this image processing system.

  In FIG. 1, the data server 160 is shown as an external device of the information processing apparatus 100. However, the data server 160 may be a device in the information processing apparatus 100, for example, a device such as a hard disk built in the information processing apparatus 100. It doesn't matter.

  Next, the medical image collection apparatus 170 will be described. In the present embodiment, the medical image collection apparatus 170 will be described as an ultrasonic diagnostic imaging apparatus. The medical image acquisition apparatus 170 captures an ultrasonic tomographic image (tomographic image) of a subject in real time. The ultrasonic tomographic images of each frame imaged by the medical image acquisition apparatus 170 are sequentially input to the information processing apparatus 100 via the tomographic image acquisition unit 110. The medical image collection apparatus 170 has a sensor system for measuring the position and orientation (three-dimensional position and orientation) of an ultrasound probe (not shown) in the space in real time. The position and orientation of the ultrasound probe measured by this sensor system are sequentially input to the information processing apparatus 100 via the position and orientation acquisition unit 120. Various configurations are conceivable for the sensor system for measuring the position and orientation of the ultrasonic probe. For example, an image of an ultrasonic probe may be captured in real time, and the position and orientation of the ultrasonic probe may be obtained from the image by calculation, or a known sensor using a magnetic sensor, an ultrasonic sensor, or an optical sensor may be used. You may measure by the measuring method.

  Next, the information processing apparatus 100 will be described. For example, a general PC (personal computer) or the like can be applied to the information processing apparatus 100, or dedicated hardware can be applied.

  The reference distance acquisition unit 105 acquires the distance between the reference point attention sites transmitted from the data server 160 as the reference distance. As described above, the tomographic image acquisition unit 110 transfers the ultrasonic tomographic images sequentially transmitted from the medical image collection device 170 to the subsequent image generation unit 140. As described above, the position / orientation acquisition unit 120 transfers the position / orientation sequentially transmitted from the medical image collection apparatus 170 to the subsequent image generation unit 140.

  The reference point position acquisition unit 130 acquires the position of the reference point in the space using the position and orientation of the ultrasonic probe based on an instruction from the operator.

  The image generation unit 140 performs calculation processing to be described later based on the reference distance, the position of the reference point, and the position and orientation of the ultrasonic probe, so that the reference point is the center and the reference distance is the radius. A portion (circle) obtained by cutting the sphere at the tomographic plane corresponding to the ultrasonic tomographic image is calculated. Then, the image generation unit 140 superimposes (combines) information on a portion (arc) included in the ultrasonic tomographic image of the calculated circle on the ultrasonic tomographic image, and sends the information to the display control unit 150.

  The display control unit 150 sends the composite image sent from the image generation unit 140, that is, the image in which the information about the arc is superimposed on the ultrasonic tomographic image to the display unit 180, so that the image is displayed on the display unit 180. To display.

  The display unit 180 is configured by a CRT, a liquid crystal screen, or the like, and displays a processing result by the information processing apparatus 100 using an image, text, or the like. The display unit 180 can display, for example, a composite image or a GUI (graphical user interface) described later.

  Here, all the functional units constituting the information processing apparatus 100 may be configured by hardware, but a part thereof may be configured by software (computer program). In this case, the corresponding function is realized by the software being executed by the control unit included in the information processing apparatus 100. Further, one or more of the functional units constituting the information processing apparatus 100 may be external independent devices. Of course, each functional unit constituting the information processing apparatus 100 may be realized by software.

  In the present embodiment, each functional unit constituting the information processing apparatus 100 will be described as being configured by software. In this case, a computer having a hardware configuration as shown in FIG. 2 can be applied to the information processing apparatus 100.

  The CPU 1001 controls the operation of the entire computer using computer programs and data stored in the RAM 1002 and the ROM 1003, and executes each process described later as what the information processing apparatus 100 to which this computer applies.

  The RAM 1002 has an area for temporarily storing computer programs and data loaded from the external storage device 1007 and the storage medium drive 1008 and data received from an external device via an I / F (interface) 1009. The RAM 1002 also has a work area used when the CPU 1001 executes various processes. That is, the RAM 1002 can provide various areas as appropriate. The ROM 1003 stores setting data and a boot program for the computer.

  A keyboard 1004 and a mouse 1005 are used to input various instructions to the CPU 1001 when operated by an operator of the computer. However, devices other than the keyboard 1004 and the mouse 1005 may be used as long as they are devices for inputting various instructions to the CPU 1001.

  The external storage device 1007 functions as a large-capacity information storage device such as a hard disk drive device. The external storage device 1007 stores a computer program and data for causing the CPU 1001 to execute processes described later as those performed by the OS (operating system) and the information processing apparatus 100 to which the present computer is applied. The computer program stored in the external storage device 1007 includes a computer program for causing the CPU 1001 to execute the functions of the function units (105 to 150) illustrated in FIG. Yes. Data stored in the external storage device 1007 includes data handled as known information in the following description.

  The computer programs and data described as being stored in the external storage device 1007 are appropriately loaded into the RAM 1002 under the control of the CPU 1001 and are processed by the CPU 1001. Information described as being stored in the data server 160 may be stored in the external storage device 1007.

  The storage medium drive 1008 reads out computer programs and data stored in a storage medium such as a CD-ROM or DVD-ROM in accordance with instructions from the CPU 1001 and outputs them to the RAM 1002 or the external storage device 1007. Some of the computer programs and data described as being stored in the external storage device 1007 may be stored in this storage medium.

  The I / F 1009 includes an analog video port or a digital input / output port such as IEEE 1394, an Ethernet port for outputting information such as a composite image to the outside, and the like. The I / F 1009 is connected to the data server 160, the medical image collection device 170, and the display unit 180. As one form, some of the functions of the reference distance acquisition unit 105, the tomographic image acquisition unit 110, the position and orientation acquisition unit 120, and the reference point position acquisition unit 130 are realized by the I / F 1009. Each unit described above is connected to each other via a bus 1010.

  Next, processing performed by the computer of FIG. 2 that operates as the information processing apparatus 100 will be described with reference to FIG. 3 showing a flowchart of the processing. Note that a computer program and data for causing the CPU 1001 to execute processing according to the flowchart of FIG. 3 are stored in the external storage device 1007. The CPU 1001 loads the computer program and data from the external storage device 1007 to the RAM 1002, and executes processing using the computer program and data. As a result, the computer of FIG. 2 executes processing according to the flowchart of FIG.

(S3000: Acquisition of reference distance)
In step S3000, the CPU 1001 functions as the reference distance acquisition unit 105, and acquires the distance between the reference point attention sites transmitted from the data server 160 as the reference distance. If the data server 160 does not hold the distance between the reference point attention parts and the distance between the reference point attention parts cannot be obtained from the data server 160, the reference distance is not acquired in this step.

(S3010: Acquisition of tomographic image)
In step S3010, the CPU 1001 functions as the tomographic image acquisition unit 110, and acquires an ultrasonic tomographic image from the medical image collection apparatus 170. In this step, the CPU 1001 functions as the position / orientation acquisition unit 120 and acquires the position / orientation of the ultrasonic probe when the ultrasonic tomographic image is captured from the medical image acquisition apparatus 170.

(S3020: Acquisition of reference point position)
In step S3020, the CPU 1001 functions as the reference point position acquisition unit 130, and acquires the position of the reference point (papillary position) in the space. This acquisition process is performed when the operator operates the keyboard 1004 in a state where a predetermined part of the ultrasonic probe (for example, the center of the probe surface) is in contact with the nipple position of the subject (for example, the reference This is executed when a key to which a point position acquisition command is assigned is pressed. That is, the position of the predetermined part is obtained from the position and orientation of the ultrasonic probe acquired by the position and orientation acquisition unit 120 at the time when the above operation is executed, and this is acquired as the position of the reference point. When obtaining the position of the predetermined part from the position and orientation of the ultrasonic probe, there are a method of applying a prescribed bias to the position and orientation and a method of carrying out a prescribed matrix conversion.

  When the reference point exists in the body of the subject, the operator operates the ultrasonic probe so that the reference point is drawn on the ultrasonic tomographic image, and then When the operator points out the coordinates of the reference point, the position is acquired.

(S3030: Image composition)
In step S <b> 3030, the CPU 1001 functions as the image generation unit 140, generates an image in which information indicating a place where a corresponding site is likely to exist is superimposed on an ultrasonic tomographic image, and sends the generated image to the display unit 180.

  Here, processing performed by the image generation unit 140 (CPU 1001) will be described. First, the image generation unit 140 obtains a sphere having the radius of the reference distance obtained in step S3000 with the position of the reference point obtained in step S3020 as the center. Next, based on the position and orientation of the ultrasonic probe obtained in step S3010, a plane in the space including the ultrasonic tomographic image (that is, an ultrasonic cross section (imaging cross section)) is obtained, and the sphere is defined on the plane. Calculate the circle that can be cut. Finally, a portion (arc) included in the ultrasonic tomographic image in the calculated circle is superimposed on the ultrasonic tomographic image to generate a composite image.

  FIG. 4 shows an example of a composite image generated by this process. In FIG. 4, an arc 430 included in the ultrasonic tomographic image is included in the ultrasonic tomographic image 440 out of a circle formed by cutting out a sphere having a reference distance 420 as a center with the nipple position 410 serving as the reference point position cut by the plane. It is superimposed. Here, the arc 430 is a set of points where the three-dimensional distance from the nipple position 410 matches the reference distance obtained in step S3000. If the three-dimensional distance from the nipple to the lesioned part is maintained in the subject, the corresponding part 450 does not exist outside the vicinity of the arc 430, so the operator can search for the corresponding part 450 by relying on the arc 430. It can be carried out.

  It should be noted that auxiliary information indicating how far each point on the ultrasonic tomographic image is from the reference distance (the three-dimensional distance from the reference point to each point on the tomographic image and the reference distance) Information indicating the relationship) may be presented superimposed on the ultrasonic tomographic image. For example, several distances are obtained by adding or subtracting a predetermined value to the reference distance. Then, for each of the obtained distances, a sphere having the radius as the center around the position of the reference point obtained in step S3020 is obtained. Then, for each sphere obtained for each distance, a circle to be cut out in the above plane is obtained in the same manner, and a portion (arc) included in the ultrasonic tomographic image of the obtained circle is superimposed on the ultrasonic tomographic image, and a composite image is obtained. Is generated.

  FIG. 7 shows an example of a composite image generated by this processing when a plurality of values (for example, 5 mm, 10 mm, and 15 mm) set at equal intervals are used as predetermined values. In FIG. 7, together with the arc 430 in FIG. 4, arcs obtained for several concentric spheres whose radius is a value obtained by adding or subtracting a predetermined value to the reference distance are superimposed on the ultrasonic tomographic image 440 as auxiliary lines 735. . Also, character information (such as “−5 mm” and “+10 mm”) indicating how far each auxiliary line is away from the reference distance is superimposed in the vicinity of the auxiliary line as additional information. By displaying such a composite image on the display unit 180, the operator can know how far each point on the image is from the distance between the reference point attention sites. The line type and color of the auxiliary line may be changed according to the distance from the reference distance. These auxiliary lines may always be drawn, or may be configured such that the operator can select whether to draw by operating the keyboard 1004 or the mouse 1005.

  If the reference distance is not acquired in step S3000, the image generation unit 140 executes the following process instead of the above process. First, a plurality of spheres having a radius of a predetermined distance (for example, each distance in increments of 10 mm from 10 mm to 100 mm) are calculated with the position of the reference point obtained in step S3020 as the center. Then, based on the position and orientation of the ultrasonic probe obtained in step S3010, a circle that can be obtained by cutting these spheres along an ultrasonic cross section is calculated. Then, a composite image is generated by superimposing on the ultrasonic tomographic image information indicating the distance (arc) included in the ultrasonic tomographic image and the respective distances in the calculated plurality of circles. FIG. 8 shows an example of a composite image generated by this processing. In FIG. 8, an arc obtained for a concentric sphere having a radius of a predetermined distance (20 mm to 60 mm) around the nipple position 410 is superimposed on the ultrasonic tomographic image 440 as an arc 830. This display is particularly effective when the operator grasps the distance between the reference point attention sites in his / her head. That is, even when the system does not obtain the distance between the reference point attention sites, the operator searches for the corresponding site 450 by relying on the distance information from the nipple presented on the ultrasonic tomographic image. Can do. Even when the reference distance is acquired in step S3000, the display (display of the distance between the reference point attention sites) and the main display (predetermined, etc.) according to instructions from the operator via the keyboard 1004 and the mouse 1005 The display may be arbitrarily switched (display of the distance of the interval). Moreover, the structure which mixes and presents these information may be sufficient. If the position of the reference point has not been acquired, the image generation unit 140 uses the ultrasonic tomographic image as an output image.

(S3040: End determination)
In step S3040, the CPU 1001 determines whether to end the entire process. For example, when it is detected that a predetermined key (end key) on the keyboard 1004 is pressed by the operator, it is determined that the process is to be ended.

  If it is determined that the process is to be ended, the CPU 1001 ends the entire process. On the other hand, if it is not determined that the process is to be terminated, the process returns to step S3010, and the processes after step S3010 are executed again on the newly acquired ultrasonic tomographic image.

  In the present embodiment, the case where the breast is the subject and the nipple is the reference point has been described as an example. However, if the subject tends to store the distance between the region of interest and the reference point, the subject or The reference point may be any.

  Thus, according to the present embodiment, the operator can easily grasp the three-dimensional distance from the reference point (nipple) to each point on the ultrasonic image. In particular, a point where the distance from the reference point (papillae) is the “distance between the reference point attention sites” or a point in the vicinity thereof can be presented to the operator where there is a high possibility that the corresponding site exists. . According to the present embodiment, since a guideline for searching for a corresponding part in the ultrasonic tomographic image is displayed in the ultrasonic tomographic image, the range in which the operator searches for the corresponding part can be limited. In addition, since the search range can be limited, the operator's workload can be reduced and the risk of incorrect association can be reduced. Further, since calculation such as deformation estimation is not performed, it is possible to support the search for the corresponding part without waiting for the operator. In addition, since less input information is required, it is possible to assist in searching for a corresponding part without bothering the operator.

<Modification of First Embodiment>
In the first embodiment, an example in which an ultrasonic diagnostic imaging apparatus is used as the medical image collection apparatus 170 is described as an example. However, the medical image collection apparatus 170 may be any other modality. When the medical image collection apparatus 170 is CT or MRI, the information processing apparatus 100 acquires a three-dimensional image in advance. In the process of step S3010, a cross section of interest is designated using a GUI as in a known medical image viewer, a cross section image is acquired from the three-dimensional image, and the position and orientation of the cross section are acquired. That's fine Further, in the process of step S3020, the three-dimensional coordinates of the reference point in the three-dimensional image may be acquired using a GUI as in a known medical image viewer.

  Further, the distance between the reference point attention sites held by the data server 160 may not be obtained from the MRI image. For example, the distance obtained by obtaining the position of the reference point and the region of interest from an image obtained in other modalities or past ultrasonic examinations may be used.

  In the first embodiment, the reference distance acquisition unit 105 acquires the distance between reference point attention parts from the data server 160, but the method of acquiring the distance between reference point attention parts is not limited to this. For example, the reference distance acquisition unit 105 may acquire the distance input by the operator using the keyboard 1004 or the mouse 1005 using a GUI (not shown) as the reference distance. Further, the reference distance acquisition unit 105 acquires a medical image of the subject from the data server 160, and the reference distance acquisition unit 105 acquires the position of interest and the position of the reference point (papillary position) from the medical image, and the reference point attention part. It may be configured to calculate the inter-distance. In addition, the reference distance acquisition unit 105 acquires the medical image of the subject and the position of the target region from the data server 160, and the reference distance acquisition unit 105 acquires only the position of the reference point from the medical image, so The structure which calculates a distance may be sufficient. When the reference distance acquisition unit 105 acquires the position of the target region or the reference point from the medical image, the operator may manually input the position, or may be automatically detected by image processing. There may be.

[Second Embodiment]
In the first embodiment, the embodiment that can be used in common for the objects that tend to store the distance between the reference point attention sites (the object is not limited) has been described. In the second embodiment, the distance between the nipple attention sites is acquired from the MRI image in the prone position, and the object, that is, the organ (breast) and the reference point (nipple) are supported as in the case of supporting the supine ultrasound examination. This is an embodiment in the case where the front and back positions (the prone position and the supine position) are specified. The present embodiment is characterized in that statistical information on the distance between reference point attention sites regarding such a specific target is acquired, and support information is controlled based on the statistical information. Furthermore, in this embodiment, taking into consideration that the statistical behavior of the distance between nipple attention regions varies depending on the attributes of the subject and the attention area, the support information is controlled based on the statistical information corresponding to the attributes of the subject and the attention area. It is characterized by doing. Here, the attributes of the subject and the attention area are attributes whose statistical behavior of the distance between the nipple attention areas is different depending on the difference, for example, the age of the subject, the bust size, the attention area within the breast Is a region to which the file belongs. Hereinafter, only the difference from the first embodiment of the information system according to the present embodiment, which is characterized in that the display is controlled using statistical information, will be described. That is, it is the same as that of the first embodiment unless otherwise mentioned below.

  A functional configuration example of the information processing system according to the present embodiment will be described with reference to the block diagram of FIG. In FIG. 5, the same functional units as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. As illustrated in FIG. 5, the information processing system according to the present embodiment includes a data server 560, an information processing apparatus 500, a medical image collection apparatus 170, and a display unit 180.

  The data server 560 holds the three-dimensional image data of the subject from which the distance between the reference point attention sites of the subject is acquired. Further, the position (three-dimensional coordinates) of the target region in the three-dimensional image data is held. In addition, information indicating which region in the breast the attention site belongs to is held. For example, as shown in FIG. 11A, information indicating whether the breast belongs to the inside or the outside is held. Alternatively, as shown in FIG. 11 (b), the A region (inner upper portion), B region (inner lower portion), C region (outer upper portion), D region (outer lower portion), and E region (areola portion) of the breast. It holds information indicating to which it belongs. It also holds information such as the subject's age and bust size (cup size, breast area volume, etc.). The data server 560 outputs the stored data to the information processing apparatus 500.

  Corresponding distance calculation unit 510 receives from data server 560 the position of the region of interest in the 3D image data of the subject, the region of interest in the 3D image data, which region in the breast the region of interest belongs to, and the age of the subject And bust size information. Then, the corresponding distance calculation unit 510 acquires the statistical information of the distance between the reference point attention parts based on these pieces of information, and based on this statistical information, the distance between the reference point attention parts (reference distance) from the reference point corresponding part distance. An estimated value (hereinafter referred to as a corresponding distance) is calculated. Then, the calculated corresponding distance value is output to the image generation unit 540.

  Here, the statistical information and the corresponding distance in the present embodiment will be described. For example, when there is statistical information that the distance between the nipple attention regions on average is α times when the body position changes from the prone position to the supine position, the value obtained by multiplying the reference distance r [mm] by α is the corresponding distance. Let r ′ be the distance where the corresponding site is most likely to exist. Further, when there is statistical information that the distance between the nipple attention sites is maintained with the standard deviation σ [mm] when the body position changes from the prone position to the supine position, r−2σ, r−σ, r, r + σ, r + 2σ. Each distance is taken as a corresponding distance (information indicating a range of distances where a corresponding part is likely to exist). When both α and σ are obtained, the distances r′−2σ, r′−σ, r ′, r ′ + σ, and r ′ + 2σ are set as corresponding distances. That is, the statistical information in the present embodiment represents the magnification α and the standard deviation σ described above. Further, the correspondence distance represents a distance and a range where a corresponding portion is likely to exist after a change in body posture, which is derived from the statistical information and the reference distance.

  Corresponding distance calculation unit 510 was calculated by performing statistical processing (without distinguishing cases) on these using the measured values of the nipple attention site distance in the prone position and the supine position collected for a large number of cases. The above α and σ are managed as statistical information that does not depend on the subject. In addition, the correspondence distance calculation unit 510 classifies a large number of cases according to the region (inside / outside or A to E region) to which the lesion belongs, and then performs statistical processing for each region. Information is managed. Similarly, the corresponding distance calculation unit 510 divides a subject's age and bust size, distance between nipple attention areas, distance between body area attention areas, mammary gland density, and the like into predetermined sections, and classifies a large number of cases according to respective criteria. The statistical information for each section in each reference calculated by performing statistical processing is managed. As statistical information, it is not essential to use values obtained by actually performing statistical processing on a large number of cases, and a configuration using manually set values (by the device designer or the like) may be used. .

  Note that the method of managing statistical information is not limited to the method of holding statistical information for each section subjected to statistical processing, and may be another method. For example, a function of x that approximates statistical information is held using at least one combination of the subject's age, bust size, distance between nipple attention regions, body surface attention region distance, mammary gland density, and the like as input parameters x. It may be. That is, the statistical information may be managed in the form of a function f_α (x) that outputs α with x as an input or a function f_σ (x) that outputs σ with x as an input.

  Based on the correspondence distance calculated by the correspondence distance calculation unit 510, the reference point position, and the position and orientation of the ultrasonic probe, the image generation unit 540 generates a sphere centered on the reference point and having a radius of the correspondence distance on the tomographic plane. Calculate the circle that can be cut. Then, the image generation unit 540 generates a composite image by superimposing information on a portion (arc) included in the ultrasonic tomographic image of the calculated circle on the ultrasonic tomographic image, and outputs the composite image to the display control unit 150. Send it out. That is, a composite image is generated in the same manner as in the first embodiment except that the corresponding distance is used instead of the reference distance.

  When each functional unit in the information processing apparatus 500 shown in FIG. 5 is implemented by a computer program, the computer shown in FIG. 2 is applied to the information processing apparatus 500 as in the first embodiment. Can do. That is, when each functional unit in the information processing apparatus 500 is implemented by a computer program, the computer program is executed by the CPU 1001, and thus the computer functions as the information processing apparatus 500.

  Next, processing performed by the computer of FIG. 2 that operates as the information processing apparatus 500 will be described with reference to FIG. 6 showing a flowchart of the processing. Note that a computer program and data for causing the CPU 1001 to execute processing according to the flowchart of FIG. 6 are stored in the external storage device 1007. The CPU 1001 loads the computer program and data from the external storage device 1007 to the RAM 1002, and executes processing using the computer program and data. As a result, the computer of FIG. 2 executes processing according to the flowchart of FIG.

  In FIG. 6, steps S6000, S6010, S6020, and S6040 are the same processing steps as steps S3000, S3010, S3020, and S3040 in the flowchart shown in FIG. 3, and thus description of these steps is omitted.

(S6003: Acquisition of calculation method)
In step S6003, the CPU 1001 acquires a corresponding distance calculation method input by the operator pressing a predetermined key on the keyboard 1004 or the like. In this embodiment, the operator selects one of the following calculation methods.

1. 1. Calculation based on statistical information that does not depend on the subject 2. Calculation based on the region to which the site of interest belongs 3. Calculation based on subject's age and bust size 4. Calculation based on the distance from the nipple of the region of interest (distance between nipple regions of interest) 5. Calculation based on the distance from the body surface of the region of interest (distance between body surface regions of interest) Calculation based on the mammary gland density of the subject For example, the display unit 180 displays a GUI for selectively displaying these six options. When the operator inputs an instruction to select one of the six options using the keyboard 1004 or the mouse 1005, the CPU 1001 detects the selected option as a calculation method to be used later. May be.

  Of course, the method for designating the calculation method is not limited to such a method, and may be determined in advance, or may be determined according to the examination contents and the department to which the computer belongs. Also, the options (calculation methods) are not limited to the above six.

(S6005: Acquisition of various data)
In step S6005, the CPU 1001 functions as the corresponding distance calculation unit 510. As a result, the CPU 1001 receives from the data server 560 the reference distance, the 3D image data of the subject, the position of the region of interest in the 3D image data, which region in the breast the region of interest belongs to, and the subject. Get age and bust size information.

(S6025: Calculation of corresponding distance)
In step S6025, the CPU 1001 functions as the corresponding distance calculation unit 510, and calculates the above-described corresponding distance based on the data acquired in step S6005. This process obtains the magnification α and the standard deviation σ as statistical information, and based on the values, the representative value r ′ of the corresponding distance from the reference distance r, and the range r′−2σ, r′−σ, r ′ + σ. , R ′ + 2σ is executed.

  If Method 1 (calculation based on statistical information that does not depend on the subject) is selected in step S6003, statistical information α and σ that do not depend on the subject are selected as statistical information to be used later.

If method 2 (calculation based on the region to which the site of interest belongs) is selected in step S6003, α corresponding to the region (inside / outside, or A to E regions) to which the site of interest acquired in step S6005 belongs. The value of σ is selected as statistical information to be used later. For example, as shown in FIG. 12A, it is assumed that the site of interest of the subject 1 exists in the C region and the site of interest of the subject 2 exists in the B region. At this time, as the processing of this step, the subject 1 selects the magnification α _C and the variance σ _{C that} are statistical information corresponding to the C region. In the subject 2, the magnification α _B and the variance σ _B which are statistical information corresponding to the B region are selected.

  If Method 3 (calculation based on subject's age and bust size) is selected in step S6003, the values of α and σ corresponding to the combination of the subject's age and bust size acquired in step S6005 are set as follows. Select as statistical information used in.

  When Method 4 (calculation based on the distance between nipple attention regions) is selected in step S6003, the values of α and σ corresponding to the distance between nipple attention regions acquired in step S6000 are used as statistical information to be used later. select.

  When Method 5 (calculation based on the distance between the body surface attention sites) is selected in step S6003, the distance between the body surface attention sites is derived, and α and σ corresponding to the derived body surface attention site distances are derived. The value is selected as statistical information to be used later. Here, the derivation of the body surface attention site distance is performed by the following processing. First, the CPU 1001 functions as the corresponding distance calculation unit 510, performs image processing on the three-dimensional image data, and derives a body surface region (a boundary region between the breast region and the outside of the body). Then, the nearest point of the body surface position with respect to the site of interest is searched, and the distance from the site of interest to the nearest point is calculated as the distance between the body surface sites of interest.

  If method 6 (calculation based on the mammary gland density of the subject) is selected in step S6003, the mammary gland density of the subject is derived, and the values of α and σ corresponding to the derived mammary gland density are thereafter determined. Select as statistical information used in. Here, the mammary gland density of the subject is derived by the following processing. For example, a breast region is acquired from three-dimensional image data, a fat region and a mammary gland region are classified therein, and a mammary gland density is calculated by obtaining a ratio of the mammary gland region included in the breast region. The breast region is a region from the body surface to the chest wall, and can be acquired by image processing as a region that includes the breast outer periphery of the left and right breasts in the body side direction. As a classification method of the fat region and the mammary gland region in the breast region, for example, the region can be classified by a method of binarizing the region with a certain threshold value based on the luminance value of the image. In this way, the ratio of the mammary gland region in the breast region can be obtained and used as the mammary gland density. Further, an average value of luminance values in the breast region may be obtained and obtained as a value representing the mammary gland density. Moreover, the structure which calculates | requires the local mammary gland density of the vicinity between a nipple and an attention site | part may be considered by considering the attention site | part and the position of a nipple (reference point position). For example, a cylinder having a predetermined radius centered on a line segment connecting the site of interest and the nipple may be defined, and the mammary gland density within the range of this cylinder may be obtained. In addition, the breast area is divided into blocks of about several tens of voxels, the mammary gland density in each block is obtained, and the mammary gland of each block weighted according to the distance from the line segment connecting the target region and the nipple to each block The weighted average value of the density may be the mammary gland density.

  If statistical information is held in the form of a function having the input parameter x as the subject's age, bust size, distance between nipple attention sites, body surface attention region distance, mammary gland density, etc. Is input to this function to calculate the magnification α and the standard deviation σ.

(S6030: Image composition)
In step S6030, the CPU 1001 functions as the image generation unit 540, and indicates a place where a corresponding portion is likely to exist based on the ultrasonic tomographic image, the position and orientation of the ultrasonic probe, the reference point position, and the corresponding distance. An image in which information is superimposed on an ultrasonic tomographic image is generated. Then, the CPU 1001 sends the generated image to the display unit 180.

  First, the image generation unit 540 (CPU 1001) sets each of the corresponding distances obtained in step S6025 (r′−2σ, r′−σ, r ′, r ′ + σ) around the position of the reference point obtained in step S6020. , R ′ + 2σ) as a radius. Here, five spheres having the same center position and different radii are obtained.

  Next, based on the position and orientation of the ultrasonic probe obtained in step S6010, a plane in the space including the ultrasonic tomographic image (that is, an ultrasonic cross section (imaging cross section)) is obtained, and the sphere ( A circle (5 circles) formed by cutting out 5 balls) is calculated. Finally, a portion (arc) included in the ultrasonic tomographic image in the calculated circle is superimposed on the ultrasonic tomographic image to generate a composite image.

  FIG. 9 shows an example of a composite image generated by this processing. In FIG. 9, cross sections of concentric spheres each having a radius corresponding to the center of the nipple position 410 are superimposed on the ultrasonic tomographic image 440 as an arc 930 and an auxiliary line 935. In addition, character information (“−2σ”, “−σ”, “σ”, “2σ”, etc.) indicating standard deviation is superimposed and displayed near each auxiliary line as additional information. Considering the situation of FIG. 12A described above, even if the distance between the nipple attention sites is the same r, as shown in FIG. Then, information based on the statistical information of the B area is displayed in a superimposed manner.

  FIG. 10 is another example of the composite image in the present embodiment, and the region within the standard deviation σ and the region within 2σ are colored translucently and presented as the existence range 1035. By searching the corresponding part 450 with reference to the support information presented in this way, the operator can narrow down the search range.

  As described above, using the statistical information related to the distance between the nipple attention regions, the position where the corresponding region is likely to exist can be drawn in the ultrasonic tomographic image and presented to the operator. Further, by using statistical information according to the attributes of the subject and the region of interest, a more likely position can be presented. By presenting this, it is possible to show the operator a more effective search range, to prevent the operator from making an incorrect association, and to further reduce the trouble of searching for a wide range.

[Third Embodiment]
The third embodiment is an embodiment in which presentation is performed in consideration of the spatial extent (size) of the region of interest. Hereinafter, only the difference from the first embodiment of the information system according to the present embodiment will be described. That is, it is the same as that of the first embodiment unless otherwise mentioned below.

  A functional configuration example of the information processing system according to the present embodiment will be described with reference to the block diagram of FIG. In FIG. 13, the same functional units as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. As illustrated in FIG. 13, the information processing system according to the present embodiment includes a data server 1360, an information processing device 1300, a medical image collection device 170, and a display unit 180.

  The data server 1360 holds information related to the spatial extent of the target region in addition to the information such as the distance between the reference point target regions described in the first embodiment. Here, the information regarding the spatial spread of the site of interest is, for example, the maximum diameter R of the site of interest. Alternatively, the distance from the reference point to the closest point of the target region (minimum distance Dmin) and the distance to the farthest point (maximum distance Dmax). The former information can be obtained by detecting a region representing the target region by image processing from the three-dimensional image data of the subject from which the distance between the reference point target regions is acquired. Further, the latter information can be calculated based on the nipple position in the three-dimensional image data and information obtained by labeling the voxel of the site of interest. The data server 1360 outputs the stored data to the information processing apparatus 1300.

  The display range calculation unit 1310 acquires information regarding the spatial extent of the site of interest from the data server 1360. Based on this information, the display range for the reference distance is calculated. Then, the calculated display range value is output to the image generation unit 1340.

  Similar to the image generation unit 140 of the first embodiment, the image generation unit 1340 uses the reference distance, the position of the reference point, and the position and orientation of the ultrasonic probe as the reference distance. The information on the arc as the distance is superimposed on the ultrasonic tomographic image. Further, by further using the value of the display range calculated by the display range calculation unit 1310, information on a region having a width from the reference distance is superimposed on the ultrasonic tomographic image.

  When each functional unit in the information processing apparatus 1300 shown in FIG. 13 is implemented by a computer program, the computer shown in FIG. 2 is applied to the information processing apparatus 1300 as in the first embodiment. Can do. That is, when each functional unit in the information processing apparatus 1300 is implemented by a computer program, the computer program is executed by the CPU 1001, and thus the computer functions as the information processing apparatus 1300.

  Next, processing performed by the computer of FIG. 2 that operates as the information processing apparatus 1300 will be described with reference to FIG. 14 showing a flowchart of the processing. Note that a computer program and data for causing the CPU 1001 to execute processing according to the flowchart of FIG. 14 are stored in the external storage device 1007. The CPU 1001 loads the computer program and data from the external storage device 1007 to the RAM 1002, and executes processing using the computer program and data. As a result, the computer of FIG. 2 executes processing according to the flowchart of FIG.

  In FIG. 14, steps S14000, S14010, S14020, and S14040 are the same processing steps as steps S3000, S3010, S3020, and S3040 in the flowchart shown in FIG. 3, and thus description of these steps is omitted.

(S14005: Acquisition of spatial extent of the region of interest)
In step S <b> 14005, the CPU 1001 functions as the display range calculation unit 1310 and acquires information regarding the spatial spread of the region of interest from the data server 1360. For example, the maximum diameter R of the site of interest, the minimum distance Dmin from the reference point to the site of interest, and the maximum distance Dmax are acquired.

(S14007: Display range calculation)
In step S14007, the CPU 1001 functions as the display range calculation unit 1310, and calculates the display range based on the information regarding the spatial spread of the site of interest acquired in step S14005. Specifically, the lower limit value and the upper limit value of the distance from the reference point position are determined as the display range. For example, when the maximum diameter R of the target region is acquired, the display range is from the lower limit value r−R / 2 to the upper limit value r + R / 2 based on the reference distance r. When the minimum distance Dmin and the maximum distance Dmax between the nipple attention sites are acquired, the display range is from the lower limit value Dmin to the upper limit value Dmax.

(S14030: Image composition)
In step S14030, the CPU 1001 functions as the image generation unit 1340, and in consideration of the spatial extent of the corresponding part, an image in which information indicating a place where the corresponding part is likely to exist is superimposed on the ultrasonic tomographic image is displayed. Generate. Specifically, a composite image is generated in which the display range acquired in step S14007, that is, an arc corresponding to each of the upper limit value and the lower limit value of the distance is superimposed on the ultrasonic tomographic image. Alternatively, a synthetic image is generated by superimposing the region surrounded by the arc on the ultrasonic tomographic image by translucent coloring. Note that the process for obtaining the arc on the ultrasonic tomographic image at a predetermined distance from the reference point is the same as the process for the reference distance in step S3030 of the first embodiment, and thus the description thereof is omitted.

  FIG. 15 is an example of a composite image in the present embodiment. In this example, the arc 430 of the reference distance is drawn, and at the same time, the range surrounded by the spheres with the radii r−R / 2 and r + R / 2 from the reference point is colored translucently and presented as the existence range 1510. .

  As described above, presentation is performed in consideration of the spatial spread of the region of interest. By this presentation, the operator can perform a search considering the spatial extent of the target part.

[Fourth Embodiment]
In the above description, the information described as being displayed on the screen at a time is not limited to being displayed at a time, but some of them may be displayed and switched according to user instructions. May be displayed.

  In the above description, the position where the corresponding part is likely to exist is drawn in the ultrasonic tomographic image in various forms and presented to the operator, but it is not a lesion occurring on the mammary gland, etc. Depending on the type of lesion, this presentation may be incorrect. Therefore, whether or not to perform the above synthesis processing may be switched according to the type of lesion.

  For example, when the operator identifies the type of lesion and determines that the lesion does not occur on the mammary gland, it may be designated using the keyboard 1004 or the mouse 1005 so as not to perform the above synthesis process.

  In addition, the ultrasonic tomographic image and the corresponding cross-sectional image in the three-dimensional image by MRI are displayed side by side on the same screen. There is a so-called “Fusion display”. In such Fusion display, it is preferable to display images at the same cross-sectional position side by side, but the respective cross-sectional positions are shifted during imaging of an ultrasonic tomographic image due to the influence of the patient's body movement or the like.

  Therefore, a “positioning mode” and a “shooting mode” are provided. For example, a button image for instructing to select “positioning mode” and a button image for instructing to select “shooting mode” are displayed on the screen of the display unit 180. When the operator operates the keyboard 1004 or the mouse 1005 to specify a button image of “alignment mode”, the CPU 1001 sets the computer mode to “alignment mode”. On the other hand, when the operator operates the keyboard 1004 or the mouse 1005 to specify a button image of “shooting mode”, the CPU 1001 sets the computer mode to “shooting mode”. The mode setting method is not limited to this. When the “positioning mode” is set, the CPU 1001 permits the above composition processing, and when the “photographing mode” is set, the CPU 1001 prohibits the above composition processing. Accordingly, it is possible to present reference information when the operator adjusts the position of the ultrasonic probe so that images at the same cross-sectional position are displayed side by side.

  Further, the above-described combining process may be permitted when the images of the prone position and the supine position are fusion-displayed, and the above-described combining process may be prohibited when the respective images of the supine position and the supine position are fusion-displayed. .

  As described above, it is possible to permit or prohibit the synthesis process according to the purpose and situation of using the information processing system. Further, instead of prohibiting or permitting all the composition processing, for example, only one of arcs and character information (for example, 5 mm, 10 mm, 15 mm, etc.) corresponding to the arcs may be permitted / prohibited. . Further, instead of permitting / prohibiting the synthesis process, it may be controlled how many sets of character information corresponding to the arc and the arc are displayed. For example, in the “positioning mode”, the number of display sets is increased, and in the “shooting mode”, the number of display sets is decreased. Moreover, you may use combining said description suitably.

  As described above, the outline of the above description is that the tomographic image of the subject is acquired, the location in the tomographic image corresponding to the portion separated from the reference point in the subject by the specified distance is specified, and the tomographic image is specified. This is based on a technique for generating and outputting a composite image obtained by combining information indicating locations.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  DESCRIPTION OF SYMBOLS 100: Information processing apparatus 110: Tomographic image acquisition part 120: Position and orientation acquisition part 130: Reference point position acquisition part 140: Image generation part 150: Display control part 105: Reference distance acquisition part

Claims (23)

Acquisition means for acquiring a two-dimensional image of the subject;
Reference point position acquisition means for acquiring information representing the three-dimensional position of the reference point of the subject in space;
Identifying means for identifying a location in the two-dimensional image corresponding to a portion separated by a predetermined three-dimensional distance from the three-dimensional position of the reference point;
Generating means for generating a synthesized image obtained by synthesizing information indicating the location on the two-dimensional image ,
The specifying means specifies a circle obtained by cutting a sphere centered on the three-dimensional position of the reference point and having a radius of the predetermined three-dimensional distance in a cross section corresponding to the two-dimensional image, Identify the arc included in the two-dimensional image as the location,
The information processing apparatus , wherein the generation unit generates the composite image in which the arc is superimposed on the two-dimensional image . A position and orientation acquisition means for acquiring information representing the position and orientation of the two-dimensional image;
The specifying means is separated from the three-dimensional position of the reference point by the predetermined three-dimensional distance based on information indicating the three-dimensional position of the reference point and information indicating the position and orientation of the two-dimensional image. The information processing apparatus according to claim 1, wherein a location in the two-dimensional image corresponding to a part is specified. The information processing apparatus according to claim 1, further comprising display control means for displaying the composite image on a display means. It said generating means, the information processing apparatus according to any one of claims 1 to 3, characterized in that to generate the composite image by superimposing a character indicating the predetermined three-dimensional distance to the two-dimensional image. The predetermined three-dimensional distance includes a plurality of different three-dimensional distances,
In the two-dimensional image corresponding to a portion separated from the three-dimensional position of the reference point by the three-dimensional distance with respect to each three-dimensional distance included in the predetermined three-dimensional distance. Identify the location of
Said generating means according to any one of claims 1 to 4, characterized in that to generate the composite image of information indicating the specified position on the three-dimensional distance of the respective said two-dimensional image Information processing device. The information processing apparatus according to claim 5 , wherein the plurality of three-dimensional distances are equally spaced. The information processing apparatus according to any one of claims 1 to 6 , further comprising means for changing the predetermined three-dimensional distance in accordance with a change in the posture of the subject. The information processing apparatus according to any one of claims 1 to 6 , further comprising means for changing the predetermined three-dimensional distance based on characteristic information of the subject. The information processing apparatus according to claim 8 , wherein the subject is a breast, and the characteristic information is a mammary gland density. The two-dimensional image processing apparatus according to any one of claims 1 to 9, characterized in that the ultrasonic tomographic image generated using an ultrasound system. The information processing apparatus according to any one of claims 1 to 10 , wherein the subject is a breast, and the reference point is a nipple in the breast. Acquire a two-dimensional image of the subject,
Obtaining information representing the three-dimensional position of the reference point of the subject in space;
A sphere centered on the three-dimensional position of the reference point and having a radius of a predetermined three-dimensional distance is identified as a circle cut by a cross section corresponding to the two-dimensional image;
Identify the arc included in the two-dimensional image of the circle ,
A composite image in which the arc is superimposed on the two-dimensional image is generated. Obtaining information representing the position and orientation of the two-dimensional image,
Based on the information representing the three-dimensional position of the reference point and the information representing the position and orientation of the two-dimensional image, the part corresponding to a portion separated from the three-dimensional position of the reference point by the predetermined three-dimensional distance. The information processing method according to claim 12 , wherein a location in the two-dimensional image is specified. The information processing method according to claim 12 or 13, characterized in that for displaying the composite image on the display unit. The information processing method according to any one of claims 12 to 14 , wherein the composite image in which characters indicating the predetermined three-dimensional distance are superimposed on the two-dimensional image is generated. The predetermined three-dimensional distance includes a plurality of different three-dimensional distances,
For each three-dimensional distance included in the predetermined three-dimensional distance, a location in the two-dimensional image corresponding to a portion separated by the three-dimensional distance from the three-dimensional position of the reference point is specified. ,
The information processing method according to any one of claims 12 to 15 , wherein the combined image is generated by combining the two-dimensional image with information indicating a location specified for each three-dimensional distance. The information processing method according to claim 16 , wherein the plurality of three-dimensional distances are equally spaced. The information processing method according to any one of claims 12 to 17 , wherein the predetermined three-dimensional distance is changed in accordance with a change in the posture of the subject. The information processing method according to any one of claims 12 to 17 , wherein the predetermined three-dimensional distance is changed based on characteristic information of the subject. The information processing method according to claim 19 , wherein the subject is a breast, and the characteristic information is a breast density. The information processing method according to any one of claims 12 to 20 , wherein the two-dimensional image is an ultrasonic tomographic image generated using an ultrasonic diagnostic imaging apparatus. The information processing method according to any one of claims 12 to 21 , wherein the subject is a breast, and the reference point is a nipple in the breast. The computer program for functioning a computer as each means of the information processing apparatus of any one of Claims 1 thru | or 11 .

Images