JP2014042830A - Image processor, image processing method, image processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a high quality image of a target cross section by positioning tomographic image groups.SOLUTION: An object tomographic image group acquisition part 110 acquires a tomographic image group composed of a plurality of tomographic images, and a target cross section setting part 130 sets a target cross section. A positioning part 140 calculates a positioning parameter of respective tomographic images to perform positioning on the basis of a positional relation between the tomographic image group and the cross section. the tomographic image generation part 150 positions the tomographic image groups by using the positioning parameter of the respective tomographic images to generate an image of the target cross section.

Description

  The present invention relates to a medical image acquisition apparatus (modality) such as an ultrasonic diagnostic imaging apparatus, an optical coherence tomography (OCT apparatus), a magnetic resonance imaging apparatus (MRI apparatus), and an X-ray computed tomography apparatus (X-ray CT apparatus). The present invention relates to an apparatus and a method for processing an image picked up in (1).

  In the medical field, a doctor displays a tomographic image of a subject on a monitor, interprets the displayed medical image, and diagnoses a lesion. Examples of a medical image collection apparatus (hereinafter referred to as a modality) that captures a tomographic image include the following apparatuses. That is, an ultrasonic diagnostic imaging apparatus (hereinafter referred to as an ultrasonic apparatus), an optical coherence tomometer (hereinafter referred to as an OCT apparatus), a magnetic resonance imaging apparatus (hereinafter referred to as an MRI apparatus), an X-ray computed tomography apparatus ( (Hereinafter referred to as an X-ray CT apparatus).

  There are cases where it is difficult to correctly diagnose the state of a lesioned part only by observing a tomographic image taken with these modalities. Therefore, an attempt has been made to correctly diagnose the state of the lesion by generating an image of an arbitrary cross section different from that at the time of imaging from the tomographic image group and observing the lesion at a different angle.

  For example, in an ultrasound device, a doctor or engineer usually performs imaging while holding an ultrasound probe (hereinafter referred to as a probe) freely in his / her hand, so the tomographic image taken is in a space based on the human body. It is not clear which position of the image was taken. Therefore, an attempt has been made to obtain a spatial positional relationship between tomographic images based on image features in the tomographic images (for example, Non-Patent Document 1). If a spatial positional relationship between tomographic images is obtained, an image of an arbitrary cross section can be generated using a known method. Hereinafter, an image obtained by cutting out a tomographic image group with an arbitrary cross section is referred to as a cross-sectional image.

  The above-described method for obtaining an arbitrary cross-sectional image by obtaining a spatial positional relationship between tomographic images is based on the premise that the subject is a rigid body. However, the actual subject is deformed by pressing the probe in the process of capturing the tomographic image group. Therefore, the image quality of the cross-sectional image generated by the above method may be deteriorated. Therefore, attempts have been made to correct the influence of the deformation of the subject. For example, in Non-Patent Document 2, the deformation amount of the subject due to the pressing of the probe is estimated based on the association of image features between adjacent tomographic images, and each tomographic image is stretched based on the estimated deformation amount. Discloses a method for correcting the influence of deformation of the subject.

A. Gee, R.A. Prager, G .; Trees, and L.L. Berman, “Engineering a freehand 3D ultrasound system,” Pattern Recognition Letters, vol. 24, pp. 757-777, 2003. G. Trees, R.M. Prager, and L.L. Berman, “Collection of probe pressure artifacts in freehand 3D ultrasound,” Medical Image Analysis, vol. 6, pp. 199-214, 2002. W. Hoff and T.W. Vincent, “Analysis of Head Pose Accuracy in Augmented Reality,” IEEE Transactions on Visualization and Computer Graphics, vo1.6, no. 4, pp. 319-334, 2000. C. Wachinger, W.M. Wein, and N.M. Navab, “Three-dimensional ultra sound mosaicing,” Proc. MICCAI 2007, pp. 327-335, 2007.

  However, in the method of Non-Patent Document 2, the result of associating image features between adjacent tomographic images is used regardless of the position of the pixel in the tomographic image. For this reason, when the assumption that the deformation of the subject is uniform regardless of the location does not hold, the positional relationship between the tomographic image groups is shifted uniformly, and the image quality of the generated image is lowered. Has occurred.

  The present invention has been made to solve such a problem, and the position of the corresponding partial region is more suitable as the distance between the corresponding partial region and the predetermined surface of each of the tomographic images of the subject is smaller. A positioning unit that performs positioning of the tomographic image, and a generating unit that generates a cross-sectional image from the plurality of aligned tomographic images based on positional information of the predetermined surface. And

  According to the present invention having such a configuration, alignment in the vicinity of a predetermined surface is performed with priority, so that a cross-sectional image can be generated with high image quality.

It is a figure explaining the apparatus structure of the image processing apparatus which concerns on a 1st Example. It is a figure which shows the outline | summary of the alignment process which an image processing apparatus performs. It is a flowchart which shows the whole process sequence in a 1st Example. It is a flowchart which shows the calculation procedure of the alignment parameter of a tomographic image group in a 1st Example. It is a figure explaining the apparatus structure of the image processing apparatus which concerns on a 2nd Example. It is a flowchart which shows the whole process sequence in a 2nd Example. It is a flowchart which shows the calculation procedure of the alignment parameter of a tomographic image group in a 2nd Example. It is a flowchart which shows the calculation procedure of the alignment parameter of a tomographic image group in a 3rd Example.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the scope of the invention is not limited to the description or illustrated examples.

  The image processing apparatus according to the present embodiment (using an MRI tomographic image as a cross section of interest) aligns a plurality of tomographic images obtained in advance by an ultrasonic diagnostic apparatus, and based on the result, the operator A two-dimensional image (cross-sectional image) of the cross section of interest designated by the (doctor or engineer) is generated. At this time, the alignment parameter is calculated in consideration of the positional relationship between the tomographic image group and the target cross section so that the cross-sectional image is generated with high image quality. In this embodiment, the position and orientation of each tomographic image and the amount of compressive deformation of the body tissue due to the pressing of the probe are used as alignment parameters. In this embodiment, when the probe is pressed and deformed, the subject is compressed and deformed only in the longitudinal direction of each tomographic image (direction from the body surface to the body) by an amount proportional to the depth from the body surface. Assume. The image processing apparatus according to this embodiment will be described below.

  FIG. 1A shows a configuration of an image processing system in the present embodiment. As shown in the figure, the image processing apparatus 100 according to the present embodiment includes a target tomographic image group acquisition unit 110, a reference tomographic image group acquisition unit 120, an attention slice setting unit 130, an alignment parameter calculation unit 140, and a slice image generation unit. 150. Then, a target tomographic image group obtained by imaging the subject in advance by the first medical image collection device 170 and a reference tomographic image group obtained by imaging the subject in advance by the second medical image collection device 180 are obtained. It is connected to the data server 190 that holds it.

  The target tomographic image group held by the data server 190 is a plurality of two-dimensional tomographic images (ultrasound tomographic images) obtained by imaging a subject in advance by an ultrasonic device as the first medical image acquisition device 170. . It is assumed that the position and orientation of the probe are measured by the position and orientation sensor when the subject is imaged by the first medical image acquisition apparatus 170. Further, the position and orientation of the probe is converted into the position and orientation of each ultrasonic tomographic image in the reference coordinate system (the coordinate system in the space with the subject as a reference), and then associated with the ultrasonic tomographic image and the data server. 190 is held. The position / orientation sensor is configured by, for example, FASTRAK of Polhemus, USA. The position and orientation sensor may be configured in any way as long as the position and orientation of the probe can be measured. The target tomographic image group expressed in the reference coordinate system is input to the image processing apparatus 100 via the target tomographic image group acquisition unit 110.

  On the other hand, the reference tomographic image group held by the data server 190 is a plurality of MRI tomographic images obtained by imaging the subject in advance by the MRI apparatus as the second medical image acquisition apparatus 180. It is assumed that the position and orientation of each MRI tomographic image constituting the reference tomographic image group is held in the data server 190 after being converted into the standard coordinate system. The reference tomographic image group expressed in the reference coordinate system is input to the image processing apparatus 100 via the reference tomographic image group acquisition unit 120.

  The target tomographic image group acquisition unit 110 acquires the target tomographic image group input to the image processing apparatus 100 and outputs it to the alignment parameter calculation unit 140. Here, it is assumed that information of the position and orientation of the tomographic image held by the data server 190 is attached to each target tomographic image.

  The reference tomographic image group acquisition unit 120 acquires a reference tomographic image group input to the image processing apparatus 100 and outputs the reference tomographic image group to the attention slice setting unit 130. Here, it is assumed that information of the position and orientation of the tomographic image held by the data server 190 is attached to each reference tomographic image.

  The attention slice setting unit 130 acquires a reference tomographic image group that is an output of the reference tomographic image group acquiring unit 120, and selects one MRI tomographic image from the reference tomographic image group (acquires selection by the operator). ). Then, the cross section representing the selected MRI tomographic image is set as the cross section of interest, and the position and orientation thereof are output to the alignment parameter calculation section 140 and the cross section image generation section 150. Hereinafter, the selected MRI tomographic image is referred to as a selected tomographic image.

  The alignment parameter calculation unit 140 calculates the alignment parameter of the target tomographic image group in consideration of the positional relationship between the target tomographic image group and the target slice. For this purpose, a target tomographic image group that is an output of the target tomographic image group acquiring unit 110 is acquired. Further, the position and orientation of the cross section of interest, which is the output of the cross section of interest setting unit 130, is acquired. Based on these, alignment parameters are calculated (corrected). That is, the position and orientation of each tomographic image obtained from the target tomographic image group acquisition unit 110 is corrected, and the absolute value of the amount of compressive deformation of the body tissue due to the pressing of the probe (hereinafter referred to as the absolute compressive deformation amount). Calculation is performed for each tomographic image. Then, the calculated alignment parameter is output to the cross-sectional image generation unit 150. Note that the alignment described here does not only indicate that the alignment result is displayed, but the alignment parameter is calculated and data for displaying the aligned tomographic image group is generated. To complete the alignment.

  The cross-sectional image generation unit 150 generates a cross-sectional image from the target tomographic image group based on the alignment parameter. For this purpose, the position and orientation of the target section, which is the output of the target section setting unit 130, is acquired. Also, the alignment parameter of the target tomographic image group that is the output of the alignment parameter calculation unit 140 is acquired. Based on these, an image of a cross section of interest (cross-sectional image) is generated and output. A method of generating a predetermined cross-sectional image from a group of tomographic images that have been aligned can be realized by a known technique.

  Note that at least one of the units (target tomographic image group acquisition unit 110, reference tomographic image group acquisition unit 120, attention slice setting unit 130, alignment parameter calculation unit 140, and sectional image generation unit 150) illustrated in FIG. The unit may be realized as an independent device. Alternatively, it may be implemented as software that implements its function by being installed in one or a plurality of computers and executed by the CPU of the computer. In this embodiment, each unit is realized by software and installed in the same computer.

  FIG. 1B is a diagram illustrating a basic configuration of a computer for realizing the functions of the respective units illustrated in FIG. 1A by executing software. The CPU 1001 controls the entire computer using programs and data stored in the RAM 1002 and the ROM 1003. Further, the execution of software in each part is controlled to realize the function of each part. The RAM 1002 includes an area for temporarily storing programs and data loaded from the external storage device 1007 and the storage medium drive 1008, and a work area required for the CPU 1001 to perform various processes. The ROM 1003 generally stores computer programs and setting data.

  A keyboard 1004 and a mouse 1005 are input devices, and an operator can input various instructions to the CPU 1001 using these devices. The display unit 1006 includes a CRT, a liquid crystal display, and the like, and displays the generated cross-sectional image. In addition, a reference tomographic image and other images are displayed as necessary. In addition, a message to be displayed, a GUI, and the like can be displayed.

  The external storage device 1007 is a device that functions as a large-capacity information storage device such as a hard disk drive, and stores an OS (operating system), a program executed by the CPU 1001, and the like. In the description of this embodiment, information that is described as being known is stored here, and loaded into the RAM 1002 as necessary. The storage medium drive 1008 reads a program or data stored in a storage medium such as a CD-ROM or DVD-ROM in accordance with an instruction from the CPU 1001 and outputs it to the RAM 1002 or the external storage device 1007. The I / F 1009 includes an analog video port or a digital input / output port such as IEEE 1394, an Ethernet (registered trademark) port for outputting various types of information to the outside, and the like. The data input by each is taken into the RAM 1002 via the I / F 1009. Some of the functions of the target tomographic image group acquisition unit 110 and the reference tomographic image group acquisition unit 120 are realized by the I / F 1009.

  The above-described components are connected to each other by a bus 1010.

  An overview of processing executed as a whole by the image processing system having the above-described configuration will be described with reference to FIG. The target cross-section setting unit 130 sets a target cross section for the ultrasonic tomographic image group acquired from the first medical image acquisition apparatus 170 by the target tomographic image group acquiring unit 110 of the image processing apparatus 100 (FIG. 2A). . For example, the cross section of interest is set from an MRI tomographic image group that has been aligned with the ultrasonic tomographic image group. The cross-sectional image generation unit 150 generates a cross-sectional image based on the target cross-section from the ultrasonic tomographic image group.

  When generating a cross-sectional image based on the cross section of interest, the image processing apparatus 100 performs alignment by optimizing the ultrasonic tomographic image group with respect to the cross section of interest. For this purpose, the alignment parameter calculation unit 140 specifies a plurality of corresponding partial regions for two adjacent ultrasonic tomographic images (FIG. 2B). The corresponding partial area is specified using image information. Then, among the identified partial areas, the alignment is performed by calculating the alignment parameters of the two ultrasonic tomographic images so that the correspondence between the positions becomes more accurate as the distance from the target cross section is smaller. Then, by aligning the arbitrary adjacent ultrasonic tomographic images, the ultrasonic tomographic image group is aligned. The cross-sectional image generation unit 150 can generate a high-quality cross-sectional image for the cross-section of interest using the aligned ultrasonic tomographic image group.

  FIG. 3 is a flowchart showing an overall processing procedure performed by the image processing apparatus 100. In the present embodiment, the flowchart is realized by the CPU 1001 executing a program that realizes the function of each unit. It is assumed that the program code according to the flowchart is already loaded from, for example, the external storage device 1007 to the RAM 1002 in the previous stage of performing the following processing.

(S3000) (Data acquisition)
In step S3000, the image processing apparatus 100 acquires a target tomographic image group as a process of the target tomographic image group acquisition unit 110. Further, a reference tomographic image group is acquired as a process of the reference tomographic image group acquiring unit 120.

(S3010) (Select section of interest)
In step S3010, as the processing of the attention slice setting unit 130, the image processing apparatus 100 first selects a tomographic image from the reference tomographic image group according to an instruction from the operator, and sets this as a selected tomographic image. For example, when an instruction of “forward feeding” is acquired from the operator via a UI (not shown), the display unit 1006 sequentially forwards the tomographic images one by one according to the order of the imaging times of the tomographic images constituting the reference tomographic image group. Display above. On the other hand, when the “reverse feed” instruction is acquired, the tomographic images are displayed one by one in the reverse order of the imaging times of the tomographic images constituting the reference tomographic image group. When a “selection” instruction is acquired from the operator via a UI (not shown), the currently displayed tomographic image is selected as the selected tomographic image. Then, the position and orientation of the selected tomographic image in the reference coordinate system are set as the position and orientation of the cross section of interest.

(S3020) (Calculation of alignment parameter)
In step S3020, the image processing apparatus 100 calculates (corrects) the alignment parameter of the target tomographic image group based on the position and orientation of the target cross section set in step S3010 as the processing of the alignment parameter calculation unit 140. Details of the processing of the alignment parameter calculation unit 140 in this step will be described later in detail using the flowchart shown in FIG.

(S3030) (Generation of cross-sectional image)
In step S3030, the image processing apparatus 100 generates the cross-sectional image of the target cross-section set in step S3010 as the process of the cross-sectional image generation unit 150. Specifically, first, each target tomographic image is stretched based on the absolute compression deformation amount of the subject that is a part of the alignment parameter obtained in step S3020. Then, an image of the cross section of interest is generated using the enlarged target tomographic image group and the position and orientation obtained in step S3020. Note that a method for enlarging a tomographic image when an absolute compressive deformation amount is given and a method for generating an arbitrary cross-sectional image from a group of tomographic images with known positions and orientations are well known, and thus detailed description thereof is omitted.

(S3040) (Output of cross-sectional image)
In step S3040, the image processing apparatus 100 displays the cross-sectional image generated in step S3030 on the display unit 1006. Further, this is output to the outside via the I / F 1009 as necessary. Alternatively, it is stored in the RAM 1002 as a state that can be used by other applications.

  As described above, the processing of the image processing apparatus 100 is performed.

  FIG. 4 is a flowchart showing a processing procedure of the alignment parameter calculation unit 140 in step S3020 described above.

(S4000) (Selection of unprocessed tomographic image)
In step S4000, the alignment parameter calculation unit 140 selects, from the tomographic images constituting the target tomographic image group, one tomographic image that has not been subjected to the processes of steps S4010 to S4037 described below as an unprocessed tomographic image. . In this embodiment, the tomographic images are selected in the order of the imaged times. In this case, normally, adjacent tomographic images are sequentially aligned.

(S4010) (Selection of processing target area)
In step S4010, the alignment parameter calculation unit 140 divides the unprocessed tomographic image into partial regions, and selects a plurality of processing target regions therefrom. In this embodiment, when the partial region of interest is a fully developed speckle region, the region is selected as a processing target region. Here, the speckle is mottled noise generated in an ultrasonic tomographic image due to interference of ultrasonic signals from a living tissue.

  Specifically, the following processing is executed. First, the unprocessed tomographic image is divided into, for example, 10 × 10 non-overlapping partial regions. Next, it is determined by a known method whether each partial area is a sufficiently developed speckle noise area (having a property that pixel values in the partial area follow an exponential distribution). For example, the determination is made based on whether the ratio between the average value of the pixel values in the partial area and the standard deviation is close to 1. Then, a partial area determined to be a sufficiently developed speckle noise area is selected as a processing target area.

(S4020) (Association of processing target areas)
In step S4020, the alignment parameter calculation unit 140 searches for a region corresponding to each partial region selected as the processing target region in step S4010 in the tomographic image captured at the time immediately before the unprocessed tomographic image. In the present embodiment, an area having the highest similarity is searched within a predetermined search range by template matching using a normalized cross correlation criterion. Then, the area determined to have the highest similarity is selected as the corresponding area.

(S4030) (Calculation of relative alignment parameter)
In step S4030, the alignment parameter calculation unit 140 determines between the two tomographic images based on the position information of the plurality of processing target areas selected in step S4010 and the position information of the plurality of corresponding areas selected in step S4020. Calculate alignment parameters. At that time, a calculation with a weight according to the distance from the cross section of interest is performed. Here, the alignment parameters between the two tomographic images are the translational components (tx, ty, tz), the rotational components (roll, pitch, yaw), and the relative body tissue relative to the depth y from the body surface. It is assumed to be composed of the compression deformation amount p (y). Note that p (y) represents a relative displacement amount between two tomographic images at a depth y.

  Specifically, first, assuming that the subject is not compressed and deformed, an in-plane moving component (tx, ty, roll) is calculated by the following equation.

  Here, xdst_n = [xdst_n ydst_n 0 1] T represents the position in the tomographic image coordinate system of the nth (n = 1 to N) processing target region. N is the number of processing target areas selected in step S4010. Moreover, xsrc_n = [xsrc_n ysrc_n 0 1] T represents the position in the tomographic image coordinate system of the corresponding area corresponding to the nth processing target area. Further, the symbol + added to the left matrix on the right side of Equation 1 represents a generalized inverse matrix of the matrix.

  The above equation 1 solves the following expression representing the relationship between the position of the processing target region and the corresponding region and the in-plane moving component that is satisfied when the rotation angle is small, with respect to the in-plane moving component. Can be derived by

  In actual processing, in Formula 1, when the distance from the nth processing target region to the target cross section is short, for example, by arranging a plurality of rows related to the nth region, the distance from the target cross section is determined. Weight. For example, a maximum of three rows may be arranged in inverse proportion to the distance. That is, the first threshold value and the second threshold value are prepared in advance, and three processing target regions whose distance to the target cross section is less than the first threshold value are greater than or equal to the first threshold value and less than the second threshold value. Two rows are allocated to the processing target area, and one row is allocated to the processing target area equal to or greater than the second threshold. In addition, a third threshold value that is larger than the second threshold value may be further used to exclude the processing target region from the processing target when the distance to the target cross section is equal to or greater than the third threshold value.

  The distance from the processing target area to the target cross section is calculated as follows. First, the position of the processing target area in the reference coordinate system is obtained based on the position and orientation of the unprocessed tomographic image and the position of the processing target area in the tomographic image coordinate system. Then, the distance between them may be calculated based on the position of the processing target region in the reference coordinate system and the position and orientation of the target cross section. When the processing of this step is performed first, the relative value of the position and orientation of the two tomographic images obtained from the target tomographic image group acquisition unit 110 is used as the initial value of the position and orientation of the unprocessed tomographic image. .

  Next, an out-of-plane moving component (tz, pitch, yaw) is calculated under the assumption that the subject is not compressed and deformed. Here, when the processing target region is a sufficiently developed speckle noise region, it is known that an average correlation curve indicating a relationship between a three-dimensional distance to a corresponding region and a correlation value is a Gaussian curve. . In this embodiment, it is assumed that the image processing apparatus 100 holds an average correlation curve indicating the relationship between the three-dimensional distance and the correlation value as known information. The alignment parameter calculation unit 140 uses the average correlation curve to calculate a three-dimensional distance dzn between the regions based on the correlation value between each processing target region and its corresponding region. Then, the out-of-plane moving component is calculated by the following equation.

  Equation 3 above is derived by solving the following equation representing the relationship between the three-dimensional distance between regions and the out-of-plane movement component, which is established when it is assumed that the rotation angle is small. it can.

  Also in Equation 3, in the actual processing, when the distance from the n th processing target region to the target cross section is short, for example, by arranging a plurality of rows related to the n th region, the distance from the target cross section is determined. Give weight.

  Furthermore, the position and orientation between the two tomographic images calculated by the above-described processing and the position and orientation between the two tomographic images measured by the position and orientation sensor are integrated. For example, by using the method disclosed in Non-Patent Document 3, the respective positions and orientations are integrated based on the respective error covariance matrices to calculate a likely position and orientation between two tomographic images. Here, each error covariance matrix can be calculated based on an error allowed in the association of processing target areas, an error in correlation calculation, a measurement error of the position and orientation sensor, and the like. Note that the estimated value of the out-of-plane movement component described above generally has low reliability, and therefore, a configuration in which the measured value of the position / orientation sensor is believed as it is may be used.

  Next, the relative compression deformation amount p (y) of the subject due to the probe pressing is estimated by a method in which the method disclosed in Non-Patent Document 2 is improved as described below. Specifically, first, based on the position and orientation between the two tomographic images calculated by the above processing, as in Non-Patent Document 2, the unprocessed tomographic image is parallel to the tomographic image captured at the immediately preceding time. Project to be. Then, the relative compression deformation amount p (y) of the subject at each depth y is estimated based on the association of image features between adjacent tomographic images. In Non-Patent Document 2, the amount of relative compression deformation at each depth is estimated by matching between adjacent tomographic images for each horizontal line of the projected unprocessed tomographic image (a partial area of horizontal width × 1 pixel). It was. On the other hand, in this embodiment, the partial region of width × 1 pixel is further divided into a plurality of processing target regions that do not overlap each other, and matching between the processing target regions is performed. Then, each matching result is averaged with a weight corresponding to the distance between the processing target region and the target cross section, thereby obtaining the relative compression deformation amount p (y) at each depth.

  Through the above processing, the alignment parameter between the two tomographic images (the unprocessed tomographic image and the tomographic image captured at the immediately preceding time) is calculated.

(S4035) (Determining whether or not convergence has occurred)
In step S4035, the alignment parameter calculation unit 140 determines whether the alignment parameter calculation process in step S4030 has converged. Specifically, the difference between the alignment parameter calculated in the previous process and the alignment parameter calculated in the current process is calculated, and if the calculated difference value is smaller than a predetermined threshold value, It is determined that the adjustment parameter calculation processing has converged. If it is determined that it has not converged, the position ydst_n in the depth direction of the processing target region is corrected by the following equation.

Then, based on the corrected position, the process of S4030 is performed again. On the other hand, if it is determined that it has converged, the process proceeds to step S4037.

(S4037) (Calculation of absolute alignment parameter)
In step S4037, the alignment parameter calculation unit 140 adds the translation component and the rotation component obtained by the above processing to the position and orientation in the reference coordinate system of the tomographic image captured at the immediately preceding time, thereby obtaining the reference coordinate system. The position and orientation of the unprocessed tomographic image are calculated. Further, the absolute compression deformation amount of the unprocessed tomographic image is calculated by adding the relative compression deformation amount obtained by the above processing to the absolute compression deformation amount of the tomographic image captured at the immediately preceding time. In this embodiment, it is assumed that the absolute compression deformation amount of the tomographic image captured at the first time is zero.

(S4040) (Determine whether or not processing of all faults has been completed)
In step S4040, the alignment parameter calculation unit 140 determines whether or not all the tomographic images have been processed. If the processing has not been completed, the process returns to step S4000. In this case, alignment of the tomographic image taken next to the tomographic image that has been processed is performed. In this way, alignment is performed in a chain manner on a plurality of tomographic images.

  On the other hand, if the process has been completed, the process proceeds to step S4050.

(S4050) (Output of alignment parameter)
In step S4050, the alignment parameter calculation unit 140 outputs the alignment parameters of each tomographic image calculated in the processes from step S4000 to step S4040 to the cross-sectional image generation unit 150.

  As described above, according to the image processing apparatus of the present embodiment, it is possible to calculate (correct) the alignment parameter of the tomographic image group with a weight according to the distance from the target cross section. Thereby, alignment is performed so as to minimize the error in the vicinity of the cross section of interest, so that a cross-sectional image can be generated with high image quality.

  In addition, since the cross-sectional image corresponding to the selected MRI tomographic image can be generated from the ultrasonic tomographic image group with high image quality, the selected MRI tomographic image and the generated cross-sectional image can be easily compared.

(Modification 1 of the first embodiment) (Weighting method)
In the present embodiment, the alignment of the tomographic image group is performed by assigning a weight according to the distance from the target cross section by arranging a plurality of rows related to the processing target region that are close to the target cross section in Expression 1 and Expression 3. The parameter was calculated (corrected). However, the method of applying a weight according to the distance from the target cross section is not limited to this. For example, a method of multiplying each row (representing the distance between the respective regions) of the right-side matrix in Equations 1 and 3 by a weight according to the distance from the cross section of interest may be used. At this time, the weight wn corresponding to the n-th processing target region may be calculated by the following equation, for example, so that the processing target region closer to the target cross section is given a higher weight.

  Here, dn represents the distance from the cross section of interest to the nth processing target region, and c represents a predetermined constant.

(Modification 2 of the first embodiment) (Only position and orientation / deformation parameters only)
In the present embodiment, the case where the position and orientation of each tomographic image and the amount of compressive deformation of the body tissue due to the pressing of the probe are used as alignment parameters has been described as an example. However, if the position and orientation of each tomographic image can be assumed to be known by measurement with a high-accuracy position and orientation sensor (no correction is required), only the amount of compressive deformation of the body tissue due to the pressing of the probe is used as the alignment parameter. It may be used as Conversely, if the deformation of the subject can be ignored, only the position and orientation of each tomographic image may be used as the alignment parameter.

(Modification 3 of the first embodiment) (Cross-sectional image generation after reconstructing the volume once)
In the present embodiment, in the process of step S3030, a cross-sectional image is generated from the target tomographic image group based on the position and orientation of the target cross-section. However, the method for generating the cross-sectional image is not limited to this.

  For example, ultrasonic three-dimensional volume data may be once reconstructed from the target tomographic image group by a known method based on the calculated alignment parameter, and a cross-sectional image may be generated from the three-dimensional volume data. Hereinafter, the tomographic image group and the three-dimensional volume data are collectively referred to as three-dimensional image data.

  Alternatively, new three-dimensional image data obtained by performing image processing on the ultrasonic three-dimensional image data and adjusting the appearance may be generated, and a cross section of the new three-dimensional image data may be generated as a cross-sectional image. For example, three-dimensional image data subjected to edge enhancement processing, pseudo color processing based on organ segmentation results, or the like may be generated, and the cross-sectional image thereof may be generated. Of course, after the cross-sectional image is generated from the ultrasonic three-dimensional image data, the above-described image processing may be performed on the cross-sectional image.

  Furthermore, if an image is generated from three-dimensional image data based on a set cross section, the generated cross section image may not be an image obtained by imaging pixel values or voxel values on the cross section. For example, after setting a predetermined range in the normal direction around the cross section, the maximum value projection image obtained by calculating the maximum value of the pixel value or voxel value in the normal direction within the range for each point on the cross section It may be an image. Here, the image including the above-described image generated with respect to the set cross section is referred to as a “cross-sectional image” in a broad sense.

(Modification 4 of the first embodiment) (Evaluation of image quality of generated image)
You may further combine the method of a present Example with the method of evaluating and aligning the image quality of a production | generation image. That is, the alignment evaluation value may be defined as a function of the image quality of the generated image and the alignment parameter, and the evaluation result of the image quality of the generated image may be reflected in the alignment. In this case, the calculation (correction) of the alignment parameters of each tomographic image and the evaluation of the image quality of the generated image are repeated a predetermined number of times or until a predetermined image quality is exceeded, thereby generating a cross-sectional image with higher image quality. it can.

  Further, instead of evaluating the image quality of the generated image, the generated image may be evaluated by comparison with the selected tomographic image. For this comparison, for example, a known evaluation criterion for evaluating the consistency and similarity between images, such as a mutual information criterion, may be used.

(Modification 5 of the first embodiment) (Specify a section other than the MRI tomographic image as the target cross section)
In this embodiment, the case where a tomographic image is selected from the MRI tomographic image group and the position and orientation in the reference coordinate system is set as the position and orientation of the target cross section has been described as an example. However, the cross section of interest is not limited to this and may be an arbitrary cross section.

  For example, an image of an arbitrary cross section in which the feature of the lesion is appropriately depicted from the MRI tomographic image group may be generated, and the position and orientation in the reference coordinate system may be set as the position and orientation of the target cross section. In that case, the set cross section is not limited to a flat surface but may be a curved surface. Even if the cross section is a curved surface, all the other processes can be performed in the same manner as described above by calculating the distance between the cross section and the processing target region.

  Further, for example, two tomographic images in which features of a cross-sectional image to be generated appear in the ultrasonic tomographic image group may be designated, and a plane passing through the characteristic part in each tomographic image may be set as the target cross-section. In this case, the tomographic image designated with respect to the cross section of interest is fixed, and the other tomographic images are aligned with the designated tomographic image in order from the closest. One or three or more tomographic images may be designated and fixed.

(Modification 6 of the first embodiment) (Other than speckle noise region)
In the present embodiment, an example has been described in which an ultrasonic tomographic image is divided into partial regions, and a speckle noise region that is sufficiently developed therefrom is selected and used as a processing target region. However, the present invention is not limited to this, and an image feature such as a point feature or a line feature may be extracted from an ultrasonic tomographic image by a well-known method, and its vicinity region may be used as a processing target region. In this case, in the process of step S4030, the measurement value of the position and orientation sensor may be used as it is without estimating the out-of-plane movement component.

(Modification 7 of the first embodiment) (Calculate position and orientation after correcting compression deformation)
In this embodiment, the amount of compressive deformation is calculated after calculating the position and orientation between tomographic images on the assumption that the subject is not compressed and deformed. However, the present invention is not limited to this. First, the amount of compression deformation may be calculated using the measurement values of the position and orientation sensor as they are, and the position and orientation between tomographic images may be calculated after correcting the position information of the processing target region. . In this embodiment, in the process of step S4035, it is determined whether or not the alignment parameter calculation process in step S4030 has converged, but the process of step S4035 may not be performed. In this case, since the process of step S4030 is performed only once, it is possible to realize a process that emphasizes speed.

(Modification 8 of the first embodiment) (3D probe)
In the present embodiment, the weighting according to the distance from the target cross section is applied, and the alignment parameter of the two-dimensional tomographic image group obtained by the 2D probe is calculated. However, the present invention is not limited to this, and the alignment parameter of the volume data group (three-dimensional tomographic image group) obtained by the 3D probe may be calculated by assigning a weight according to the distance from the target section. For example, in the method disclosed in Non-Patent Document 4, the volume data group is aligned by minimizing the sum of the similarities of the overlapping micro regions. The process of calculating the sum of the similarities is performed as follows: You may attach and weight according to the distance from an attention cross section.

(Modification 9 of the first embodiment) (Any area is used as a reference for alignment)
In the present embodiment, the optimized alignment is performed for the designated section of interest, but the application example of the present invention is not limited to this. For example, an optimized alignment may be performed on a predetermined three-dimensional region of the subject. In that case, for example, the tomographic image group may be aligned so that the position of the corresponding partial region is matched as the distance from the center of gravity of the three-dimensional region is small.

(Modification 10 of the first embodiment) (Selection of unprocessed tomographic image)
In the present embodiment, the alignment parameter calculation unit 140 selects tomographic images to be aligned in order of photographing time. However, the present invention is not limited to this, and adjacent tomographic images may be aligned in order. Alternatively, a reference tomographic image may be selected first, and adjacent tomographic images may be chain-aligned from the reference tomographic image. In this case, since the positions of a plurality of tomographic images are determined according to the reference tomographic image, a higher-quality cross-sectional image can be generated. In addition, when selecting a tomographic image as a reference, instead of sequentially aligning adjacent tomographic images in sequence, the correspondence between the reference tomographic image and an unprocessed tomographic image is always specified and positioned. Matching may be performed. In this case, a high-quality cross-sectional image can be generated in accordance with the selected reference image.

  In the first embodiment (with a real-time ultrasonic tomographic image as the cross section of interest), a selected tomographic image is selected from a group of MRI tomographic images obtained by imaging the subject in advance by the MRI apparatus, and the selected tomographic image is selected. A case has been described in which the cross section representing is set as the cross section of interest. However, the setting method of the attention section is not limited to this, and a section representing a tomographic image obtained by imaging the subject in real time by the ultrasonic apparatus may be set as the attention section. The image processing apparatus according to the present embodiment aligns tomographic image groups obtained by imaging in advance by the ultrasonic diagnostic apparatus, and based on the result, the same cross section as the tomographic image obtained by imaging in real time Are sequentially generated from the tomographic image group. At this time, the alignment parameter is calculated in consideration of the positional relationship between the tomographic image group and the target cross section so that the cross-sectional image is generated with high image quality. Hereinafter, only the difference from the first embodiment will be described for the image processing apparatus according to this embodiment.

  FIG. 5 shows the configuration of the image processing apparatus according to the present embodiment. The same parts as those in FIG. 1 are given the same numbers and symbols, and the description thereof is omitted. As illustrated in FIG. 5, the image processing apparatus 500 according to the present exemplary embodiment includes a target tomographic image group acquisition unit 110, a reference tomographic image acquisition unit 520, an attention cross-section setting unit 530, an alignment parameter calculation unit 540, and a cross-sectional image generation unit 550. The image composition unit 560 is configured. The image processing apparatus 500 is connected to a first medical image acquisition apparatus 170 that captures a tomographic image of a subject in real time, and sequentially inputs the tomographic image and its position and orientation. Furthermore, the image processing apparatus 500 is connected to a data server 590 that holds a target tomographic image group obtained by imaging a subject in advance by the first medical image collection apparatus 170.

  As in the first embodiment, the data server 590 uses a plurality of ultrasonic tomographic images obtained by imaging an object in advance by the ultrasonic device as the first medical image acquisition device 170 as a target tomographic image group. keeping. The target tomographic image group expressed in the reference coordinate system is input to the image processing apparatus 500 via the target tomographic image group acquisition unit 110 as in the first embodiment. Unlike the first embodiment, the data server 590 does not hold a reference tomographic image group.

  The reference tomographic image acquisition unit 520 acquires tomographic images that are sequentially input from the first medical image acquisition device 170 to the image processing device 500, and uses the acquired tomographic images as reference tomographic images sequentially to the attention cross-section setting unit 530. Output to.

  The attention slice setting unit 530 sequentially acquires the reference tomographic image, which is the output of the reference tomographic image acquisition unit 520, and sets the slice representing the reference tomographic image as the attention slice. Then, the position and orientation are sequentially output to the alignment parameter calculation unit 540 and the cross-sectional image generation unit 550.

  The alignment parameter calculation unit 540 sequentially determines the alignment parameters of the target tomographic image group corresponding to each of the target cross-sectional images that are sequentially updated while considering the positional relationship between the target tomographic image group and the target cross-sectional image. To calculate. For this purpose, a target tomographic image group that is an output of the target tomographic image group acquiring unit 110 is acquired. Further, the position and orientation of the cross section of interest, which is the output of the cross section of interest setting unit 530, is sequentially acquired. Based on these pieces of information, the alignment parameters corresponding to the currently acquired section of interest are sequentially calculated using the information of the alignment parameters calculated in the past (corresponding to the section of attention acquired in the past). To calculate. Then, the obtained alignment parameters are sequentially output to the cross-sectional image generation unit 550.

  The cross-sectional image generation unit 550 generates a cross-sectional image from the target tomographic image group based on the alignment parameter. For this purpose, the position and orientation of the cross section of interest, which is the output of the cross section of interest setting unit 530, is sequentially acquired. In addition, the position and orientation of each ultrasonic tomographic image, which is the output of the alignment parameter calculation unit 540, are sequentially acquired. Based on these, an image of a cross section of interest (cross-sectional image) is generated and sequentially output to the image composition unit 560.

  The image synthesis unit 560 sequentially acquires the reference tomographic image that is the output of the reference tomographic image acquisition unit 520 and the cross-sectional image that is the output of the cross-sectional image generation unit 550, synthesizes both, and sequentially outputs them. .

  Note that the basic configuration of a computer that implements the functions of the above-described units constituting the image processing apparatus 500 by executing software is the same as that in FIG. 2 in the first embodiment.

  FIG. 6 is a flowchart showing an overall processing procedure performed by the image processing apparatus 500. In the present embodiment, the flowchart is realized by the CPU 1001 executing a program that realizes the function of each unit. It is assumed that the program code according to the flowchart is already loaded from, for example, the external storage device 1007 to the RAM 1002 in the previous stage of performing the following processing.

(S6000) (Data acquisition)
In step S6000, the image processing apparatus 500 acquires the target tomographic image group as the processing of the target tomographic image group acquisition unit 110.

(S6003) (Selection and association with processing target area)
In step S6003, the image processing apparatus 500 selects a processing target region for each of the tomographic images constituting the target tomographic image group acquired in step S6000 as processing of the alignment parameter calculation unit 540. Then, a corresponding area of the selected processing target area is searched for in the tomographic image captured at the time immediately before the time when each tomographic image is captured.
Specifically, the same processing as Step S4010 and Step S4020 in the first embodiment is executed for all tomographic images.

(S6005) (Data acquisition)
In step S6005, the image processing apparatus 500 acquires the latest tomographic image input to the image processing apparatus 500 at the current time as the reference tomographic image as processing of the reference tomographic image acquisition unit 520.

(S6010) (Setting of attention cross section)
In step S6010, the image processing apparatus 500 sets the position and orientation of the reference tomographic image in the standard coordinate system as the position and orientation of the target cross section as the processing of the attention cross section setting unit 530.

(S6020) (Calculation of alignment parameter)
In step S6020, the image processing apparatus 500 calculates (corrects) the alignment parameter of the target tomographic image group based on the position and orientation of the target cross section set in step S6010 as the processing of the alignment parameter calculation unit 540. Details of the processing of the alignment parameter calculation unit 540 in this step will be described later in detail using the flowchart shown in FIG.

(S6030) (Cross-section image generation)
In step S6030, the image processing apparatus 500 generates a cross-sectional image of the target cross-section set in step S6010 from the target tomographic image group as a process of the cross-sectional image generation unit 550. Since this process is the same as the process of step S3030 in the first embodiment, detailed description thereof is omitted.

(S6040) (Image composition)
In step S6040, the image processing apparatus 500 synthesizes the reference tomographic image acquired in step S6005 and the cross-sectional image generated in step S6030 as processing of the image synthesis unit 560. Then, the synthesized image is displayed on the display unit 1006. Further, this is output to the outside via the I / F 1009 as necessary. Alternatively, it is stored in the RAM 1002 as a state that can be used by other applications.

  For example, the reference tomographic image and the cross-sectional image may be drawn with different colors and superimposed or displayed, or only one of the reference tomographic image and the cross-sectional image may be selected and displayed. Further, one screen may be divided vertically or horizontally, a reference tomographic image may be displayed on one side, and a cross-sectional image may be displayed on the other side, or may be displayed on two screens, respectively.

(S6050) (Determination of termination)
In step S6050, the image processing apparatus 500 determines whether to end the entire process. For example, when the operator clicks the end button arranged on the display unit 1006 with the mouse 1005, the end determination is input. When it is determined that the processing is to be ended, the entire processing of the image processing apparatus 500 is ended. On the other hand, if it is not determined to end, the process returns to step S6005, and the processes from step S6005 to step S6040 are performed again on the newly acquired reference tomographic image.

  Thus, the processing of the image processing apparatus 500 is performed.

  FIG. 7 is a flowchart showing a processing procedure of the alignment parameter calculation unit 540 in step S6020 described above.

  Since the process of step S7000 is the same as the process of step S4000 in the first embodiment, detailed description thereof is omitted.

(S7030) (Calculation of relative alignment parameter)
In step S7030, the alignment parameter calculation unit 540 calculates an alignment parameter between the tomographic image and the tomographic image captured at the immediately preceding time for the unprocessed tomographic image selected in step S7000. This process is performed based on the plurality of process target areas related to the tomographic image selected in step S6003 and the position information of the corresponding areas. Since other processes are the same as those in step S4030 in the first embodiment, detailed description thereof is omitted. However, as the initial value of the position and orientation of the unprocessed tomographic image, a value obtained based on the cross section of interest at the previous time is used.

  Hereinafter, the processes in steps S7035, S7037, S7040, and step S7050 are the same as the processes in steps S4035, S4037, S4040, and step S4050 in the first embodiment, and thus detailed description thereof is omitted.

(S7060) (Coordinate correction)
In step S7060, the alignment parameter calculation unit 540 corrects the position information of the processing target region obtained in step S6003 based on the absolute compression deformation amount of each tomographic image estimated in step S7030. As a result, when the alignment parameter calculation process in step S7030 is performed on a new section of interest, a correction result obtained by subtracting the absolute compressive deformation amount obtained based on the section of attention at the previous time (processing target) The position information of the area and its corresponding area is applied.

  As described above, the image processing apparatus according to the present embodiment is based on the positional relationship between the tomographic image group and the tomographic image obtained in real time, and the alignment parameters of the tomographic images constituting the tomographic image group. Is calculated. By doing so, it is possible to generate an image (cross-sectional image) having the same cross section as the tomographic image obtained by imaging in real time with high image quality. Furthermore, the position and orientation of the cross section of interest set sequentially does not change much from the cross section of interest at the previous time, so the convergence calculation is performed by using the alignment parameters obtained based on the cross section of interest at the previous time. You can speed up.

  In addition, since a cross-sectional image corresponding to an ultrasonic tomographic image obtained by imaging in real time can be generated from a past ultrasonic tomographic image group with high image quality, a cross-sectional image generated from a past ultrasonic tomographic image group And current ultrasonic tomographic images can be easily compared.

(Modification 1 of the second embodiment) (Acquire MPR image with 3D probe)
In the present embodiment, the case where a tomographic image is captured in real time by the ultrasonic diagnostic imaging apparatus has been described as an example, but data acquired by the ultrasonic diagnostic imaging apparatus is not limited to this. For example, the method of the above embodiment can also be applied when acquiring an MPR (Multi Planar Reformat) image with a three-dimensional ultrasonic probe. That is, the method of the above-described embodiment may be applied to each attention cross section with each of the plurality of cut surfaces as the attention cross section.

(Modification 2 of the second embodiment) (UI designated cross section is a cross section of interest)
In this embodiment, the case where the reference tomographic image and the cross-sectional image are combined and displayed has been described as an example, but only the cross-sectional image may be displayed as in the first embodiment. In that case, the image composition unit 560 becomes unnecessary.

  Furthermore, instead of setting the position and orientation of the reference tomographic image to the position and orientation of the target section, the position and orientation of the target section specified by the operator may be acquired by the following method, for example. In that case, the reference tomographic image acquisition unit 520 is also unnecessary. First, the operator uses the mouse 1005, the keyboard 1004, and the like to specify the rotation direction and rotation angle of the cross section of interest and the movement in each axial direction. For example, by selecting a cross-sectional image displayed on the display unit 1006 with the mouse 1005 and performing a drag operation while holding down the left mouse button, the amount of change in posture of the cross-section of interest that rotates the cross-section of interest in the dragged direction. Is calculated. At this time, the rotation direction of the cross section of interest may be calculated according to the movement direction of the mouse during the drag operation, and the rotation angle of the cross section of interest may be calculated according to the amount of movement of the mouse during the drag operation. For example, a mouse displacement may be set as a two-dimensional vector on the cross-sectional image, a value proportional to the magnitude of the vector may be set as the rotation angle, and a vector on the cross-sectional image orthogonal to the vector may be set as the rotation axis. Next, the amount of movement of the cross section of interest in the image plane is specified by performing a drag operation while pressing the right button of the mouse. Then, the amount of movement in the normal direction of the cross section of interest is specified by rotating the mouse wheel in the front-rear direction. Not only the mouse drag operation, but also a specific key on the keyboard may be assigned a rotation operation in each direction, a change in rotation angle, or a movement in each axial direction. Alternatively, an operation button or the like may be arranged on the display screen and operated by clicking with the mouse. When the change amount of the position and orientation is obtained as described above, a new position and orientation is calculated by adding the change amount to the current position and orientation of the target cross section.

  In addition, the configuration may be such that the position and orientation measurement value of the sensor is acquired as the position and orientation of the target cross section when the operator moves and rotates a position and orientation indicator equipped with a position and orientation sensor. In this case, a probe equipped with a position and orientation sensor may be used as a position and orientation indicator.

(Another example of "weight according to the distance from the cross section of interest")
In the first embodiment, a case has been described in which a weight corresponding to the distance from the cross section of interest is set in the calculation for calculating the alignment parameter of each tomographic image constituting the tomographic image group. However, the method of setting the weight according to the distance from the target section is not limited to this, and for example, the closer to the target section, the more processing target regions may be selected. The image processing apparatus according to the present embodiment selects a processing target region based on the positional relationship between the tomographic image group and the target cross section when generating an image of the target cross section (cross section image) from the tomographic image group, and generates the tomographic image. The alignment parameter of each tomographic image constituting the group is calculated (corrected). By doing so, a cross-sectional image is generated with high image quality. Hereinafter, only the difference from the first embodiment will be described for the image processing apparatus according to this embodiment.

Since the configuration of the image processing apparatus in this embodiment is the same as that shown in FIG. 1, the description thereof is omitted.
A flowchart showing the overall processing procedure performed by the image processing apparatus according to the present embodiment is the same as that shown in FIG. However, only the details of the processing in step S3020 are different.

  FIG. 8 is a flowchart showing the processing procedure of the alignment parameter calculation unit 140 in step S3020 of the third embodiment.

  Since the process in step S8000 is the same as the process in step S4000 in the first embodiment, a detailed description thereof will be omitted.

(S8005) (Calculation of intersection line)
In step S8005, the alignment parameter calculation unit 140 calculates an intersection line between the unprocessed tomographic image and the cross section of interest. Since the position and orientation of the unprocessed tomographic image are given as sensor initial values and the position and orientation of the cross section of interest are obtained in step S3030, the intersection of the two planes is calculated by a well-known method. Can do.

(S8010) (Selection of processing target area)
In step S8010, the alignment parameter calculation unit 140 divides the unprocessed tomographic image into partial regions, and selects a plurality of processing target regions therefrom. Also in the present embodiment, when the partial area of interest is a sufficiently developed speckle area, that area is selected as a processing target area. Specifically, the following processing is executed. First, an unprocessed tomographic image is divided into, for example, 5 × 5 partial areas that do not overlap each other. In this embodiment, the partial area that intersects the intersection calculated in step S8005 is further divided into 2 × 2 partial areas that do not overlap each other. Next, whether or not each partial area is a sufficiently developed speckle area is determined based on whether or not the ratio of the average value of the pixel values in the partial area to the standard deviation is close to 1. Then, a partial area determined to be a sufficiently developed speckle area is selected as a process target area. As a result of the processing in this step, generally, the closer to the cross section of interest, the more processing target regions are selected.

  Since the process of step S8020 is the same as the process of step S4020 in the first embodiment, detailed description thereof is omitted.

(S8030) (Calculation of relative alignment parameter)
In step S8030, the alignment parameter calculation unit 140 calculates an alignment parameter between the tomographic image and the tomographic image captured at the immediately preceding time for the unprocessed tomographic image selected in step S8000. Since the process of this step is the same as that of step S4030 in the first embodiment, detailed description thereof is omitted. However, weighting calculation according to the distance from the cross section of interest is not performed.

  Hereinafter, the processes in steps S8035, S8037, S8040, and step S8050 are the same as the processes in S4035, S4037, S4040, and step S4050 in the first embodiment, and thus detailed description thereof is omitted.

  As described above, the image processing apparatus according to the present embodiment configures a tomographic image group by selecting a larger number of processing target regions closer to the target cross section based on the positional relationship between the tomographic image group and the target cross section. The alignment parameter of each tomographic image is corrected. As a result, alignment is performed so that the error becomes smaller as the section is closer to the target section, so that a section image can be generated with high image quality.

(Modification 1 of the third embodiment)
The method of this embodiment may be combined with the method of the first embodiment. That is, the closer to the cross section of interest, the more processing target regions are selected, and the weighting corresponding to the distance from the cross section of interest is further applied to calculate (correct) the alignment parameters of the tomographic image group. Thereby, alignment is performed so as to minimize the error in the vicinity of the cross section of interest, so that a cross-sectional image can be generated with high image quality.

  Further, when integrating the position and orientation between two tomographic images calculated based only on image features and the position and orientation between two tomographic images measured by the position and orientation sensor, the closer to the cross section of interest, The former weight may be increased. In addition, any method may be used as long as the alignment parameter is obtained in consideration of the positional relationship with the cross section.

(Other examples)
Although the embodiments have been described in detail above, the present invention can take an embodiment as, for example, a system, apparatus, method, program, or storage medium. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  In the present invention, the functions of the above-described embodiments are achieved by supplying a software program directly or remotely to a system or apparatus, and the computer of the system or apparatus reads and executes the supplied program code. Including the case. In this case, the supplied program is a computer program corresponding to the flowchart shown in the drawing in the embodiment.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the computer-readable storage medium for supplying the computer program include the following. For example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD- R).

  As another program supply method, a client computer browser is used to connect to a homepage on the Internet, and the computer program of the present invention is downloaded from the homepage to a recording medium such as a hard disk. In this case, the downloaded program may be a compressed file including an automatic installation function. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  Further, the program of the present invention may be encrypted, stored in a storage medium such as a CD-ROM, and distributed to users. In this case, a user who has cleared a predetermined condition is allowed to download key information for decryption from a homepage via the Internet, execute an encrypted program using the key information, and install the program on the computer. You can also.

  In addition to the functions of the above-described embodiment being realized by the computer executing the read program, the embodiment of the embodiment is implemented in cooperation with an OS running on the computer based on the instruction of the program. A function may be realized. In this case, the OS or the like performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, so that part or all of the functions of the above-described embodiments are realized. May be. In this case, after a program is written in the function expansion board or function expansion unit, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 110 Target tomographic image group acquisition part 120 Reference tomographic image group acquisition part 130 Attention cross-section setting part 140 Positioning parameter calculation part 150 Cross-sectional image generation part

Claims (14)

An alignment unit that aligns the tomographic image so that the position of the corresponding partial region matches the smaller the distance between the corresponding partial region and the predetermined plane for each of the tomographic images of the subject;
An image processing apparatus comprising: generating means for generating a cross-sectional image from the plurality of aligned tomographic images based on position information of the predetermined surface.   The image processing apparatus according to claim 1, wherein the generation unit generates a cross-sectional image included in the same surface as the predetermined surface.   The image processing apparatus according to claim 1, wherein the alignment unit weights the corresponding partial region according to a distance from the cross section, and performs the alignment based on the weighting. Designating means for designating the predetermined surface;
The image processing apparatus according to claim 1, wherein the alignment unit performs the alignment in response to the cross section being changed by the designation unit.   The image processing apparatus according to claim 1, wherein the tomographic image is an ultrasonic tomographic image.   The image processing apparatus according to claim 5, wherein the alignment unit sets an area where speckle noise is sufficiently developed for the tomographic image as the partial area.   The alignment means calculates the distance between the corresponding partial areas using the fact that an average correlation curve indicating the relationship between the distance and the correlation value of the corresponding partial area becomes a Gaussian curve. The image processing apparatus according to claim 5, wherein the distance is calculated.   The image processing apparatus according to claim 5, wherein the alignment unit performs alignment according to an amount of deformation caused by pressing an ultrasonic probe that acquires the tomographic image against the subject.   The image processing apparatus according to claim 1, wherein the alignment unit performs alignment of adjacent tomographic images. A selection unit that selects a tomographic image serving as a reference for alignment from the plurality of tomographic images;
The image processing apparatus according to claim 1, wherein the alignment unit performs alignment in a chain manner from the selected tomographic image. An alignment means for aligning the tomographic image so that the position of the corresponding part matches the smaller the distance between the corresponding part and the predetermined region for each of the tomographic images of the subject;
Generating means for generating a cross-sectional image from the plurality of tomographic images that have been aligned based on position information of the predetermined region;
An image processing apparatus comprising: A step of aligning the tomographic image so that the position of the corresponding partial region is matched as the distance between the corresponding partial region and the predetermined surface for each of the plurality of tomographic images of the subject is small;
Generating a cross-sectional image from a plurality of tomographic images that have been aligned based on position information of the predetermined surface;
An image processing method comprising: A process of aligning the tomographic images so that the positions of the corresponding partial regions match each other as the distance between the corresponding partial region and the predetermined surface of each of the plurality of tomographic images of the subject decreases;
A process of generating a cross-sectional image from the plurality of tomographic images that have been aligned based on position information of the predetermined surface;
A program that causes a computer to execute. Acquisition means for acquiring a plurality of tomographic images of the subject;
An alignment unit that aligns the tomographic images so that the position of the corresponding partial region matches the smaller the distance between the corresponding partial region and the predetermined plane for each of the plurality of tomographic images;
An image processing system comprising: generating means for generating a cross-sectional image from the plurality of aligned tomographic images based on position information of the predetermined surface.

Images