WO2018079344A1 - Ultrasound image processing device and program - Google Patents

Abstract

With an adjustment unit (62), a representative cross-section detection unit (66), and a feature point/size detection unit (68), with regard to volume data (40) which includes a three-dimensional image of a fetal heart, the volume data (40) is rotated such that the three-dimensional image of the fetal heart is in a stipulated orientation. A plurality of feature points are detected in the three-dimensional image of the fetal heart which has been placed in the stipulated orientation. On the basis of the coordinates of the plurality of feature points in an actual (rotated) coordinate system, a normalized coordinate setting unit (70) sets a normalized coordinate system. A conversion function generating unit (72) generates a correspondence between the actual coordinate system and the normalized coordinate system. On the basis of the correspondence, a cross-section identification unit (74) identifies an actual data observation cross-section in the actual coordinate system from a normalized observation cross-section which has been defined in the normalized coordinate system.

Description

Ultrasonic image processing apparatus and program

The present invention relates to an ultrasonic image processing apparatus and a program, and more particularly to a technique for processing volume data.

An ultrasonic image can be obtained by transmitting and receiving an ultrasonic beam to and from a subject using an ultrasonic diagnostic apparatus. Conventionally, an ultrasonic beam is scanned in a three-dimensional space, and echo data is sequentially collected from the three-dimensional space. Based on the collected echo data, three-dimensional ultrasonic data (volume data) or volume data is obtained. An STIC (Spatio-Temporal Image Correlation) method for generating four-dimensional ultrasound data arranged in the time direction is known. There is also known a technique for acquiring four-dimensional ultrasonic data in real time by transmitting and receiving an ultrasonic beam in an ultrasonic probe having a two-dimensional array transducer.

A user such as a doctor may specify an observation cross section (main cross section) in the volume data and perform observation or diagnosis using a tomographic image in the specified observation cross section. For example, volume data including a fetal heart image may be acquired in advance, and a tomographic image of an arbitrary observation cross section in the volume data may be displayed on the display unit for diagnosis or observation. In general, the cross section to be observed is predetermined for each diagnosis content. For example, in the case of the heart, when an abnormality of the left ventricle, left atrium, right ventricle, and right atrium is observed, a four-chamber cross section (a cross section including the left ventricle, left atrium, right ventricle, and right atrium) is observed. It becomes a power section. Therefore, the user needs to appropriately specify the observation cross section in the volume data according to the diagnosis contents.

∙ Operation to specify an appropriate observation section in volume data may be difficult. For example, in the case of the fetal heart, since the fetus can take various postures within the mother's body, the fetal heart can be in various positions relative to a predetermined position on the mother's surface (ie, a predetermined ultrasonic wave transmitting / receiving surface). The orientation (posture) can be taken. That is, for each volume data, the position or orientation of the fetal heart can vary. Therefore, it may be difficult or time-consuming to specify a target observation cross section in the fetal heart from the volume data.

In view of the above, conventionally, techniques for automatically specifying an observation cross section in volume data have been proposed (for example, Patent Documents 1 to 6).

JP 2014-36863 A Japanese Patent No. 5479138 US Patent Publication No. 2015/0190112 JP 2009-72593 A Special table 2009-513221 gazette Special table 2015-534872 gazette

In a conventional method of automatically specifying an observation cross section in volume data, there is a method of first specifying a reference cross section and specifying an observation cross section based on the reference cross section. For example, in Patent Document 1, a reference cross section is first specified from volume data, and a plane obtained by horizontally moving the specified reference cross section by a predetermined distance is used as an observation cross section. In Patent Document 2, a reference cross section is specified based on three reference points detected in an observation target (heart) included in volume data, and a plane obtained by rotating the reference cross section by a predetermined angle in a predetermined direction is an observation cross section. It is said. In Patent Document 3, a user inputs a plurality of feature points (reference points) in an observation target (heart) included in volume data, and specifies a plurality of arbitrary cross sections based on the plurality of reference points.

In the conventional method of specifying the observation cross section, the cross section specified as the observation cross section may not be an accurate observation cross section intended by the user. For example, when specifying a cross section horizontally moved by a predetermined distance from a four-chamber cross section as a reference cross section as a five-chamber cross section (a cross section including the left ventricle, left atrium, right ventricle, right atrium, and aorta), the shape of the heart (in this case In particular, the distance between the four-chamber cross-section and the five-chamber cross-section has individual differences. Therefore, in this method, an accurate five-chamber cross-section may not be specified depending on the subject.

An object of the present invention is to easily and more accurately specify an observation cross section in consideration of individual differences of target tissues in volume data obtained by transmission and reception of ultrasonic waves.

The ultrasonic image processing apparatus according to the present invention is based on the target tissue image included in the volume data obtained by transmission and reception of ultrasonic waves, and the real coordinate system included in the volume data and the calculated normalized coordinate system. Correspondence generation means for calculating a correspondence relation, and based on the correspondence relation, from a normalized observation cross section defined in the normalized coordinate system, a cross section specifying means for specifying an actual data observation cross section in the real coordinate system, Image forming means for forming a tomographic image corresponding to the identified actual data observation section from the volume data.

According to the above configuration, first, the correspondence between the real coordinate system and the normalized coordinate system of the volume data is calculated. The correspondence relationship indicates coordinate conversion between the real coordinate system and the normalized coordinate system. A normalized observation section is defined in the normalized coordinate system. For example, the normalized observation cross section is defined by three coordinates in the normalized coordinate system. In the normalized coordinate system, a target tissue image having a prescribed size or orientation can be defined. Therefore, a normalized observation cross section corresponding to each observation cross section can be uniquely defined without considering individual differences in the target tissue image. According to the correspondence relationship between the real coordinate system and the normalized coordinate system, each real data observation section on the real coordinate system corresponding to each normalized observation section defined in the normalized coordinate system can be specified. The correspondence between the real coordinate system and the normalized coordinate system can be calculated for each target tissue image. Therefore, by specifying the actual data observation cross section based on the normalized observation cross section and the correspondence relationship, even if there is an individual difference in the target tissue image, the individual difference is absorbed, and in any target tissue image A more accurate actual data observation section is specified in the volume data.

Desirably, the correspondence generation means calculates the correspondence based on a representative point group detected from the target tissue image. Preferably, the representative point group includes a reference point of the target tissue image and both end points of the target tissue image on each coordinate axis of the real coordinate system, and the normalized coordinate system includes the reference point and Defined based on the endpoints.

Preferably, the normalized coordinate system has at least two different scales.

Since the normalized coordinate system has a plurality of different scales, it is possible to specify a more accurate actual data observation section in the volume data in consideration of the size balance of each part included in the target tissue and distortion of the arrangement relationship. be able to. For example, considering the case where the target tissue is the heart, the difference in the shape of the heart due to individual differences may not only be different in outline but also in the ratio of the size of the left and right ventricles, for example. In such a case, by changing the left and right scales based on the septum of the heart in the normalized coordinate system, it is possible to specify the actual data observation section while eliminating the balance of the left and right ventricle sizes. .

Preferably, a plurality of the normalized observation cross sections are defined, and the cross section specifying means is configured to select the realization corresponding to the selected normalized observation cross section based on a selected normalization observation cross section selected from the plurality of normalization observation cross sections. Identify the data observation cross section.

In the normalized coordinate system, a plurality of normalized observation sections can be defined in advance. According to the present invention, even if any one of the plurality of normalized observation sections is selected, the actual data observation section corresponding to the selected normal observation section based on the correspondence between the actual coordinate system and the normalized coordinate system. Can be specified.

Desirably, it further includes a representative cross-section specifying means for specifying a representative cross-section in the volume data, wherein the correspondence relationship generating means includes the coordinates in the real coordinate system and the normality of a plurality of representative points detected in the representative cross-section. The correspondence is calculated based on the relationship with the coordinates in the coordinated coordinate system.

Preferably, the representative cross-section specifying means includes a posture specifying means for specifying a posture of the volume data so that a target tissue image included in the temporary representative cross-section specified in the volume data matches template data, and the posture Representative volume search means for searching for the representative cross section in the vicinity of the temporary representative cross section in the volume data of the posture defined by the defining means.

According to this configuration, first, the volume data is rotated so that the specified temporary representative cross-section predetermined (orientation) is in the prescribed orientation. The direction of the representative cross-sectional image is the direction indicated by the template data. For example, the direction on the apex side of the heart is defined. This orientation corresponds to the orientation of the target tissue image defined in the normalized coordinate system. Then, by searching for the vicinity of the temporary representative cross-section in the rotated volume data, the representative cross-section facing the specified direction can be specified. By accurately specifying the representative cross section, a more accurate correspondence between the real coordinate system and the normalized coordinate system can be suitably calculated.

Preferably, a display unit that displays the tomographic image, and in the normalized coordinate system, a label space region that corresponds to each part of the target tissue image and includes label information indicating each part is defined, When the normalized observation cross section crosses the label space region, the display unit displays label information corresponding to the label space region crossed by the normalized observation cross section together with a tomographic image corresponding to the normalized observation cross section. indicate.

According to the configuration, it is possible to display the label information corresponding to the label space region traversed by the normalized section, regardless of which normalized section is selected. Thereby, the user can easily grasp which part is which part in the displayed tomographic image. Preferably, the label information is displayed so that the correspondence between the part included in the tomographic image corresponding to the normalized observation section and the part corresponding to the label information becomes clear.

According to the present invention, it is possible to easily and more accurately specify an observation cross section in consideration of individual differences of target tissues in volume data obtained by transmitting and receiving ultrasonic waves.

1 is a schematic configuration diagram of an ultrasonic diagnostic apparatus according to the present embodiment. It is a figure for demonstrating the three-dimensional scanning in this embodiment. It is a figure which shows several volume data corresponding to each time phase. It is a figure which shows the observation cross section in the heart of a fetus. It is a figure which shows the detailed structure of a volume data processing part. It is a figure which shows the mode of the reversal binarization process with respect to a temporary four-chamber cross section. It is a graph which shows the threshold value for a binarization process. It is a figure which shows template data. It is a figure which shows the mode of a matching process with provisional four-chamber cross-sectional data and template data. It is a figure which shows the mode of the process which searches a four-chamber cross section in temporary four-chamber cross-section data vicinity. It is a figure which shows the some feature point detected in the four-chamber cross-sectional image. It is an enlarged view near the feature point A. FIG. 6 is an enlarged view near a feature point C. It is a figure which shows a heart area | region surface. It is a figure which shows the mode of the process which searches the edge part of the heart stereo image in a rz-axis direction. It is a figure which shows a cardiac space area | region. It is a figure which shows a normalization coordinate system. It is a figure which shows the scale of the nx-axis and ny-axis direction in a normalization coordinate system. It is a figure which shows the scale of the ny-axis and nz-axis direction in a normalization coordinate system. It is a figure which shows eight area | regions from which a scale differs in a normalization coordinate system. It is a figure for explanation of a correction term in the 1st conversion function. It is a figure which shows the mode of the conversion process from a normalization coordinate system to a real coordinate system. It is a figure which shows the 1st example of a display of the tomographic image of a real data observation cross section. It is a figure which shows the 2nd example of a display of the tomographic image of an actual data observation cross section. It is a figure which shows the 3rd example of a display of the tomographic image of an actual data observation cross section. It is a figure which shows the 4th example of a display of the tomographic image of an actual data observation cross section. It is a figure which shows the label space area | region defined in the normalization coordinate system. It is a figure which shows the label plane area | region on the actual data observation cross section corresponding to the normalization observation cross section which crosses a label space area | region. It is a figure which shows the 1st example of a label display. It is a figure which shows the 2nd example of a label display. It is a figure which shows the mode of the three-dimensional region growing process of a label space area | region. It is a figure which shows the detailed mode of a three-dimensional region growing process. It is a figure which shows the mode of the boundary production | generation process of the valve part in a three-dimensional region growing process.

Hereinafter, embodiments of the ultrasonic image processing apparatus according to the present invention will be described.

FIG. 1 is a schematic configuration diagram of an ultrasonic diagnostic apparatus 10 as an ultrasonic image processing apparatus. The ultrasonic diagnostic apparatus 10 is generally a medical device that is installed in a medical institution such as a hospital and performs ultrasonic diagnosis on a living body. In the following, the present embodiment will be described using an example in which a target tissue is a fetal heart and a tomographic image of an observation cross section in the fetal heart is formed, but the present invention is not limited thereto.

Probe 12 is an ultrasonic probe that transmits and receives ultrasonic waves to and from the fetal heart. The probe 12 is communicably connected to the apparatus main body including the transmission / reception unit 14 and below by a cable or wirelessly. The probe 12 has a transducer array including a plurality of transducer elements, and ultrasonic waves are transmitted and received by the transducer array. A scanning plane is formed in a space including the fetal heart by transmitting and receiving ultrasonic waves from the transducer array. In this embodiment, for example, a fetal heart that is a subject is included by mechanically moving a scanning surface that is electronically formed by a plurality of one-dimensionally arranged vibration elements (1D array transducers). An ultrasonic beam is scanned in the dimensional space. Alternatively, a plurality of vibration elements (2D array transducers) arranged two-dimensionally may be electronically controlled to scan the ultrasonic beam three-dimensionally.

FIG. 2 is a diagram for explaining the three-dimensional scanning in the present embodiment. In FIG. 2, the three-dimensional space including the fetal heart is represented by an xyz coordinate system which is a real coordinate system. In the present embodiment, the scanning surface S is formed so as to be substantially parallel to the xy plane, and the plurality of scanning surfaces S are formed along the z-axis direction while slowly moving the scanning surface S in the z-axis direction. It is formed. The scanning plane S is slowly moved in the z-axis direction over a plurality of cardiac cycles (beating cycles) of the fetal heart.

Returning to FIG. 1, the transmission / reception unit 14 forms a transmission beam of ultrasonic waves by supplying a transmission signal corresponding to each transducer included in the transducer array of the probe 12. The transmission / reception unit 14 receives a plurality of reception signals from each transducer included in the transducer array of the probe 12.

The phasing addition unit 16 performs phasing addition processing on the plurality of reception signals received by the transmission / reception unit 14 to form an ultrasonic reception beam, and outputs echo data obtained along the reception beam.

The beam processing unit 18 performs various signal processing such as gain correction processing, logarithmic amplification processing, envelope detection processing, and filter processing on the echo data output from the phasing addition unit 16. Thereby, beam data corresponding to each echo data is formed.

The DSC (Digital Scan Converter) 20 has an interpolation function and a coordinate conversion function, and forms a display frame, that is, an ultrasonic image, based on a plurality of beam data output from the beam processing unit 18. In the present embodiment, a B-mode image that is a tomographic image is formed. Here, the DSC 20 forms an ultrasonic image for each scanning plane S (see FIG. 2). Thereby, a plurality of ultrasonic images arranged in the z-axis direction are formed. The plurality of ultrasonic images are stored in the storage unit 22 described later. The ultrasonic image formed by the DSC 20 may be directly output to the image composition unit 32 (described later).

The storage unit 22 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), or a hard disk. The storage unit 22 stores a program for operating each unit of the ultrasound diagnostic apparatus 10. Further, as shown in FIG. 1, a previous memory 24 and a data memory 26 are constructed in the storage unit 22. A plurality of ultrasonic images formed by the DSC 20 are stored in the previous memory 24. The data memory 26 stores four-dimensional volume data (a plurality of volume data arranged in the time direction) formed by a reconstruction processing unit 28 described later.

The reconstruction processing unit 28 searches a plurality of reference images at a frame interval (image interval) corresponding to the cardiac cycle of the fetal heart from a plurality of ultrasonic images stored in the previous memory 24. When a plurality of reference images are searched, the reconstruction processing unit 28 sets the plurality of ultrasonic images stored in the previous memory 24 as a plurality of images by using each of the plurality of reference images as a unit of division. Divide into groups. Then, a plurality of tomographic image data corresponding to each time phase in the cardiac cycle is extracted from each of the plurality of image groups, thereby realizing reconstruction processing (reconstruction processing). By the reconstruction process, a plurality of volume data including a fetal heart three-dimensional image as a target tissue image is formed from a plurality of ultrasonic images stored in the previous memory 24. The plurality of volume data corresponds to each time phase in the cardiac cycle. Since the plurality of volume data are arranged in the time direction, it can be said that the reconstruction processing unit 28 forms the four-dimensional volume data. The four-dimensional volume data formed by the reconstruction processing unit 28 is stored in the data memory 26.

FIG. 3 shows a plurality of volume data 40 stored in the data memory 26. In the present embodiment, the data memory 26 stores a plurality of volume data 40 for one cardiac cycle, but the volume data 40 over a longer period may be stored. Further, in the present embodiment, the position and posture of the probe 12 are adjusted by a user such as a doctor, so that at least a cross section (provisional four chambers) in the vicinity of the four-chamber cross section in each scanning plane S group constituting each volume data. Cross section). The provisional four-chamber section does not need to be an accurate four-chamber section, and may be a section that is complexly displaced from the exact four-chamber section in the xyz-axis direction. Alternatively, the orientation (rotation direction) and size of the provisional four-chamber cross section in the xy plane need not be specific. It should be noted that ultrasonic waves are transmitted and received in a relatively wide area, a heart stereoscopic image is detected from the volume data obtained thereby, and volume data including at least a provisional four-chamber section is automatically cut out as a scanning plane S by image processing. You may do it.

Returning to FIG. 1, the volume data processing unit 30 performs processing on the four-dimensional volume data stored in the data memory 26, and automatically selects one or a plurality of actual data observation sections desired by the user for each volume data. And forming a tomographic image in the actual data observation cross section. The detailed configuration of the volume data processing unit 30 and the details of the processing will be described later.

The image composition unit 32 synthesizes images or characters indicating various information with the tomographic image (B-mode image) of the actual data observation section formed by the volume data processing unit 30 to form display screen data. is there. The display screen data may include an ultrasonic image formed by the DSC 20.

The display unit 34 is composed of, for example, a liquid crystal panel or an organic EL panel. Display screen data formed by the image composition unit 32 is displayed on the display unit 34.

The control unit 36 is constituted by, for example, a CPU or a microcontroller, and controls each unit of the ultrasonic diagnostic apparatus 10 according to a program stored in the storage unit 22.

The configuration outline of the ultrasonic diagnostic apparatus 10 is as described above. In the configuration illustrated in FIG. 1, each of the transmission / reception unit 14, the phasing addition unit 16, the beam processing unit 18, the DSC 20, the reconstruction processing unit 28, the volume data processing unit 30, and the image composition unit 32 is, for example, It can be realized using hardware such as an electronic circuit or a processor, and a device such as a memory may be used as necessary in the realization. In addition, functions corresponding to the above-described units may be realized by cooperation between hardware such as a CPU, a processor, and a memory, and software (program) that defines the operation of the CPU and the processor.

Hereinafter, before explaining the details of the volume data processing unit 30, the observation cross section (main cross section) in the fetal heart will be described. The observation cross section means a cross section mainly used by a doctor or the like for diagnosis or observation.

FIG. 4 shows a schematic diagram of the fetal heart. In FIG. 4, a fetal heart 42, aorta 44, pulmonary artery 46, superior vena cava 48, and lung 50 are shown. Normally, the apex of an adult heart is directed to the stomach side (the lower side in FIG. 4), but in the case of a fetus, air is not contained in the lung 50, and the heart 42 has a diaphragm (not shown in FIG. 4). Therefore, the direction of the heart 42 is generally the direction in which the apex 42a faces sideways. In such a fetal heart, for example, an observation cross section (cross section orthogonal to the paper surface) indicated by reference numeral 52 in FIG. 4 shows a four-chamber cross section. The observation cross section indicated by reference numeral 54 translated from the head to the head side shows the five-chamber cross section. Furthermore, the observation cross section indicated by reference numeral 56 translated from the head side shows a three blood vessel cross section (a cross section including the pulmonary artery, the aorta, and the superior vena cava), and further, the reference numeral 58 translated from the head side. The observation cross section shown by (3) shows a three-vascular tracheal cross section (a cross section including the pulmonary artery, aorta, superior vena cava and trachea). Note that the observation cross section shown in FIG. 4 is a part, and there are various other observation cross sections (for example, observation cross sections that are not parallel to the four-chamber cross section). According to the volume data processing unit 30, one or a plurality of actual data observation sections are automatically specified in the four-dimensional volume data stored in the data memory 26.

FIG. 5 shows a detailed configuration of the volume data processing unit 30. Hereinafter, the details of the processing of the volume data processing unit 30 will be described along the processing flow.

The filter unit 60 extracts volume data 40 of a specific time phase from a plurality of volume data 40 (see FIG. 3) stored in the data memory 26. In the present embodiment, volume data 40 corresponding to the end diastole is extracted from the cardiac cycle. This is because the end-diastolic volume data 40 in which the heart chamber becomes the largest is more suitable for later processing because the four-chamber cross section is specified later as the reference cross section. The filter unit 60 extracts volume data 40 at a specific time phase by performing image processing on each volume data 40. For example, the volume data 40 corresponding to the end diastole can be detected based on the volume of the blood flow part (for example, in the heart chamber) of the fetal heart in each volume data 40. Specifically, in each volume data 40, the outline of the heart is extracted by three-dimensional image processing, and the volume data 40 having the largest volume of the heart cavity that is the inner space is identified as the volume data 40 at the end diastole. can do.

Further, the filter unit 60 performs a process such as a smoothing filter for smoothing the omission of the tissue on the data or the boundary part on the extracted end-diastolic volume data 40. The filter process reduces the possibility of erroneous detection or erroneous determination in subsequent processes.

The adjustment unit 62 first specifies a provisional four-chamber section as a provisional representative section in the end-diastolic volume data 40 extracted and filtered by the filter unit 60. In the present embodiment, as shown in FIG. 6, the xy section at the center in the z-axis direction is specified as the provisional four-chamber section 90 in the volume data 40. The provisional four-chamber cross section 90 thus identified is generally not an accurate four-chamber cross section. That is, it does not include an accurate four-chamber cross-sectional image. The provisional four-chamber cross section 90 is not limited to the cross section at the center in the z-axis direction, and may be specified based on other criteria as long as the provisional four-chamber cross section 90 can be specified in the volume data 40.

Next, the adjustment unit 62 performs binarization processing on the volume data 40. That is, the adjustment unit 62 also functions as a binarization processing unit. By the binarization processing, subsequent processing can be simplified. In the present embodiment, the adjustment unit 62 performs inversion binarization processing on the volume data 40. FIG. 6 shows a provisional four-chamber cross section 92 in the volume data 40 subjected to the reverse binarization process. Note that the threshold value for binarization processing is determined by analyzing the luminance distribution of each voxel in the volume data 40. In general, the luminance distribution of each voxel in the volume data 40 is a bimodal graph as shown in FIG. In the present embodiment, the adjustment unit 62 performs binarization processing using the luminance corresponding to the first inflection point (the inflection point closest to the luminance value 0) in the bimodal graph as a threshold value.

Further, the adjustment unit 62 defines the posture of the volume data 40 so that the orientation and size of the provisional four-chamber cross-sectional image 92a (see FIG. 6) included in the provisional four-chamber cross section 92 become the prescribed orientation and size. I do. That is, the adjustment unit 62 also functions as a posture defining unit. In the present embodiment, the adjustment unit 62 performs a process of defining the posture based on the template data 64 stored in advance in the storage unit 22.

FIG. 8 shows a schematic diagram of the template data 64. The template data 64 defines the direction and size of the four-chamber cross-sectional image. The template data 64 is used to define the orientation and size of the provisional four-chamber cross-sectional image 92a. The template data 64 in the present embodiment is image data. The template data 64 has an oval part 64a corresponding to the heart cavity part of the four-chamber cross-sectional image. The egg-shaped portion 64a has an asymmetric shape in the up-down direction, and this indicates the prescribed direction of the four-chamber cross-sectional image, particularly the left-right direction as the first direction. In this specification, the left-right direction means the left-right direction when the fetal heart is viewed from the front (see FIG. 4). In the present embodiment, the direction of the apex is shown in the template data 64. Specifically, one end portion of the egg-shaped portion 64a is a narrow end portion 64b, the other end portion is a thick end portion 64c, and the narrow end portion 64b side is the apex side of the four-chamber cross-sectional image. Show.

In addition, the size (area) of the egg shaped portion 64a indicates the prescribed size (area) of the four-chamber cross-sectional image. Furthermore, the template data 64 includes a slit 64d that extends from the narrow end portion 64b of the egg-shaped portion 64a toward the thick end portion 64c. The slit 64d corresponds to the septum of the four-chamber cross-sectional image.

In the template data 64, “1” is set as the luminance value for the pixel of the egg-shaped portion 64a, and “0” is set as the luminance value for the other pixels including the slit 64d.

In the present embodiment, the adjustment unit 62 uses such template data 64 to rotate, enlarge, and enlarge the volume data 40 so that the provisional four-chamber cross-sectional image 92a has a specified orientation and a predetermined size. A reduction process or a parallel movement process of the three-dimensional heart image included in the volume data 40 is performed. Here, these processes are collectively referred to as a matching process.

The state of the matching process is shown in FIG. The luminance value of each pixel corresponding to the heart chamber portion of the provisional four-chamber section 92 that has been inverted and binarized is approximately “1”, and the luminance value of the pixel corresponding to the other portion (such as the myocardium or septum). Is “0”. On the other hand, the luminance value of the pixel of the egg-shaped portion 64a corresponding to the heart cavity portion of the template data 64 is “1”, and the luminance value of the pixel corresponding to the other portion is “0”. The adjustment unit 62 superimposes the provisional four-chamber section 92 and the template data 64 so that the sum of the products of the luminance values of the corresponding pixels is maximized between the provisional four-chamber section 92 and the template data 64. The provisional four-chamber cross section 92 is rotated, enlarged / reduced, or translated. Accordingly, the volume data 40 is also rotated, enlarged / reduced, and the heart stereoscopic image included in the volume data 40 is translated. When calculating the sum of the products of the luminance values of the respective coordinates, the weight is increased as the distance from the slit 64d is increased in the horizontal direction (left and right direction in FIG. 8) of the template data 64, and the weight is decreased as the distance from the slit 64d is increased. Is preferably added. Thus, the orientation and size of the provisional four-chamber cross-sectional image 92a can be determined based on the position of the septum of the slit 64d and the provisional four-chamber cross-sectional image 92a. Through the above processing, the orientation of the volume data 40 can be defined so that the orientation and size of the provisional four-chamber cross-sectional image 92a are as defined in the template data 64. FIG. 9 shows a provisional four-chamber cross section 94 including a provisional four-chamber cross-sectional image 94a subjected to matching processing.

The rotation process of the volume data 40 can be expressed by an equation using a quaternion. The quaternion is expressed by the following expression by four elements that are one real part and three imaginary parts.

Here, t is a real part and x, y, and z are imaginary parts.
A certain coordinate (X, Y, Z) in the coordinate system is expressed as follows according to a quaternion.

Based on the above assumption, by using a quaternion, rotation processing around an arbitrary rotation axis in a three-dimensional coordinate system can be simply expressed. Specifically, the rotation process of the angle θ with the unit vector (α, β, γ) starting from the origin as the rotation axis is expressed by the following two quaternions (conjugate quaternions).

According to the two quaternions Q and R described above, the value of the coordinate P after the rotation processing is represented by the imaginary part (XR, YR, ZR) of the solution of the following equation.

When the rotation axis does not start from the coordinate origin, when obtaining the coordinates after the rotation process, the rotation axis and the coordinates of the target of the rotation process are translated and the rotation axis is set to the coordinate origin. Then, the coordinates after the rotation processing are obtained by the above formula, and the target coordinates can be obtained by translating the coordinates in the reverse direction.

The adjustment unit 62 holds each parameter related to the rotation processing of the volume data 40 in the storage unit 22. Specifically, a unit vector (α, β, γ) representing the rotation axis and θ representing the rotation angle are held. When the volume data 40 is enlarged / reduced, the enlargement / reduction ratio is retained. Furthermore, when a translation is performed with respect to the heart stereoscopic image, a vector indicating the translation is held.

In the present embodiment, the template data 64 is used to perform the matching process, but the orientation and size of the provisional four-chamber cross-sectional image 92a are adjusted by performing image recognition processing on the provisional four-chamber section 92. You may do it. For example, a feature point (for example, apex portion, central part of the septum, etc.) included in the provisional four-chamber cross-sectional image 92a is detected, and based on the detected feature point, the provisional four-chamber cross-sectional image 92a (and thus the volume data 40) is You may make it perform the process to perform. In such a process, a machine learning method that uses the accumulated results of past adjustment processes may be used.

The representative cross-section detector 66 as representative cross-section search means searches for the (true) four-chamber cross section as the representative cross-section in the adjusted volume data with reference to the provisional four-chamber cross section 94 matched by the adjusting section 62. Process. FIG. 10 shows a processing flow of the representative cross-section detection unit 66. In the upper left diagram of FIG. 10, a post-adjustment volume data 96 whose posture has been adjusted by the adjustment unit 62 and a provisional four-chamber cross section 94 including the provisional four-chamber cross-sectional image 94a that has been adjusted are shown.

First, the representative cross-section detection unit 66 superimposes the temporary four-chamber cross-sectional image 94a included in the temporary four-chamber cross section 94 and the template data 64, and the rotation axis that is the tangent to the lower end (the lower end of the thick end portion 64c) in the template data 64. The temporary four-chamber cross section 94 and the template data 64 are both rotated about 98a (see the upper right figure in FIG. 10). Then, while rotating the provisional four-chamber section 94 and the template data 64, the brightness of each coordinate corresponding to the cross-sectional image included in the rotated provisional four-chamber section 94 and the template data 64 is similar to the matching process shown in FIG. The sum of the products of the values is calculated, and in the adjusted volume data 96, the first detection cross section 100 (see the lower left figure in FIG. 10) that is the cross section in which the sum is maximized (that is, the area in the heart chamber is maximized). To detect.

Next, the representative cross-section detection unit 66 sets both the first detection cross-section 100 and the template data 64 around the rotation axis 98b that is the tangent to the upper end (the upper end of the narrow end 64b) of the template data 64 in the first detection cross-section 100. Rotate. Then, while rotating the first detection section 100 and the template data 64, the sum of the products of the brightness values of the corresponding coordinates of the rotated first detection section 100 and the template data 64 is calculated. The second detection cross section 102 (see the lower right diagram in FIG. 10), which is the cross section in which the sum is the maximum, is detected. Note that either the rotation process based on the lower end of the template data 64 or the rotation process based on the upper end may be performed first.

Furthermore, the representative cross-section detector 66 rotates both the second detection cross-section 102 and the template data 64 around the central axis 98c (axis along the slit 64d) 98c of the template data 64 in the second detection cross-section 102. Then, while rotating the second detection cross section 102 and the template data 64, the sum of the products of the brightness values of the corresponding coordinates of the rotated second detection cross section 102 and the template data 64 is calculated. The cross section where the sum is the maximum is detected.

By the above-described processing, a cross section having a larger heart chamber area is searched in the vicinity of the provisional four-chamber cross section 94 in the adjusted volume data 96. The cross section having the maximum cardiac chamber area searched in this way is specified as the four-chamber cross section. The apex direction of the four-chamber cross-sectional image included in the four-chamber cross section specified in this way indicates the apex direction of the three-dimensional heart image in the adjusted volume data 96. As described above, in the present embodiment, the adjustment unit 62 and the representative cross-section detection unit 66 constitute a representative cross-section specifying unit or a first direction specifying unit.

In addition, each rotation process by the representative cross-section detection unit 66 is also executed using an expression including a quaternion, and the representative cross-section detection unit 66 holds each parameter related to each rotation process in the storage unit 22. Specifically, a unit vector (α, β, γ) representing each rotation axis and θ representing each rotation angle are held.

The feature point / size detection unit 68 performs image processing on the four-chamber cross section detected by the representative cross-section detection unit 66 to detect a plurality of feature points in the four-chamber cross-sectional image included in the four-chamber cross section. FIG. 11 shows feature points detected in the four-chamber cross-sectional image 104 a included in the four-chamber cross-section 104. FIG. 11 shows the coordinate axes of the rotating real coordinate system in which the four-chamber cross section is the rxry plane and the direction orthogonal to the rxry plane is the rz axis.

In this embodiment, seven feature points are detected in the four-chamber cross-sectional image 104a. Specifically, as shown in FIG. 11, a total of seven feature points A to G including the upper and lower ends, the left and right ends, and the three feature points indicating the centers as reference points are detected.

Feature point A indicates the upper end of the four-chamber cross-sectional image 104a. As shown in FIG. 12A, an approximate curve that passes through a plurality of points (represented by x marks in FIG. 12A) detected as a result of the edge search in the vicinity of the upper end of the four-chamber cross-sectional image 104a is calculated. The vertex is identified as the feature point A. The feature point B indicates the lower end of the four-chamber cross-sectional image 104a and is identified by the same method as the feature point A.

Feature point C indicates the center of the four-chamber cross-sectional image 104a. The feature point C is defined as a midpoint between the feature point D and the feature point E. The feature point D indicates one of the left and right ventricular end portions on the septal side and the valve side. As shown in FIG. 12B, the right edge of the septum is searched from the upper end to the lower end of the four-chamber cross-sectional image 104a, and when the inflection point is detected, the inflection point is specified as the feature point D. Is done. The inflection point can be detected as a point where the amount of change in the ry coordinate between the edge point detected immediately before and the edge point detected this time is 0 or negative. The feature point E indicates the other septum side and valve end of the left or right ventricle. The left edge of the septum is searched from the upper end to the lower end of the four-chamber cross-sectional image 104a, and when the inflection point is detected, the inflection point is specified as the feature point E.

Feature point F indicates the left end of the four-chamber cross-sectional image 104a. As shown in FIG. 12B, the edge search is performed along the left line from the upper end to the lower end of the four-chamber cross-sectional image 104a, and when the inflection point is detected, the inflection point becomes the feature point F. Identified. The feature point G indicates the right end of the four-chamber cross-sectional image 104a. The edge search is performed along the right line from the upper end to the lower end of the four-chamber cross-sectional image 104a, and when the inflection point is detected, the inflection point is specified as the feature point G.

Next, the feature point / size detection unit 68 performs processing for detecting the size of the heart stereoscopic image in the adjusted volume data 96. Specifically, the feature point / size detection unit 68 performs a process of detecting a rectangular parallelepiped-shaped region (hereinafter referred to as “heart solid region”) circumscribing the heart stereoscopic image in the adjusted volume data 96.

In the size detection process, first, the feature point / size detection unit 68 specifies the rx coordinates of the left and right ends of the four-chamber cross-sectional image 104a in the four-chamber cross-section 104 as shown in FIG. As described above, the left and right ends of the four-chamber cross-sectional image 104 a are detected by the feature point / size detection unit 68. Specifically, the rx coordinate of the feature point F is the rx coordinate (min_rx) at the left end of the four-chamber cross-sectional image 104a, and the rx coordinate of the feature point G is the rx coordinate (max_rx) at the right end of the four-chamber cross-sectional image 104a. Similarly, the feature point / size detection unit 68 specifies the ry coordinates of the upper and lower ends of the four-chamber cross-sectional image 104a in the four-chamber cross-section 104. As described above, the upper and lower ends of the four-chamber cross-sectional image 104a are also detected by the feature point / size detection unit 68. Specifically, the ry coordinate of the feature point A is the rx coordinate (min_ry) of the upper end of the four-chamber cross-sectional image 104a, and the ry coordinate of the feature point B is the ry coordinate (max_ry) of the lower end of the four-chamber cross-sectional image 104a.

Through the above-described processing, the heart region surface 106 that is one surface constituting the three-dimensional heart region is specified in the four-chamber cross section 104. Specifically, a rectangle defined by rx = min_rx, rx = max_rx, ry = min_ry, ry = max_ry is the heart region surface 106.

Next, the feature point / size detection unit 68 performs processing for detecting both ends of the heart stereoscopic image in the rz-axis direction. FIG. 14 shows how the processing is performed. As described above, the feature point / size detection unit 68 translates the processing surface parallel to and the same size as the heart region surface 106 specified in the four-chamber cross section 104 in the negative direction of the rz axis, and the luminance on the processing surface. The number of pixels with the value “1” is monitored. Then, the point of the luminance value “1” on the processing surface 106 a immediately before the number of pixels of the luminance value “1” is 0, that is, the processing surface becomes a black image, is specified as the feature point H. When there are a plurality of points having the luminance value “1” on the processing surface 106a, the center of gravity of the plurality of points is obtained and set as the feature point H. Then, the rz coordinate of the feature point H (that is, the processing surface 106a) is specified as min_rz that is the end of the heart solid region in the negative direction of the rz axis. Similarly, the number of pixels with the luminance value “1” on the processing surface is monitored while the processing surface parallel to the heart region surface 106 and having the same size is translated in the positive direction of the rz axis. The point of the luminance value “1” on the processing surface 106b immediately before the number becomes 0 is specified as the feature point I. The processing in the case where there are a plurality of points having the luminance value “1” on the processing surface 106 b is the same as the method for specifying the feature point H. Then, the rz coordinate of the feature point I (processing surface 106b) is specified as max_rz which is the end of the heart solid region in the positive direction of the rz axis. As described above, in the present embodiment, nine feature points including seven points in the four-chamber cross-sectional image 104a and two points indicating both ends in the rz-axis direction are specified in the cardiac stereoscopic image.

Here, from the structural features of the fetal heart, with the heart region plane 106 as a reference, the one-side partial heart stereoscopic image on one side in the rz-axis direction and the other-side partial heart stereoscopic image on the other side in the rz-axis direction The shapes are different from each other. From this, by comparing the one-side partial heart stereoscopic image and the other-side partial heart stereoscopic image, the orientation of the cardiac stereoscopic image in the up-down direction as the second direction can be specified. In the present specification, the vertical direction means a vertical direction when the fetal heart is viewed from the front (see FIG. 4), and a direction orthogonal to the horizontal direction. In FIG. 14, the vertical direction is indicated by the rz-axis direction. Specifically, the section length from one end of the three-dimensional heart region to the heart region surface 106 (four-chamber cross section) (that is, the length along the rz axis from min_rz to the rz coordinate of the heart region surface 106), Based on the section length from the other end of the three-dimensional heart region to the heart region surface 106 (that is, the length along the rz axis from the rz coordinate of the heart region surface 106 to max_rz), the vertical direction of the three-dimensional heart image is specified. can do. Specifically, the longer section length is the head side, and the shorter section length is the stomach side. Thus, the feature point / size detection unit 68 also functions as a second direction specifying unit.

By the above process, the three-dimensional heart region 108 is specified in the adjusted volume data 96 as shown in FIG. Specifically, the heart solid region 108 is a region where rx coordinates are min_rx to max_rx, ry coordinates are min_ry to max_ry, and rz coordinates are min_rz to max_rz. As described above, in the present embodiment, six feature points (feature points A, B, F, G, H, and I) among the nine feature points (feature points A to I) identified in the heart stereoscopic image. Based on this, the three-dimensional heart region 108 is specified.

The feature point / size detection unit 68 stores each value of min_rx, max_rx, min_ry, max_ry, min_rz, and max_rz in the storage unit 22 and holds them. Further, among the identified feature points, at least the coordinates of the feature points A, B, and C in the rotating coordinate system are held in the storage unit 22. Here, the coordinates of the feature point A are (ax, ay, az), the coordinates of the feature point B are (bx, by, bz), and the coordinates of the feature point C are (cx, cy, cz). Note that ay = min_ry, by = max_ry, and az = bz = cz.

The normalized coordinate setting unit 70 sets a normalized coordinate system in the heart solid region 108 specified by the feature point / size detection unit 68. Specifically, a process of associating the coordinates of the rotating real coordinate system with the coordinates of the normalized coordinate system in the heart solid region 108 is performed. FIG. 16 shows a three-dimensional heart region 110 converted to normalized coordinates. The normalized coordinate system has a feature point C (the center point of the four-chamber cross-sectional image) in the rotating real coordinate system as the origin. That is, the coordinates (cx, cy, cz) in the rotating real coordinate system correspond to the coordinates (0, 0, 0) in the normalized coordinate system. Further, the coordinates (min_rx, min_ry, min_rz) in the rotating real coordinate system correspond to (−1, −1, −1) in the normalized coordinate system, and the coordinates (max_rx, max_ry, max_rz) in the rotating real coordinate system are normal. This corresponds to (1, 1, 1) in the generalized coordinate system. That is, both ends of each axis of the heart solid region 108 in the rotating real coordinate system correspond to −1 and 1 of each axis in the normalized coordinate system.

Here, the origin (that is, the feature point C) in the normalized coordinate system is not an accurate center of the heart solid region 110. FIG. 17A shows the position of the origin (feature point C) on the nxnz plane. As shown in FIG. 17A, regarding the nx axis, the feature point C (nx = 0) is not a midpoint between nx = 1 and nx = -1. The same applies to the ny axis. FIG. 17B shows the position of the origin (feature point C) on the nynz plane. As described above, in the rotating real coordinate system, the rz coordinate of the heart region surface 106 is not the midpoint between min_rz and max_rz. Therefore, as shown in FIG. (Nz = 0) is not a midpoint between nz = 1 and nz = -1.

That is, the normalized coordinate system has multiple scales. Specifically, each of the nx axis direction, the ny axis direction, and the nz axis direction has different scales on both sides of the origin. Although details will be described later, since the normalized coordinate system has a plurality of scales, it is possible to specify the actual data observation cross section after successfully absorbing the individual differences of the fetal heart.

The conversion function generation unit 72 as the correspondence generation unit is defined by a real coordinate system (see FIGS. 3 and 6) defined by the x-axis, y-axis, and z-axis, and the nx-axis, ny-axis, and nz-axis. A conversion function is generated as a correspondence relationship with the normalized coordinate system (see FIG. 16). In the present embodiment, the conversion function generation unit 72 includes a first conversion function that is a conversion function between a normalized coordinate system and a rotating real coordinate system (see FIG. 15 and the like), and a rotating real coordinate system and a real coordinate system. A second conversion function that is a conversion function between and is generated. The first conversion function is to convert coordinates in the normalized coordinate system into coordinates in the rotating real coordinate system, and the second conversion function is to convert coordinates in the rotating real coordinate system into coordinates in the real coordinate system. is there.

First, the first conversion function will be described with reference to FIG. The first conversion function is generated based on a correspondence relationship between the coordinates of the feature point in the rotating real coordinate system and the coordinates of the feature point in the normalized coordinate system. As shown in FIG. 18, in the normalized coordinate system, the three-dimensional heart region 110 is composed of a surface represented by nx = 0, a surface represented by ny = 0, and a surface represented by nz = 0. It is divided into two regions (R1 to R8 in FIG. 18). The eight areas are areas having different scales. In accordance with this, the conversion formulas for converting the coordinates of the normalized coordinate system into the actual rotation coordinate system are different in each region. In the present embodiment, the eight conversion expressions corresponding to the eight regions are collectively referred to as a first conversion function.

When the coordinates of the normalized coordinate system are (Xn, Yn, Zn) and the corresponding coordinates of the rotating real coordinate system are (Xr, Yr, Zr), the conversion formula in each region is expressed as follows.

Region R1 (nx ≧ 0, ny ≧ 0, nz ≧ 0)

Region R2 (nx <0, ny ≧ 0, nz ≧ 0)

Region R3 (nx ≧ 0, ny ≧ 0, nz <0)

Region R4 (nx <0, ny ≧ 0, nz <0)

Region R5 (nx ≧ 0, ny <0, nz ≧ 0)

Region R6 (nx <0, ny <0, nz ≧ 0)

Region R7 (nx ≧ 0, ny <0, nz <0)

Region R8 (nx <0, ny <0, nz <0)

Referring to FIG. 15 and FIG. 18, each conversion formula regarding Xr, Yr, and Zr will be described taking the region R1 as an example. First, a conversion formula from the nx coordinate to the rx coordinate will be described. The rx coordinate cx of the feature point C in the rotated actual coordinate system corresponds to the origin (0) of the nx coordinate in the normalized coordinate system, and the rx coordinate max_rx in the rotated actual coordinate system is the nx coordinate in the normalized coordinate system. Corresponds to 1. That is, the nx coordinate takes a value of 0 to 1 in the region R1 in the normalized coordinate system.

According to the first term on the right side of the expression indicating Xr in the region R1, the difference between max_rx and cx is calculated, and the difference is multiplied by the nx coordinate Xn of the normalized coordinate system. Thus, a value indicating the distance from rx = cx to rx = Xr in the rotating real coordinate system is calculated. The basic value of Xr is calculated by adding cx, which is the rx coordinate of the feature point C, to the value (second term on the right side).

A correction term (third term on the right side) that takes into account the gap or distortion of the septum is further added to the basic value calculated as described above. The septal shift or distortion is a shift or distortion with respect to the template data 64. This is caused by a positional shift of each part of the fetal heart such as the left ventricle, the left atrium, the right ventricle, and the right atrium, a size balance shift, and the like. In the fetal heart, the septal line may not be parallel to the ry axis of the rotating real coordinate system, that is, may be displaced from the slit 64d of the template data 64 due to individual differences. In such a case, the line connecting the feature points A, C, and B specified in the four-chamber cross-sectional image (see FIG. 15 and the like) in the rotating real coordinate system is not parallel to the ry axis, or the feature points A, C and B may not be on a straight line. In the expression indicating Xr in the region R1, the Xr coordinate is corrected based on the amount of deviation in the rx-axis direction between the feature point C and the feature point B in the actual rotation coordinate system.

FIG. 19 shows a diagram for explaining the correction term. In FIG. 19, it is a figure which shows the positional relationship of the feature points B and C in a rotation real coordinate system, and rx = max_rx. As shown in FIG. 19, the distance in the rx-axis direction between rx = max_rx and rx = cx that is the rx coordinate of the feature point C is represented by (max_rx−cx). Similarly, the distance in the rx axis direction between rx = max_rx and rx = bx which is the rx coordinate of the feature point B is represented by (max_rx−bx). Based on these, the distance (shift amount) in the rx axis direction between rx = bx that is the rx coordinate of the feature point B and rx = cx that is the rx coordinate of the feature point C is (max_rx−cx) − (max_rx). -Bx). The correction term represents a value obtained by multiplying the deviation amount between the feature point C and the feature point B in the rx-axis direction by Yn which is the ny coordinate of the normalized coordinate system. In the region R1, Yn takes a value from 0 to 1. For example, when Yn = 1, the deviation amount itself between the feature point C and the feature point B in the rx-axis direction is added as a correction term. Further, for example, when Yn = 0.5, half of the shift amount in the rx-axis direction between the feature point C and the feature point B is added as a correction term. Thus, the larger the amount of deviation between the feature point C and the feature point B in the rx-axis direction (that is, the greater the deviation of the septum), the more correction values are added to Xr according to the value of Yn. It will be. That is, Xr corresponding to the septal shift is calculated.

The conversion formula from the ny coordinate to the ry coordinate and the conversion formula from the nz coordinate to the rz coordinate have the same concept as that except for the correction term in the conversion formula from the nx coordinate to the rx coordinate. Is omitted. Further, the conversion formulas for the coordinates of the other regions (R2 to R8) can be described in the same manner as the conversion formulas for the region R1 described above, and thus the description thereof is omitted here. The correction term in the conversion formula from the nx coordinate to the rx coordinate in the regions R5 to R8 indicates correction of the Xr coordinate based on the inclination with respect to the ry axis of the straight line connecting the feature point A and the feature point C in the rotating real coordinate system. It is.

Each conversion formula corresponding to each area described above may be stored in the storage unit 22 in advance. The conversion function generation unit 72 fits each value of min_rx, max_rx, min_ry, max_ry, min_rz, max_rz, ax, bx, cx, cy, and cz detected by the feature point / size detection unit 68 to the conversion equation. Thus, the first conversion function is generated. min_rx is determined by the feature point F, max_rx is determined by the feature point G, min_ry and ax are determined by the feature point A, max_ry and bx are determined by the feature point B, min_rz is determined by the feature point H, and max_rz Is defined by a feature point I, and cx, cy, and cz are defined by a feature point C. Thus, it can be said that the first conversion function is generated by the feature point group detected in the heart stereoscopic image of the rotating real coordinate system.

According to the first transformation function, the normalized observation section defined in the normalized coordinate system (specifically, at least three normalized coordinates indicating the observation section) is the actual data rotation observation section (specifically, the actual data) in the rotation actual coordinate system. Three rotation real coordinates indicating the rotation observation section) are calculated.

As described above, the first conversion function is generated based on a plurality of feature points specified in the heart image of the fetus to be observed or diagnosed. That is, the first conversion function varies depending on the observation or diagnosis target. On the other hand, in the normalized coordinate system, since the heart image of the fetus is defined with a predetermined orientation or size, each observation cross section is uniquely specified. As described above, according to the present embodiment, each observation section can be uniquely specified in the normalized coordinate system, but the diagnosis target is calculated by the first conversion function calculated for each observation or diagnosis target. The individual differences are absorbed, and the real rotation data observation section can be specified in the real rotation coordinate system. Moreover, the normalized coordinate system has a plurality of scales. For example, in the present embodiment, different scales are provided in the normalization axis directions with respect to the center of the four-chamber cross section. Thereby, in the coordinate conversion by the first conversion function, conversion considering the balance of each part to be diagnosed is performed. That is, according to the present embodiment, a suitable rotation actual data observation cross section in which not only the individual difference in the outer shape of the diagnosis target but also the individual difference in the shape of each part included therein is absorbed is specified. Further, in the first conversion function, a correction term is added in consideration of a positional shift of each part to be diagnosed or a shape distortion. In the above-described example, a correction term is added in consideration of septal deviation or distortion. Therefore, according to the first transformation function, individual differences such as a positional shift of each part to be diagnosed or a size balance shift are eliminated, and a suitable rotation actual data observation section is specified.

Next, the second conversion function will be described. The second transformation function is a function that realizes the three-dimensional affine transformation in which the matching process (rotation process, enlargement / reduction process, and parallel movement process) described in FIG. 9 and the rotation process described in FIG. 10 are performed in the reverse direction. is there. That is, the second conversion function includes a conversion formula indicating a rotation process in which the rotation process for the matching process and the representative cross-section search is performed in the reverse direction, a conversion formula for performing the enlargement / reduction process in the matching process in the reverse direction, and the matching process. It is comprised by the conversion type | formula which performs the parallel movement process in reverse. By the second conversion function, the real rotation coordinate system is converted to the real coordinate system, that is, the real rotation data observation cross section (more specifically, the three real rotation coordinates indicating the real rotation data observation cross section) is the real data observation cross section (more specifically, the real data 3 real coordinates indicating the observation section).

First, the conversion formula shown in the rotation process in the second conversion function will be described. First, a process of performing each rotation process described in FIG. 10 in the reverse direction will be described. The rotation process is a process of rotating the same rotation axis by the same angle in the opposite direction on the same rotation axis. As described above, the rotation process in the matching process is represented by two quaternions Q and R. At the time of searching for the representative cross section, the representative cross section detecting unit 66 holds the parameters α, β, γ, and θ in the quaternions Q and R for each rotation (three types of rotations in the above description) in the storage unit 22. Therefore, the conversion function generation unit 72 generates each conversion expression indicating the reverse rotation of each rotation process using each parameter. Specifically, the parameters α, β, and γ of the quaternions Q and R described above are maintained as they are (that is, each rotation axis is the same as the representative section search process), and θ is set to −θ ( That is, each quaternion Q ′ and R ′ is generated (indicating rotation in the opposite direction to the representative cross-section search process). The second conversion function includes the quaternions Q ′ and R ′. Furthermore, the conversion equation indicating the rotation processing of the second conversion function includes a conversion equation for rotating the same rotation axis in the opposite direction by the same angle as the rotation processing in the matching processing described in FIG. Similarly, the conversion function generating unit 72 maintains the parameters α, β, and γ of the quaternions Q and R held by the adjusting unit 62 in the storage unit 22 during the matching process as they are (that is, rotation). The axes are the same as in the matching process), and quaternions Q ′ and R ′ are generated with θ being −θ (that is, indicating rotation in the opposite direction to the matching process). The second conversion function further includes the quaternions Q ′ and R ′.

In addition, the conversion expression indicating the enlargement / reduction process in the second conversion function is a conversion expression indicating a process for returning the enlargement / reduction process in the matching process, and this is held in the storage unit 22 by the adjustment unit 62. It is determined based on the enlargement / reduction ratio in the matching process. Furthermore, the conversion formula indicating the translation process in the second conversion function is a conversion formula indicating a process for returning the translation process in the matching process, and this is a matching process held in the storage unit 22 by the adjustment unit 62. Is determined on the basis of a vector indicating the parallel movement at.

The second conversion function is generated as described above. Similar to the first conversion function, the second conversion function also differs depending on the observation or diagnosis target.

By generating the first conversion function and the second conversion function, the coordinates of the real coordinate system corresponding to each coordinate in the normalized coordinate system can be calculated. That is, the normalized coordinate system is associated with the real coordinate system.

Based on the first conversion function and the second conversion function, the cross-section specifying unit 74 serving as the cross-section specifying means performs an actual observation in the real coordinate system from the normalized observation cross-section defined in the normalized coordinate system stored in the cross-section table 76. Identify the data observation cross section. The cross section table 76 defines one or more normalized observation cross sections corresponding to one or a plurality of observation cross sections. In the present embodiment, as described above, each normalized observation cross section is defined by three coordinates. For example, the normalized observation sections corresponding to the three blood vessel sections are (nx, ny, nz) = (− 1, 1, 0.72), (−1, −1, 0.72), and (1, 1 , 0.72). The three blood vessel cross section is a cross section obtained by translating the four-chamber cross section specified as the representative cross section, but the normalized observation cross section may not be a cross section parallel to the four-chamber cross section.

FIG. 20 shows a state in which the normalized observation section is converted to the actual data observation section. When a normalization observation cross section is selected, for example, when selected by the user, the cross section specifying unit 74 uses the first and second conversion functions generated by the conversion function generation unit 72 to select the normalization. Three coordinates in the normalized coordinate system defining the observation cross section are converted into three coordinates in the real coordinate system. For example, as shown in FIG. 19, (nx, ny, nz) = (− 1, 1, 0.72), (−1, −1, 0.72) indicating a cross section of three blood vessels in the normalized coordinate system. , And (1, 1, 0.72) are selected, the first transformation function converts the three coordinates to coordinates (rx1, rx2, rx3), (rx1, rx2) in the rotating real coordinate system. , Rx3) and (rx1, rx2, rx3). Thereby, in the rotation real coordinate system, the three blood vessel cross section 112 as the rotation real data whose direction and size are specified is specified. Further, by performing rotation, parallel movement, and enlargement / reduction processing on the three-vessel section 112 using the second transformation function, the three-vessel section 114 as the actual data observation section in the real coordinate system is specified. .

The cross-section position correcting unit 78 corrects the position of the actual data observation cross-section specified by the cross-section specifying unit 74 based on a user instruction. When the user confirms the tomographic image of the actual data observation cross section formed by the tomographic image forming unit 80 described later and displayed on the display unit 34, and determines that it is shifted from the target observation cross section, the operation unit (Not shown) can be used to instruct correction of the position of the actual data observation section. In response to an instruction from the user, the cross-section position correction unit 78 corrects the position of the actual data observation cross-section.

The tomographic image forming unit 80 as an image forming unit is specified by the cross-section specifying unit 74 or corrected by the cross-sectional position correcting unit 78 in each of a plurality of volume data 40 (see FIG. 3) stored in the data memory 26. A tomographic image is formed in the actual data observation section. Thereby, for example, when an actual data observation cross section of a three blood vessel cross section is specified, a plurality of tomographic images of the three blood vessel cross sections in each time phase are formed. The plurality of tomographic images generated in this way are processed by the image composition unit 32 and displayed on the display unit 34.

Hereinafter, the display mode of the tomographic image of the actual data observation cross-section specified by the cross-section specifying unit 74 will be described.

FIG. 21 shows a first display example of a tomographic image of an actual data observation cross section. In the first display example, the tomographic image 120 of the identified actual data observation cross section (four-chamber cross section in FIG. 21) is enlarged and displayed. Since the actual data observation section is specified in each of the plurality of volume data 40 arranged in the time direction, a moving image can be displayed as the tomographic image 120 by continuously switching the tomographic image corresponding to each volume data 40. . In the tomographic image 120, a label 120a indicating each part of the heart (for example, left ventricle, left atrium, right ventricle, right atrium) may be displayed. The display process of the label 120a will be described later.

Also, a guide image 122 indicating the position of the observation cross section is displayed together with the tomographic image 120. In the guide image 122, a three-dimensional model of the heart prepared in advance and a cross-sectional index 122a indicating the position of the actual data observation cross-section with respect to the heart are shown. The guide image 122 may be displayed based on a normalized coordinate system. For example, the position and orientation of the cross-sectional index 122a relative to the three-dimensional model of the heart may be determined based on the selected normalized observation cross section. By displaying the guide image 122, the user can preferably grasp which cross section the displayed tomographic image 120 corresponds to.

In addition, the guide image 122 includes a direction indicator 122b indicating the rotation direction of the tomographic image 120. In the present embodiment, the cross-section index 122a is shown as a rectangular surface, and the direction index 122b is attached to one corner of the surface. A direction indicator 120b is also attached to one corner of the rectangular frame of the tomographic image 120. The direction indicators 120b and 122b correspond to each other, and the user can grasp the rotation direction of the tomographic image 120 by checking the direction indicators 120b and 122b. The positions to which the direction indicators 120b and 122b are attached may be determined according to the rotation angle in the second conversion function.

FIG. 22 shows a second display example of the tomographic image of the actual data observation cross section. In the second display example, a plurality of actual data observation sections are specified in each volume data 40, and the tomographic image groups 124 corresponding to the specified plurality of actual data observation sections are displayed in parallel. Thus, in this embodiment, it is possible to select a plurality of normalized observation sections, and it is possible to simultaneously display actual data observation sections corresponding to the selected plurality of normalized observation sections. The guide image 126 is also displayed in the second display example. The guide image 126 shows a three-dimensional model of the heart and a plurality of cross-sectional indices 126a indicating the positions of a plurality of actual data observation cross sections. Note that correspondence between a plurality of tomographic images and a plurality of cross-sectional indices 126a may be shown. For example, the color of the frame of each tomographic image may correspond to the color of the cross-sectional index 126a.

Further, as shown in FIG. 23, a tomographic image group 128 corresponding to a plurality of observation cross sections (parallel multi-sections) parallel to the four-chamber cross section as a representative cross section may be displayed. The guide image 130 is also displayed in the parallel multi-section display, and a plurality of cross-sectional indices 130a indicating the positions of the four-chamber cross section and a plurality of observation cross sections parallel thereto are displayed. Further, as shown in FIG. 24, a tomographic image group 132 corresponding to a four-chamber cross section as a representative cross section and two observation cross sections (orthogonal cross sections) orthogonal thereto may be displayed. The guide image 134 is also shown in the orthogonal cross-section display, and a plurality of cross-sectional indices 134a indicating the positions of the four-chamber cross section and two observation cross sections orthogonal to the four-chamber cross section are displayed.

Hereinafter, label display processing by the label processing unit 82 and the image composition unit 32 will be described.

In FIG. 25, a plurality of label space regions (140a-b (hereinafter collectively referred to as a label space region 140) defined in the heart solid region 110 of the normalized coordinate system may be described. Is)) is shown. Each label space region 140 is a three-dimensional region and corresponds to each part of the fetal heart. In the example of FIG. 24, the label space region 140a corresponds to the left ventricle, the label space region 140b corresponds to the right ventricle, the label space region 140c corresponds to the left atrium, and the label space Region 140d corresponds to the right atrium.

As described above, since the orientation and size of the fetal heart are defined in the normalized coordinate system, the position of the label space region 140 corresponding to each part can be defined in the normalized coordinate system. The label space area 140 may have a predetermined shape. However, as described later, the label space area 140 corresponds to a heart stereoscopic image (see FIG. 16) converted into a normalized coordinate system (that is, for each subject). The shape may be adjusted accordingly.

Each label space area 140 has label information indicating each part of the corresponding heart. For example, the label space region 140a corresponding to the left ventricle includes “LV” character data indicating the left ventricle as label information, and the label space region 140b corresponding to the right ventricle includes the character “RV” indicating the right ventricle. Data is included as label information.

For example, when a normalized observation section is selected by the user, a normalized observation section 142 is defined in the three-dimensional heart region 110 of the normalized coordinate system as shown in FIG. When the defined normalized observation cross section 142 crosses the label space area 140, a label plane area 144 that is an area intersecting with the label space area 140 is generated on the normalized observation cross section 142. In the example shown in FIG. 26, since the normalized observation cross section 142 crosses the label space area 140a and the label space area 140b, the label plane area 144a corresponding to the label space area 140a on the normalized observation cross section 142, In addition, a label plane area 144b corresponding to the label space area 140b is generated.

When the normalized observation section 142 crosses the label space area 140, that is, when the label plane area 144 is generated on the normalized observation section 142, the label processing unit 82 centroid of the label plane area 144 in the normalized coordinate system. The coordinates 146 are calculated. For the calculation process of the center of gravity, a known image processing technique can be used. In the example shown in FIG. 26, the barycentric coordinates 146a of the label plane area 144a and the barycentric coordinates 146b of the label plane area 144b are calculated.

When the barycentric coordinates 146 are calculated on the normalized observation cross section 142, the label processing unit 82 uses the first conversion function and the second conversion function to convert the barycentric coordinates 146 in the normalized coordinate system into the barycentric coordinates in the real coordinate system. Convert to Then, the image compositing unit 32 uses a position based on the converted barycentric coordinates of the real coordinate system in a tomographic image corresponding to the real data observation cross section (the barycentric coordinates of the real coordinate system are located on the real data observation cross section) Display the label information.

FIG. 27A shows a tomographic image 148 displayed on the display unit 34. As shown in FIG. 27A, the label information 152a is displayed around the barycentric coordinate 150a in the real coordinate system corresponding to the barycentric coordinate 146a (see FIG. 26) in the normalized coordinate system. In FIG. 27A, the barycentric coordinates 150a are indicated by black dots, but the barycentric coordinates 150a may not be displayed. In the present embodiment, the letters “LV” indicating the left ventricle are displayed as the label information 152a. Similarly, the label information 152b is displayed around the barycentric coordinate 150b in the real coordinate system corresponding to the barycentric coordinate 146b in the normalized coordinate system. In the present embodiment, the characters “RV” indicating the right ventricle are displayed as the label information 152b.

FIG. 27B shows another display example of the label information 152. The label information 152 may be displayed in the outer region of the tomographic image 148 as shown in FIG. 27B. Thereby, it is possible to display the label information while suppressing a decrease in the visibility of the tomographic image. In order to clarify which part each label information 152 corresponds to, the leader line is drawn from the barycentric coordinates 150, and the label information 152 corresponding to the tip of the leader line is displayed. Also good.

Hereinafter, processing for adjusting the shape of the label space area 140 will be described. In the present embodiment, the label processing unit 82 adjusts the shape of the label space region 140 by a three-dimensional region growing process based on a heart stereoscopic image in a normalized coordinate system. FIG. 28 shows the state of the three-dimensional region growing process. In FIG. 28, one cross section in the heart stereo image included in the heart stereo region 110 is shown.

The three-dimensional region growing process is a process for expanding or reducing each label space region 140 so as to be adapted to each part of the heart stereoscopic image. In the example of FIG. 28, the label space region 140a corresponding to the left ventricle is expanded so as to fit the left ventricle region of the cardiac stereoscopic image, and the adjusted label space region 160a is formed. The same applies to each label space region 140 corresponding to the right ventricle, the left atrium, and the right atrium.

Hereinafter, the three-dimensional region growing process will be described with reference to FIG. 29, taking the label space region 140a corresponding to the left ventricle as an example. First, “1” is set as a data value in each voxel constituting the label space area 140a. The label processing unit 82 calculates the product of the data value of each voxel constituting the label space region 140a and the luminance value of the inverted binarized heart stereoscopic image at the coordinates corresponding to each voxel. Then, the voxel whose calculation result is “0” is excluded from the label space area 140a. In the three-dimensional heart image, the luminance of the heart chamber is “1” and the luminance values of the other positions are “0”. A label space region 162a is generated in which a portion outside the (left ventricle) is excluded (see the left diagram in FIG. 29).

Next, the label processing unit 82 calculates the product of the voxel adjacent to the label space region 162a and the luminance value of the inverted binarized heart stereoscopic image at the coordinates corresponding to the voxel. If the calculation result is “1”, the adjacent voxel is added to the label space area 162a. When the calculation result is “0”, the voxel is not added to the label space area 162a. By repeating this process until the shape of the label space region 162a does not change, an adjusted label space region 160a suitable for the heart chamber (left ventricle) is formed. It should be noted that in the three-dimensional heart region, a region (processing boundary) that can be taken by the left ventricle is determined in advance, and the expansion processing may be performed within the processing boundary.

For example, when the three-dimensional region growing process is performed on the label space region 140 corresponding to the site in the heart chamber such as the left ventricle, the processing of the valve part becomes a problem. On the other hand, since the valve portion may not be closed in the three-dimensional heart image, the label space region 160 corresponding to a certain portion is adjacent via the valve portion by the three-dimensional region growing process. It may expand to other parts. For example, the adjusted label space region 160a corresponding to the left ventricle may expand to the left atrial region.

Therefore, in the present embodiment, the label processing unit 82 generates a boundary surface of the three-dimensional region glowing process at the valve portion of the heart stereoscopic image with the annulus portion (the base portion of the valve) as a reference. FIG. 30 shows the state of the boundary surface generation process in the valve portion.

First, the label processing unit 82 detects a feature point 170 indicating the annulus. The feature point 170 may be detected, for example, by a technique similar to the detection process of the feature points D to G shown in FIG. 11 on the four-chamber cross section of the heart stereoscopic image. Here, it is assumed that feature points 170a and 170b indicating the two annulus portions of the mitral valve are detected (see the upper left diagram in FIG. 30). When the two feature points 170a and 170b are detected, the label processing unit 82 generates a straight line 172 that connects the two feature points (see the central diagram in the upper part of FIG. 30).

Next, the label processing unit 82 performs an edge search process in the direction orthogonal to the straight line 172 on both sides of the straight line 172 (see the upper right diagram in FIG. 30). By the edge search process, a plurality of high-luminance pixels 174 corresponding to valves are detected on one side of the straight line 172 (see the left diagram in the lower part of FIG. 30). Further, the label processing unit 82 generates an approximate curve (secondary curve) 176 based on the plurality of high-luminance pixels 174 (see the lower center diagram in FIG. 30). Finally, the label processing unit 82 rotates the generated approximate curve 176 around a rotation axis that passes through the midpoint of the straight line 172 and is orthogonal to the straight line 172 (see the lower right diagram in FIG. 30). Thereby, in the three-dimensional heart region 110, a boundary surface is formed as a processing boundary of the three-dimensional region growing process between the left ventricle and the left atrium.

According to the present embodiment described above, based on the fetal heart stereo image included in the volume data obtained by transmitting and receiving ultrasonic waves, the real coordinate system of the volume data and the heart of a predetermined size and orientation First and second conversion functions are generated that indicate a correspondence relationship with a normalized coordinate system in which a stereoscopic image is defined. Thereby, the first and second conversion relations are generated for each fetal heart, that is, for each target tissue. And based on the 1st and 2nd conversion function, the real data observation cross section in a real coordinate system is specified from the normalization observation cross section previously defined in the normalization coordinate system. Through the above processing, the actual data observation cross section can be identified more preferably when the individual difference of the fetal heart is absorbed.

Further, according to the present embodiment, it is possible to specify the prescribed orientation (posture) of the fetal heart image in the volume data. In the embodiment described above, the left-right direction (apical portion side) and the up-down direction (head side, stomach side) of the fetal heart are specified. Therefore, for example, when a three-dimensional heart image included in the volume data is displayed in a three-dimensional manner, a three-dimensional heart image can be displayed. Alternatively, when displaying a tomographic image cut out from volume data, a cardiac tomographic image oriented in a predetermined direction can be displayed. As a result, the user can more easily observe or diagnose the fetal heart image using the volume data.

As mentioned above, although embodiment which concerns on this invention was described, this invention is not limited to the said embodiment, A various change is possible unless it deviates from the meaning of this invention.

10 ultrasonic diagnostic equipment, 12 probes, 14 transmission / reception unit, 16 phasing addition unit, 18 beam processing unit, 20 DSC, 22 storage unit, 24 pre-memory, 26 data memory, 28 reconfiguration processing unit, 30 volume data processing unit , 32 Image composition unit, 34 Display unit, 36 Control unit, 60 Filter unit, 62 Adjustment unit, 64 Template data, 66 Representative section detection unit, 68 Feature point / size detection unit, 70 Normalized coordinate setting unit, 72 Conversion function Generation unit, 74 cross-section specifying unit, 76 cross-section table, 78 cross-section position correcting unit, 80 tomographic image forming unit, 82 label processing unit.

Claims (9)

1. Correspondence generation means for calculating the correspondence between the actual coordinate system of the volume data and the calculated normalized coordinate system based on the target tissue image included in the volume data obtained by transmitting and receiving ultrasonic waves;
   Based on the correspondence relationship, from a normalized observation cross section defined in the normalized coordinate system, a cross section specifying means for specifying an actual data observation cross section in the real coordinate system;
   An image forming means for forming a tomographic image corresponding to the specified actual data observation section from the volume data;
   An ultrasonic image processing apparatus comprising:
2. The correspondence generation unit calculates the correspondence based on a representative point group detected from the target tissue image.
   The ultrasonic image processing apparatus according to claim 1.
3. The representative point group includes a reference point of the target tissue image, and both end points of the target tissue image in each coordinate axis of the real coordinate system,
   The normalized coordinate system is defined based on the reference point and the end points.
   The ultrasonic image processing apparatus according to claim 2.
4. The normalized coordinate system has at least two different scales;
   The ultrasonic image processing apparatus according to claim 1.
5. A plurality of the normalized observation cross sections are defined,
   The cross-section specifying means specifies the actual data observation cross section corresponding to the selected normalized observation cross section based on the selected normalization observation cross section selected from the plurality of normalization observation cross sections.
   The ultrasonic image processing apparatus according to claim 1.
6. Representative section specifying means for specifying a representative section in the volume data;
   Further including
   The correspondence generation means calculates the correspondence based on the relationship between the coordinates in the real coordinate system and the coordinates in the normalized coordinate system of a plurality of representative points detected in the representative section.
   The ultrasonic image processing apparatus according to claim 2.
7. The representative section specifying means is
   Posture defining means for defining the posture of the volume data so that the target tissue image included in the temporary representative cross section specified in the volume data matches the template data;
   In the volume data of the posture defined by the posture defining means, representative cross-section search means for searching for the representative cross-section in the vicinity range of the temporary representative cross-section,
   including,
   The ultrasonic image processing apparatus according to claim 6.
8. A display unit for displaying the tomographic image;
   Further comprising
   In the normalized coordinate system, a label space region corresponding to each part of the target tissue image and including label information indicating each part is defined,
   When the normalized observation cross section crosses the label space region, the display unit displays label information corresponding to the label space region crossed by the normalized observation cross section together with a tomographic image corresponding to the normalized observation cross section. indicate,
   The ultrasonic image processing apparatus according to claim 1.
9. Computer
   Correspondence generation means for calculating the correspondence between the actual coordinate system of the volume data and the calculated normalized coordinate system based on the target tissue image included in the volume data obtained by transmitting and receiving ultrasonic waves;
   Based on the correspondence relationship, from a normalized observation cross section defined in the normalized coordinate system, a cross section specifying means for specifying an actual data observation cross section in the real coordinate system;
   An image forming means for forming a tomographic image corresponding to the specified actual data observation section from the volume data;
   An ultrasonic image processing program which is made to function as:

Images