JP2001104312A - Stereoscopic panorama image compositing device for ultrasonic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic panorama image compositing device of low cost and excellent operability capable of generating stereoscopic panorama images of ultrasonic images at real time. SOLUTION: The device is provided with a probe to obtain ultrasonic data in two directions perpendicular to each other, an ultrasonic/imaging means to convert the ultrasonic data obtained from the probe into images, a moving image display means to display the images obtained by the ultrasonic/imaging means as moving images, a composite parameter detecting means to detect motion quantity and rotation quantity between images in specific directions generated by moving operation to the probe, and a stereoscopic panorama image compositing means to dispose an image in one more direction in a three- dimensional space in accordance with the motion quantity and rotation quantity detected by the composite parameter detecting means, and synthesize them as a stereoscopic panorama image to be displayed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for synthesizing a stereoscopic panoramic image of an ultrasonic image, and more particularly to a real-time three-dimensional stereoscopic panoramic image display from a sequence of tomographic images photographed by a manual probe operation. The present invention relates to an apparatus for synthesizing a stereoscopic panoramic image of an ultrasonic image.

[0002]

2. Description of the Related Art In recent years, diagnosis using ultrasonic waves has been widely used. An ultrasonic diagnostic apparatus applies a probe to the body surface,
This is a device that displays tomographic images of organs and the like in a patient's body in real time by transmitting ultrasonic waves of 10 to 10 MHz into the body and receiving ultrasonic waves reflected on the surface of the organs. Diagnosis using ultrasound is safer and does not cause pain or adverse effects on patients as compared with diagnosis by surgery or X-ray. This is particularly important in the diagnosis of pregnant women and infants.

[0003] Typical measurement applications of an ultrasonic diagnostic apparatus include fetal growth measurement and circulatory organ measurement. Since these measurement sites exceed the range in which a probe can display a tomographic image at a time, a function of displaying a tomographic image in a three-dimensional and panoramic manner while scanning the probe along a human body is required. A single capture can only provide a very narrow two-dimensional cross-sectional image, but if the probe is attached to a mechanical scanner with a position sensor and the scanner is moved across the human body and the cross-sectional image is stored, When the user has finished moving up, an entire stereoscopic panoramic image can be generated and displayed from all the stored cross-sectional images.

By the way, ultrasonic waves are transmitted into the body through a layer of jelly-like substance applied to the skin, and a probe receiving the reflected waves must be held in direct contact with the skin of the human body. No. For this reason, a sophisticated operation technique is required to accurately scan a probe attached to a mechanical scanner along a curved surface of a human body having an uneven surface such as a calf or a thyroid.

[0005] It is desirable to realize an apparatus capable of displaying a cross-sectional image three-dimensionally in real time while manually scanning a probe along the surface of a human body. In order to combine images without attaching a position sensor or a rotation sensor, it is necessary to detect a movement amount and a rotation amount from the image. However, in order to obtain a three-dimensional image, the tomographic image sequence captured while shifting the probe in a direction orthogonal to the tomogram does not include information on the amount of movement or rotation of the probe, and is calculated by correlation calculation between images. It is currently difficult to detect the above information.

[0006]

As described above, the conventional ultrasonic diagnostic apparatus requires a mechanical scanner having a position sensor in order to display a stereoscopic panoramic image. There was a problem.

An object of the present invention is to provide an ultrasonic image stereoscopic panoramic image synthesizing apparatus which is capable of generating a stereoscopic panoramic image of an ultrasonic image in real time and which is low in cost and excellent in operability.

[0008]

In order to solve the above-mentioned problems, a probe for acquiring ultrasonic data from two orthogonal directions and an ultrasonic / imaging means for converting the ultrasonic data acquired from the probe into an image. Moving image display means for displaying an image obtained by the ultrasonic / imaging means as a moving image; and synthetic parameter detection for detecting a moving amount and a rotating amount between images in a specific direction generated by a moving operation of the probe. Means, and a stereoscopic panoramic image synthesizing means for arranging an image in the other direction in a three-dimensional space, synthesizing it as a stereoscopic panoramic image and displaying the image in accordance with the movement amount and the rotation amount detected by the synthesizing parameter detecting means Have. The ultrasonic receiving portion of the probe has a shape including a right angle component, such as an L-shaped or T-shaped.

The composite parameter detecting means includes a projection distribution storing means for storing a projection distribution between continuous images, and a projection distribution correlation for obtaining a movement amount by a correlation between the projection distributions for each projection distribution between the continuous images. Means, and the amount of movement between the projection distributions is defined as the amount of movement of the image by the operation of moving the probe. Then, the projection distribution stored in the projection distribution storage means is normalized by the number obtained by integrating the result of integrating the values of the pixels constituting the image in the vertical direction. Further, in the creation of the projection distribution, when the pixel has a value in a specific range, the pixel is regarded as an effective pixel, and only the effective pixel is set as a calculation target of the projection distribution.

Further, the synthesis parameter detecting means sets a plurality of image processing areas, and stores projection distribution between continuous images by the number of the plurality of image processing areas. Projection distribution correlating means for respectively obtaining a movement amount by a correlation between the projection distributions for the corresponding projection distributions of the corresponding image processing regions, and a difference between the plurality of movement amounts, and Estimate the amount of rotation. Then, the projection distribution stored in the projection distribution storage means is normalized by the number obtained by integrating the result of integrating the values of the pixels constituting the image in the vertical direction. Further, in the creation of the projection distribution, when the pixel has a value in a specific range, the pixel is regarded as an effective pixel, and only the effective pixel is set as a calculation target of the projection distribution. In addition, the amount of rotation is estimated by using the amount of movement obtained from the set of image processing regions that are farthest apart.

In another means of the present invention, two orthogonal
A probe that acquires ultrasonic data from a direction, an ultrasonic / imaging unit that converts the ultrasonic data acquired from the probe into an image, and a moving image that displays the image obtained by the ultrasonic / imaging unit as a moving image Display means, a combined parameter detecting means for detecting a moving amount and a rotating amount between images in a specific direction generated by a moving operation of the probe, and a moving amount and a rotating amount detected by the combining parameter detecting means, The images in the above directions are arranged on a two-dimensional plane, synthesized as a panoramic image, and displayed in a panoramic image synthesizing unit. A stereoscopic panoramic image synthesizing means for arranging in a three-dimensional space, synthesizing as a stereoscopic panoramic image, and displaying the stereoscopic panoramic image is provided.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is an example of a system configuration diagram of an apparatus for synthesizing a stereoscopic panoramic image of an ultrasonic image for realizing the present invention. Basically, it is the same as the system configuration of a digital computer that is currently widely used,
In addition to the above, there are provided an ultrasonic oscillator 100 for generating ultrasonic waves of a predetermined frequency, an ultrasonic / imaging unit 120 for receiving ultrasonic echoes and converting them into images.

First, reference numeral 110 denotes an ultrasonic wave 210 having a predetermined frequency.
This is an ultrasonic oscillator that generates -1. Generates 2 to 10 megahertz ultrasound that is inaudible to the human ear. The frequency to be generated depends on an instruction from the CPU 130. An ultrasonic wave 120 converts an ultrasonic echo signal 210-2 received by the probe 200 into a digital image.
An imaging unit.

The converted digital image is taken into the video memory 155 at the same time as being taken into the memory 140.
The video memory 155 stores an image displayed on the display 150 as digital data. Reference numeral 156 denotes a D / A converter of a type generally called a RAMDAC. The D / A converter 156 sequentially reads data written in the video memory 155 in accordance with the scanning line speed, and displays the data on the display 15.
Draw at 0. Therefore, when the data in the video memory 155 is updated, the updated content is immediately reflected on the content displayed on the display 150.

The display 150 is a device for displaying an image, and may be, for example, a small CRT or a plasma display, or may be a liquid crystal type display device. The capture of these images
By repeating at a frequency of about 30 times per second, a continuous moving image can be displayed on the display.

The auxiliary storage device 110 is a large-capacity recording device such as a hard disk, and is a device for semi-permanently recording digital data. This can be removed from the main unit with the storage device like a PCMCIA hard disk card, or like a magneto-optical disk, etc.
A recording device of a type in which only the recording medium can be attached and detached may be used.

The CPU 130 executes software programs for realizing the functions described in the present invention, including control for diagnosis. The program 140-1 is resident in the memory 140, and data 140-2 necessary for executing the program is also stored in the memory 140 as needed. The printer 180 prints the processed image. The keyboard 160 and the pointing device 170 are information input devices. The input information is transmitted to the CPU 130 and is appropriately processed. A data bus 132 interconnects the devices described above.

In the system configuration of the ultrasonic image stereoscopic panoramic image synthesizing apparatus described above, in this embodiment, an image is synthesized from the captured image sequence in accordance with the scanning operation of the probe. First, according to the control program stored in the memory 140, the images captured during the diagnosis are captured in the memory, and the images are combined in a three-dimensional space, sequentially transferred to the video memory 155, and the stereoscopic panoramic image 151 is displayed on the display 150. I do. When the diagnosis is completed, the stereoscopic panoramic image is stored in the auxiliary storage device 190 as an image data structure 190-1. Reference numeral 190-1-1 denotes image header information, which includes information such as the type of the image, the size of the image, the date and time of diagnosis, and the site of diagnosis. Reference numeral 190-1-2 denotes image data, which is stored uncompressed or compressed.

FIG. 2 shows an example of application of the device of the present invention to a human body. The L-shaped probe 200 can acquire a cross-sectional image in the XZ axis and a cross-sectional image in the YZ axis at the same time.
The XZ-axis cross-sectional images obtained by manually scanning the probe in the X-axis direction along the body surface 300 are characterized in that most temporally adjacent images coincide with each other. However, as shown in FIG. 5, the position is shifted between the images according to the moving distance in the scanning direction. In addition, a rotational displacement occurs between the images according to the curvature of the body surface 300. Therefore, in the ultrasonic image stereoscopic panoramic image synthesizing apparatus 100 of the present invention, the amount of movement and the amount of rotation between images are calculated by image processing to obtain synthesis parameters. Using the synthesis parameters, the cross-sectional images in the YZ axes are arranged in a three-dimensional space, synthesized as a stereoscopic panoramic image, and the images are sequentially displayed on the display 150. This makes it possible to provide a synthesizing device that is easy to operate without using a position sensor. The conventional probe can obtain only one cross-sectional image at a time, but the L-shaped probe of the present invention can simultaneously obtain two cross-sectional images at the same time. There is an effect that it can be used as a supplement.

FIG. 14 shows another embodiment of the probe of the present invention, which is characterized in that it is a T-shaped probe.
Since the probe of this embodiment has a symmetrical surface in contact with the body surface, it has an effect that it is easy to scan when brought into close contact with the body surface, and it becomes easy to accurately receive ultrasonic images from two directions. .

FIG. 3 shows an embodiment of a display screen displayed on the display of the present invention. The display screen includes a moving image display window 301 and an operation instruction window 330.

The moving image display window 301 is a window for displaying a tomographic image under diagnosis in real time. Second 3
By displaying an image at a frequency of 0 times, the state inside the body can be observed as a moving image. 302 represents X- obtained by the present invention.
Reference numeral 303 denotes a cross-sectional image display area on the Z axis, and reference numeral 303 denotes a cross-sectional image display area on the YZ axis. Reference numeral 312 denotes a processing area for detecting a movement amount.

The operation instruction window 330 is a window for instructing an operation necessary for diagnosis, and includes, for example, a three-dimensional panorama button 331, a reset button 334, and a save button 3
36, and a print button 337. Selection of these buttons can be achieved by operating the pointer 339 with the pointing device 170 and clicking the pointing device 170 on a desired button. In addition, by attaching a touch panel on the display 150, directly,
A desired button can be pressed with a finger.

First, the diagnosis by the ultrasonic wave is started by pressing the stereoscopic panorama button 331. When the stereoscopic panorama button 331 is pressed, the display changes to the display screen of FIG. A panorama image display window 310 and a stereoscopic panorama image display window 320 appear, and the display screen is first cleared.
From this state, when the body surface is scanned by the probe, the panoramic image 311 and the stereoscopic panoramic image 321 are sequentially synthesized and displayed. FIG. 4 is an example of a finally obtained stereoscopic panoramic image. If the reset button 334 is pressed during the creation of the stereoscopic panoramic image, the state becomes the initial state of the diagnosis, and the stereoscopic panoramic image is cleared. In addition, results can be easily stored and printed. When the save button 336 is pressed, the panoramic image and the stereoscopic panoramic image are stored in the auxiliary storage device 190. By pressing the print button 337, a panoramic image and a stereoscopic panoramic image can be printed by the printer 180.

FIG. 5 shows a geometric relationship between images obtained by scanning the L-shaped probe on the body surface in the X-axis direction. 300 is a body surface, and 505 is X- acquired at time t.
The Z-axis tomographic image It, 506 is the XZ-axis tomographic image It + 1 acquired at time t + 1. 507 is a YZ axis tomographic image Jt acquired at time t, and 508 is a YZ axis tomographic image Jt + 1 acquired at time t + 1.
It is.

Since the body surface 300 is a curved surface, the image It and the image It + 1 have a geometrical relationship between movement and rotation as shown in the drawing, that is, the origin is at the center O of the image It, and the horizontal axis of the image It is Is a coordinate system with the X axis and the axis orthogonal to the Y axis, the image It + 1 moves dx along the X axis and rotates dΘ around the origin O. When the dx and dΘ are known, the geometric relationship between the images can be described by the following equation (1).

[0027]

P ′ = Ap + B (1) In Equation 1, A is a rotation matrix, B is a shift amount, p is coordinates of the image It, and p ′ is coordinates of the image It + 1. Then, since this geometric relationship is known, the image Jt can be accurately arranged in the three-dimensional space.

FIG. 6 shows the principle of movement amount detection according to the present invention.
The detection of the movement amount is performed by calculating the projection distribution of the image It and the image It + 1 in the vertical direction, and then calculating the correlation between the projection distributions. The position at which the correlation value becomes minimum while shifting the position in the horizontal direction is defined as the movement amount dx. I do. The projection distribution is calculated only in the image processing area 312.

The correlation value is the sum of absolute differences between the values of each element between the projection distributions, as shown in the equation in FIG. In the present invention, a global feature amount called a projection distribution is used for calculating the amount of movement of an image, so that the amount of movement can be obtained stably and the amount of calculation is extremely small, so that the amount of movement can be detected in real time with normal CPU capability. This has the effect.

FIG. 7 shows the principle of rotation amount detection according to the present invention.
In the case of detecting the movement amount, it is sufficient to set one image processing area. However, in the case of detecting the rotation amount, one image processing area is not sufficient, and it is necessary to set two or more. FIG. 7 shows an example in which six image processing areas 700 are aligned and set in the vertical direction. Here, IDs 1 to 6 are assigned to the image processing area from below.
A number (not shown) is given. With respect to such an image processing area, a projection distribution is obtained for each area, and each movement amount dxi is calculated.
Is calculated. If there is rotation between images, each movement amount increases or decreases monotonically in order of the ID number in principle.

Therefore, the amount of rotation dΘ is calculated from the difference in the amount of movement between the farthest regions by the formula shown in FIG. Since the image processing area used for the movement amount detection here is set locally, the reliability of the movement amount detection is inferior to the movement amount detection shown in FIG. For example, when scanning a probe, if the user encounters a bone or the like, the ultrasonic beam does not pass through the body any more, so that no echo returns from a location after that. For this reason, the obtained tomographic image becomes dark in places after the bone. If the darkened portion largely covers each image processing area, the moving amount cannot be obtained correctly.

Therefore, in the present invention, first, pixels are evaluated for each area, and if there are many dark pixels, the movement amount is not detected. Then, in order to determine the reliability of the detected movement amount, the rotation amount is calculated only when the monotonicity of the movement amount is checked in the order of ID number and the monotonicity is confirmed. At this time, the calculation of the amount of rotation uses a difference value of the amount of movement between the most distant image processing regions for which the amount of movement is determined. In the present invention, a large number of image processing regions are set for calculating the amount of rotation of an image, and even if a situation where the amount of movement cannot be detected partially occurs, the amount of rotation is calculated from the amount of movement obtained from the remaining image processing regions. Because
Increases reliability. In this embodiment, six image processing areas are set. For example, even if four of the six image processing areas cannot be detected, the rotation amount can be detected from the remaining two, so that the detection rate is high. Also, since the amount of calculation is extremely small, the usual C
There is an effect that the rotation amount can be detected in real time by the PU capability.

FIG. 8 shows an example of the structure of a tomographic image by ultrasonic waves. The top of the tomographic image 505 is the body surface side. The color part 801 is a part where the blood flow is measured in the blood vessel part by the color Doppler method, and is specially colored. A gray portion 802 is a portion where the ultrasonic echo returns. The dark portion 803 is the portion where the ultrasonic echo did not return,
When irradiating ultrasonic waves, a dark part is generated when there are bones and cavities on the way. In the present invention, when detecting the amount of movement or the amount of rotation, processing is performed by regarding only the gray portion 802 as an effective pixel, thereby preventing deterioration in detection accuracy.

FIG. 9 shows an embodiment of the control program 140-1 for the stereoscopic panoramic image synthesizing apparatus of the present invention.
The flowchart mainly shows the panoramic image generation processing during diagnosis. The control program 140-1 is executed with reference to the control data 140-2 shown in FIGS.

In FIG. 9, step 900 is an initialization process, in which an initial screen of the stereoscopic panoramic image synthesizing apparatus is displayed on a display. Then, in step 905, the variable
Reset status to 0. While the power of the stereoscopic panoramic image synthesizing device is turned on (step 910), the following stereoscopic panoramic image generation processing is performed.

First, in step 915, an ultrasonic image is fetched and stored in the array Input_Image_Buf 140-2-1. Next, the ultrasonic image captured in step 920 is displayed in the moving image display window 301. Step 925
Then, it is checked whether or not the stereoscopic panorama button 331 has been pressed, and the determination processing in step 930 described below is performed.

If the status is 0 and the stereoscopic panorama button is pressed, start processing 931 of stereoscopic panoramic image generation
Execute Here, the panorama image display window 3
10 and the previous image displayed in the stereoscopic panoramic image display window 320 is deleted to prepare for displaying a new panoramic image and stereoscopic panoramic image. The details of this process will be described later with reference to FIG. Next, at step 932, the variable status is set to 1 to make a state in which a stereoscopic panoramic image is being generated.

If the variable status is 1 and the stereoscopic panorama button is pressed, step 932 is executed and the variable stat
“us” is set to “0” to end the stereoscopic panoramic image generation.

Next, in the determination processing of step 940, it is determined whether or not the reset button 334 has been pressed. If the reset button 334 has been pressed, the processing for starting the stereoscopic panorama image generation already described in step 941 is executed. And the variable st
Atus is set to 1 so that a panoramic image is being generated.

Then, in step 950, it is determined whether the state is that a stereoscopic panoramic image is being generated. If the variable st
If atus = 1, a 3D panoramic image is being generated, so
In step 951, a stereoscopic panoramic image generation process is performed, and the result is referred to as Panoramic_Image_Buf140-2-2 and Cubic_I.
This is stored in mage_Buf 140-2-12. The details of this process will be described later with reference to FIG. And Panoramic_Im
age_Buf140-2-2 and Cubic_Image_Buf140-
Each image stored in 2-12 is displayed on a panoramic image display window 310 and a stereoscopic panoramic image display window 320.

Next, at step 960, a save button 336
Is determined, and if so, step 961 is executed. Step 961 is a stereoscopic panoramic image storage process, and Panoramic_Image_Buf140-2-2.
The panoramic image and the stereoscopic panoramic image generated so far and stored in Cubic_Image_Buf 140-2-12 are stored in the auxiliary storage device 190.

Next, at step 970, the print button 337
Is determined, and if so, step 971 is executed. Step 971 is a panoramic image printing process, in which each image generated so far and stored in Panoramic_Image_Buf 140-2-2 and Cubic_Image_Buf 140-2-12 is output to the printer 180.

The above steps 915 to 970
The stereoscopic panoramic image of the ultrasonic image can be synthesized by repeating the processing of.

FIG. 12 shows details of the start processing 931 of the stereoscopic panoramic image generation. First, in step 1000, a variable k for counting the total number of images that have contributed to the generation of a stereoscopic panoramic image is reset to zero. Next, in step 1010, arrays Total_dx140-2-9 and Total_dΘ for storing movement and rotation parameters used for generating a stereoscopic panoramic image.
The head element of 140-2-10 (FIG. 11) is reset to 0. Then, in step 1020, the stereoscopic panoramic image 32 displayed on the stereoscopic panoramic image display window 320
Erase 1

Thereafter, the image of the partial area 302 of the image input in step 1030 is written to the array Panoramic_Image_Buf 140-2-2 (FIG. 10) for storing the panoramic image. This first image is a reference image for panorama creation, and the movement parameters and rotation parameters of the subsequent images are relative differences from this image. And step 10
At 40, Panoramic_ is displayed in the panorama image display window 310.
Display the contents of Image_Buf.

Next, the image of the partial area 303 of the image input in step 1050 is written into the array Cubic_Image_Buf140-2-12 (FIG. 11) for storing the stereoscopic panoramic image. Then, in step 1060, the contents of Cubic_Image_Buf are displayed in the stereoscopic panoramic image display window 320.

FIG. 13 shows details of the stereoscopic panoramic image generation processing 951 of the present invention.

Steps 1100 to 1120 are processing for obtaining the movement amount dx between successive images.

First, in step 1100, Input_Image_Buf
The vertical projection distribution of the designated area 312 in the sectional image of the partial area 302 stored in 140-2-1 is calculated and stored in the array X_Proj_Current 140-2-3. here,
Input_Image_Buf stores cross-sectional images in two directions as one frame.

Next, at step 1105, it is determined whether or not k is larger than 0.
Execute 1120. In step 1110, the projection distribution X_Proj_Current 140-2-3 of the currently input image and the projection distribution X_Proj_Last 140-2 of the image input immediately before are input.
The matching is performed while shifting the position dx between the position dx and -4 (FIG. 10). In step 1120, the position dx with the best matching is set as the movement amount between two temporally consecutive images. Note that this matching is specifically calculated by the formula shown in FIG.

Steps 1130 to 1165 are processes for obtaining the rotation amount dΘ between successive images.

First, in step 1130, Input_Image_Buf
The number of effective pixels in a plurality of designated areas 700 (FIG. 7) of the image stored in the area 312 in 140-2-1, that is, the pixels of the gray portion 802 (FIG. Count by column with the array Rotate_Count_C
Write to urrent140-2-6. And step 11
At 40, the vertical projection distribution for the effective pixels in the plurality of designated areas 700 is calculated, and an array Rotate_Proj_Cu
Write to rrent140-2-5. Note that the value of the normal projection distribution can be calculated by adding the values for each column, but here, the values normalized by the number of effective pixels in each column are further stored in an array.

Next, at step 1160, it is determined whether or not k is larger than 0. If k is larger, steps 1161 to 1168 are executed.

In steps 1161 to 1164, local movement amounts between consecutive images are calculated for the six regions. First, in step 1162, it is determined whether the ratio of the number of effective pixels in the area i is equal to or larger than a certain value, for example, 0.5. If it is equal to or more than the predetermined value, steps 1163 and 1164 are executed. In step 1163, the projection distribution Rotate_Proj_Current 140-2-5 of the currently input image and the projection distribution Rotate_Proj_Last 140-2-7 of the image input immediately before are input.
The matching is performed while shifting the position dxi between.
Then, in step 1164, the position dxi with the best matching is set as the movement amount of the region i between two temporally consecutive images.

Next, the rotation amount dΘ is calculated by the equation shown in FIG. 7 from the movement amount between the most distant regions among the regions where the movement amount can be calculated in step 1165.

In the above processing, the movement amount dx between adjacent images
And the rotation amount dΘ were obtained. Next, in step 1166, the amount of movement and the amount of rotation between the reference image and the current image are calculated. This can be easily calculated by always adding the movement amount and the rotation amount calculated this time to the calculation result up to the previous image. In step 1167, the image of the partial region 302 of the input image is subjected to two-dimensional coordinate conversion based on the amount of rotation and the amount of movement, and an array Panoramic_Image_Buf140-2-2 is obtained.
Write the image to. This makes it possible to generate a panoramic image from a sequence of cross-sectional images involving movement and rotation.

Next, in step 1168, the image of the partial region 303 of the input image is subjected to three-dimensional coordinate conversion based on the amount of rotation and the amount of movement to obtain an array Cubic_Image_Buf 140-2.
The image is written at -12 (FIG. 11). This makes it possible to generate a stereoscopic panoramic image from a sequence of cross-sectional images involving movement and rotation. In this state, the array Cubic_Image_Buf
Since a gap having no pixel value also occurs in the stereoscopic panoramic image of, the gap is compensated for by linear interpolation between surrounding pixels.

Finally, in step 1170, preparations are made for generating the next three-dimensional panoramic image. Transfer the contents of the array X_Proj_Current to X_Proj_Last and rotate the array Rotate_Proj_Current
Is transferred to Rotate_Proj_Last, and the array Rotate_Count
Transfer the contents of _Current to Rotate_Count_Last. Then, a variable k for counting the total number of images used for generating the stereoscopic panoramic image is added by one.

As described above, the stereoscopic panoramic image synthesizing apparatus according to the present invention can simultaneously obtain orthogonal cross-sectional images from the probe, so that the amount of movement and rotation of the probe can be calculated from one cross-sectional image sequence. It is now possible. As a result, it becomes possible to arrange and display the other cross-sectional image sequence in a three-dimensional space using the above-described synthesis parameters,
Conventionally, the position sensor, which has been required, becomes unnecessary, and the effect of reducing the cost of the device is obtained.

FIG. 15 is an example of a system configuration diagram of an ultrasonic image stereoscopic panoramic image synthesizing apparatus according to another embodiment of the present invention. Basically, there is no means for generating an ultrasonic image in the configuration of FIG. 1 but means for taking in an external video signal 1000 and A / D converting it. The other means are the same as in FIG. 1 and will not be described. According to this apparatus configuration, an L-shaped probe is attached to a conventional ultrasonic diagnostic apparatus to scan a body surface, and a video signal displayed on a display of the ultrasonic diagnostic apparatus at that time is input to the apparatus. By itself, a stereoscopic panoramic image of an ultrasonic image can be synthesized. In this embodiment, means for generating an ultrasonic image is not required, and a user who has a conventional ultrasonic diagnostic apparatus can obtain an inexpensive stereoscopic panoramic image synthesizing apparatus.

[0061]

According to the present invention, in addition to a tomographic image for synthesizing a three-dimensional panoramic image, a tomographic image for detecting a synthesis parameter can be simultaneously input, so that a position sensor can be obtained from a tomographic image sequence obtained by a manual probe operation. It is now possible to synthesize stereoscopic panoramic images accurately in real time without using. As a result, an inexpensive and highly operable device can be achieved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a stereoscopic panoramic image synthesizing apparatus according to an embodiment of the present invention.

FIG. 2 is a perspective view showing an application example of the present invention to a human body.

FIG. 3 is an explanatory diagram showing an example of a screen displayed in the embodiment of the present invention.

FIG. 4 is an explanatory diagram of a stereoscopic panorama display screen in a synthesis process.

FIG. 5 is an explanatory diagram of a geometric relationship between images when an L-shaped probe is operated.

FIG. 6 is a view for explaining the principle of movement amount detection in the embodiment of the present invention.

FIG. 7 is a diagram illustrating the principle of rotation amount detection according to the embodiment of the present invention.

FIG. 8 is an explanatory diagram showing a structure of an ultrasonic image.

FIG. 9 is a flowchart of a control program of the stereoscopic panoramic image synthesizing apparatus of the present invention.

FIG. 10 is an explanatory diagram of a structure of control data referred to by a control program of the present invention.

FIG. 11 is an explanatory diagram of the structure of control data referred to by the control program of the present invention.

FIG. 12 is a flowchart illustrating details of a stereoscopic panorama image generation start process according to the present invention.

FIG. 13 is a flowchart illustrating details of a stereoscopic panorama image generation process of the present invention.

FIG. 14 is a perspective view of a T-type probe according to an embodiment of the present invention.

FIG. 15 is a block diagram showing a system configuration diagram of an image synthesizing apparatus according to another embodiment of the present invention.

[Explanation of symbols]

100: stereoscopic panoramic image synthesizing device, 110: ultrasonic oscillator, 120: ultrasonic / imaging unit, 130: CPU, 1
40 memory, 132 data bus, 150 display, 155 video memory, 156 D / A converter, 160 keyboard, 170 pointing device, 180 printer, 190 auxiliary storage device, 2
00: Probe.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 G06F 15/64 B 5C054 15/72 450K F-term (Reference) 4C301 AA02 BB05 BB13 CC01 DD01 DD02 EE10 EE13 EE17 GA01 GB03 GB04 GB09 JB03 JB04 JB28 JB30 JC14 JC20 KK02 KK17 KK22 KK40 LL03 LL08 LL13 LL20 5B047 AA07 AA17 BA03 BB10 BC30 CA23 CB07 CB08 EA01 5B050 AA12 FA03 EA07 5 CA16 CB01 CB13 CB16 CD02 CD03 CD06 CE08 DA07 DA16 DB02 DC19 DC34 5B080 BA02 CA01 FA02 FA08 FA15 5C054 CC00 EA01 EA05 FD02 FE17 FE18 HA12

Claims (14)

[Claims] 1. A probe for acquiring ultrasonic data from two orthogonal directions, an ultrasonic / imaging means for converting the ultrasonic data acquired from the probe into an image, and an ultrasonic / imaging means for acquiring the ultrasonic data. Moving image display means for displaying the obtained image as a moving image, combined parameter detecting means for detecting the amount of movement and rotation between images in a specific direction generated by the moving operation of the probe, and detection by the combined parameter detecting means A stereoscopic panoramic image synthesizing means for arranging an image in the other direction in a three-dimensional space, synthesizing the stereoscopic panoramic image according to the moving amount and the rotating amount, and displaying the stereoscopic panoramic image; Stereoscopic panoramic image synthesis device. 2. A three-dimensional panoramic image synthesizing apparatus for an ultrasonic image according to claim 1, wherein said ultrasonic receiving part of said probe has an L-shape. 3. An apparatus for synthesizing a three-dimensional panoramic image of an ultrasonic image, wherein the ultrasonic receiving part of the probe according to claim 1 is T-shaped. 4. A combination parameter detecting means according to claim 1, wherein said combination parameter detecting means stores a projection distribution between successive images, and moves each of the projection distributions between successive images by a correlation between the projection distributions. A stereoscopic panoramic image synthesizing apparatus for an ultrasonic image, comprising: projection distribution correlation means for obtaining an amount, wherein the amount of movement between the projection distributions is the amount of image movement by a probe moving operation. 5. An ultrasonic wave according to claim 4, wherein the projection distribution stored in said projection distribution storage means is normalized by the number obtained by integrating the result of integrating the values of the pixels constituting the image in the vertical direction. 3D panoramic image synthesis device. 6. An ultrasonic wave generating method according to claim 4, wherein said pixel is a valid pixel when said pixel has a value within a specific range, and only said valid pixel is a calculation target of said projection distribution. 3D panoramic image synthesis device. 7. A projection distribution storage means for setting a plurality of image processing areas and storing a projection distribution between continuous images for the plurality of image processing areas. A projection distribution correlating means for obtaining a moving amount for each of the projection distributions of the corresponding image processing areas between successive images, by a correlation between the projection distributions; A stereoscopic panoramic image synthesizing apparatus for an ultrasonic image, the apparatus comprising: 8. The ultrasonic wave according to claim 7, wherein the projection distribution stored in the projection distribution storage means is normalized by the number obtained by integrating the result of integrating the values of the pixels constituting the image in the projection direction. 3D panoramic image synthesis device. 9. The method according to claim 7, wherein when the pixel has a value in a specific range, the pixel is regarded as an effective pixel.
A stereoscopic panoramic image synthesizing apparatus for an ultrasonic image, wherein only an effective pixel is set as a projection distribution calculation object. 10. An apparatus for synthesizing a stereoscopic panoramic image of an ultrasonic image, according to claim 7, wherein the amount of rotation is estimated by using the amount of movement obtained from a set of image processing areas furthest from each other. 11. The method according to claim 9, wherein only when the ratio of effective pixels occupying the image processing area is larger than a specified value, the projection distribution correlation is determined within the corresponding image processing area. 3D panoramic image synthesizing device for ultrasonic images. 12. A three-dimensional ultrasonic image display device according to claim 1, wherein an image processing area used for creating a three-dimensional panoramic image can be displayed as a graphic as required. Panoramic image synthesis device. 13. A probe for acquiring ultrasonic data from two orthogonal directions, an ultrasonic / imaging means for converting the ultrasonic data acquired from said probe into an image, and an ultrasonic / imaging means for acquiring the ultrasonic data. Moving image display means for displaying the obtained image as a moving image, combined parameter detecting means for detecting the amount of movement and rotation between images in a specific direction generated by the moving operation of the probe, and detection by the combined parameter detecting means According to the movement amount and the rotation amount, the images in the above directions are arranged on a two-dimensional plane, combined as a panoramic image, and displayed. Accordingly, a stereoscopic panoramic image synthesizing means for arranging an image in the other direction in a three-dimensional space, synthesizing the stereoscopic panoramic image, and displaying the stereoscopic panoramic image is provided. A stereoscopic panoramic image synthesis device of the ultrasound image. 14. A means for A / D converting a video signal displaying ultrasonic data from two orthogonal directions, a means for setting a processing area in the video, and a specific means generated by a moving operation of the probe. Combining parameter detecting means for detecting the moving amount and the rotating amount between the images in the directions, and arranging the image in the other direction in a three-dimensional space according to the moving amount and the rotating amount detected by the combining parameter detecting means; A stereoscopic panoramic image synthesizing apparatus for an ultrasonic image, comprising: a stereoscopic panoramic image synthesizing means for synthesizing and displaying a stereoscopic panoramic image.